Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable google search for gemini-1.5-pro and gemini-2.0-flash-exp #5063

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

owengo
Copy link

@owengo owengo commented Dec 20, 2024

Summary

This is a more a PoC than a PR, it's quite cool to have google search natively supported by the model as in google's aistudio.

It will probably be rejected for various reasons but maybe some people are interested in the feature.
In the settings of the Google Endpoint there is a "Enable Search" switch ( localized in all supported languages ).
When the switch is activated the google_search tool is added to the model ( it is not the same tool for 1.5 and for 2.0 ), and it enables the model to use google search, and there is an added snippet generated by google added in the response.
The snippet seems more or less mandatory according to google gemini doc: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/ground-gemini , https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/grounding-search-suggestions

Note that the patch manages only the GenAI api, not the vertex-ai api ( for the time being I can't test vertex ai and it seems gemini-2.0 is not yet supported on vertex-ai. if you are in Europe your data sent to GenAI is not used for training ).

Change Type

Please delete any irrelevant options.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Testing

Check the "Enable Search" setting on a google endpoint, ask about the weather in whatever place in the world and check:

  • up-to-date results
  • an added snippet with a link to google

I have tested with models gemini-1.5-pro-002 and gemini-2.0-flash-exp.
Note that all google models do not support grounding. One way to check if a given model is supported is to check in https://aistudio.google.com if the model supports grounding. The gemini-exp-1206 does not support it.

Test Configuration:

Checklist

Please delete any irrelevant options.

  • My code adheres to this project's style guidelines

  • I have performed a self-review of my own code

  • I have commented in any complex areas of my code

  • I have made pertinent documentation changes : no, but could do them if this PR is considered further

  • My changes do not introduce new warnings

@danny-avila
Copy link
Owner

interesting! I had ideas for how to integrate this but I'm open to trying this out.

I'm about to cut a release so it would be slated for next release anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants