Enable google search for gemini-1.5-pro and gemini-2.0-flash-exp #5063
+282
−25
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This is a more a PoC than a PR, it's quite cool to have google search natively supported by the model as in google's aistudio.
It will probably be rejected for various reasons but maybe some people are interested in the feature.
In the settings of the Google Endpoint there is a "Enable Search" switch ( localized in all supported languages ).
When the switch is activated the google_search tool is added to the model ( it is not the same tool for 1.5 and for 2.0 ), and it enables the model to use google search, and there is an added snippet generated by google added in the response.
The snippet seems more or less mandatory according to google gemini doc: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/ground-gemini , https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/grounding-search-suggestions
Note that the patch manages only the GenAI api, not the vertex-ai api ( for the time being I can't test vertex ai and it seems gemini-2.0 is not yet supported on vertex-ai. if you are in Europe your data sent to GenAI is not used for training ).
Change Type
Please delete any irrelevant options.
Testing
Check the "Enable Search" setting on a google endpoint, ask about the weather in whatever place in the world and check:
I have tested with models gemini-1.5-pro-002 and gemini-2.0-flash-exp.
Note that all google models do not support grounding. One way to check if a given model is supported is to check in https://aistudio.google.com if the model supports grounding. The gemini-exp-1206 does not support it.
Test Configuration:
Checklist
Please delete any irrelevant options.
My code adheres to this project's style guidelines
I have performed a self-review of my own code
I have commented in any complex areas of my code
I have made pertinent documentation changes : no, but could do them if this PR is considered further
My changes do not introduce new warnings