VertexAIGeminiGenerator

VertexAIGeminiGenerator enables text generation using Google Gemini models.

warning

Deprecation Notice

This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.

We recommend switching to the new GoogleGenAIChatGenerator integration instead.

Most common position in a pipeline	After a PromptBuilder
Mandatory run variables	“parts”: A variadic list containing a mix of images, audio, video, and text to prompt Gemini
Output variables	“replies”: A list of strings or dictionaries with all the replies generated by the model
API reference	Google Vertex
GitHub link	https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex

VertexAIGeminiGenerator supports gemini-1.5-pro and gemini-1.5-flash/ gemini-2.0-flash models. Note that Google recommends upgrading from gemini-1.5-pro to gemini-2.0-flash.

For details on available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models.

note

To explore the full capabilities of Gemini check out this article and the related Colab notebook.

Parameters Overview

VertexAIGeminiGenerator uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the official documentation.

Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.

You can find your project ID in the GCP resource manager or locally by running gcloud projects list in your terminal. For more info on the gcloud CLI, see its official documentation.

Streaming

This Generator supports streaming the tokens from the LLM directly in output. To do so, pass a function to the streaming_callback init parameter.

Usage

You should install google-vertex-haystack package to use the VertexAIGeminiGenerator:

shell

pip install google-vertex-haystack

On its own

Basic usage:

python

from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator

gemini = VertexAIGeminiGenerator()
result = gemini.run(parts = ["What is the most interesting thing you know?"])
for answer in result["replies"]:
    print(answer)

Advanced usage, multi-modal prompting:

python

import requests
from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator

URLS = [
    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot1.jpg",
    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot2.jpg",
    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot3.jpg",
    "https://raw.githubusercontent.com/silvanocerza/robots/main/robot4.jpg"
]
images = [
    ByteStream(data=requests.get(url).content, mime_type="image/jpeg")
    for url in URLS
]

gemini = VertexAIGeminiGenerator()
result = gemini.run(parts = ["What can you tell me about this robots?", *images])
for answer in result["replies"]:
    print(answer)

In a pipeline

In a RAG pipeline:

python

from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.builders import PromptBuilder
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator

docstore = InMemoryDocumentStore()
docstore.write_documents([Document(content="Rome is the capital of Italy"), Document(content="Paris is the capital of France")])

query = "What is the capital of France?"

template = """
Given the following information, answer the question.

Context:
{% for document in documents %}
    {{ document.content }}
{% endfor %}

Question: {{ query }}?
"""
pipe = Pipeline()

pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("gemini", VertexAIGeminiGenerator())
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "gemini")

res=pipe.run({
    "prompt_builder": {
        "query": query
    },
    "retriever": {
        "query": query
    }
})

print(res)

Additional References

:cook: Cookbook: Function Calling and Multimodal QA with Gemini