VertexAITextGenerator

This component enables text generation using Google Vertex AI generative models.

Mandatory run variables	“prompt”: A string containing the prompt for the model
Output variables	“replies”: A list of strings containing answers generated by the model ”safety_attributes”: A dictionary containing scores for safety attributes ”citations”: A list of dictionaries containing grounding citations
API reference	Google Vertex
GitHub link	https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex

VertexAITextGenerator supports text-bison, text-unicorn and text-bison-32k models.

Parameters Overview

VertexAITextGenerator uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the official documentation.

Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.

You can find your project ID in the GCP resource manager or locally by running gcloud projects list in your terminal. For more info on the gcloud CLI, see its official documentation.

Usage

You need to install google-vertex-haystack package to use the VertexAITextGenerator:

python

pip install google-vertex-haystack

On its own

Basic usage:

python

from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator

generator = VertexAITextGenerator()
res = generator.run("Tell me a good interview question for a software engineer.")

print(res["replies"][0])

You can also set other parameters like the number of answers generated, temperature to control the randomness, and stop sequences to stop generation. For a full list of possible parameters, see the documentation of TextGenerationModel.predict().

python

from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator

generator = VertexAITextGenerator(
    candidate_count=3,
    temperature=0.2,
    stop_sequences=["example", "Example"],
)
res = generator.run("Tell me a good interview question for a software engineer.")

for answer in res["replies"]:
    print(answer)
		print("-----")