Skip to main content

gcp_vertex_ai_chat

Generates responses to messages in a chat conversation, using the Vertex AI API.

# Common config fields, showing default values
pipeline:
  processors:
    - label: ""
      gcp_vertex_ai_chat:
        project: "" # No default (required)
        credentials_json: "" # No default (optional)
        location: "" # No default (required)
        model: "" # No default (required)
        prompt: "" # No default (optional)
        history: "" # No default (optional)
        attachment: "" # No default (optional)
        temperature: 0 # No default (optional)
        max_tokens: 0 # No default (optional)
        response_format: "text"
        tools: []

This processor sends prompts to your chosen large language model (LLM) and generates text from the responses, using the Vertex AI API.

For more information, see the Vertex AI documentation.

Examples​

Use processors as tool calls​

This example allows gemini to execute a subpipeline as a tool call to get more data.

input:
  generate:
    count: 1
    mapping: |
      root = "What is the weather like in Chicago?"
pipeline:
  processors:
    - gcp_vertex_ai_chat:
        model: gemini-2.5-flash-preview-05-20
        project: my-project
        location: us-central1
        prompt: "${!content().string()}"
        tools:
          - name: GetWeather
            description: "Retrieve the weather for a specific city"
            parameters:
              required: ["city"]
              properties:
                city:
                  type: string
                  description: the city to lookup the weather for
            processors:
              - http:
                  verb: GET
                  url: 'https://wttr.in/${!this.city}?T'
                  headers:
                    # Spoof curl user-agent to get a plaintext text
                    User-Agent: curl/8.11.1
output:
  stdout: {}

Fields​

project​

GCP project ID to use

Type: string

credentials_json​

An optional field to set google Service Account Credentials json.

Secret

This field contains sensitive information. Use a secret reference rather than a literal value.

Type: string

location​

The location of the model if using a fined tune model. For base models this can be omitted

Type: string

model​

The name of the LLM to use. For a full list of models, see the Vertex AI Model Garden.

Type: string

prompt​

The prompt you want to generate a response for. By default, the processor submits the entire payload as a string.

This field supports interpolation functions.

Type: string

system_prompt​

The system prompt to submit to the Vertex AI LLM.

This field supports interpolation functions.

Type: string

history​

Historical messages to include in the chat request. The result of the bloblang query should be an array of objects of the form of [{"role": "", "content":""}], where role is "user" or "model".

Type: string

attachment​

Additional data like an image to send with the prompt to the model. The result of the mapping must be a byte array, and the content type is automatically detected.

Type: string

temperature​

Controls the randomness of predications.

Type: float

max_tokens​

The maximum number of output tokens to generate per message.

Type: int

response_format​

The response format of generated type, the model must also be prompted to output the appropriate response type.

Type: string
Default: "text"

Options: text, json

top_p​

If specified, nucleus sampling will be used.

Type: float

top_k​

If specified top-k sampling will be used.

Type: float

stop​

Stop sequences to when the model will stop generating further tokens.

Type: array of string

presence_penalty​

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

Type: float

frequency_penalty​

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

Type: float

max_tool_calls​

The maximum number of sequential tool calls.

Type: int
Default: 10

tools​

The tools to allow the LLM to invoke. This allows building subpipelines that the LLM can choose to invoke to execute agentic-like actions.

Type: array of object
Default: []

tools[].name​

The name of this tool.

Type: string

tools[].description​

A description of this tool, the LLM uses this to decide if the tool should be used.

Type: string

tools[].parameters​

The parameters the LLM needs to provide to invoke this tool.

Type: object

tools[].parameters.required​

The required parameters for this pipeline.

Type: array of string
Default: []

tools[].parameters.properties​

The properties for the processor's input data

Type: map of object

tools[].parameters.properties.<name>.type​

The type of this parameter.

Type: string

tools[].parameters.properties.<name>.description​

A description of this parameter.

Type: string

tools[].parameters.properties.<name>.enum​

Specifies that this parameter is an enum and only these specific values should be used.

Type: array of string
Default: []

tools[].processors​

The pipeline to execute when the LLM uses this tool.

Type: array of processor

location is required for every model​

The field description above suggests location can be omitted for base models. It cannot: the field is required for fine-tuned and base models alike, and a config without it is rejected at submission with

Missing required field 'location' in gcp_vertex_ai_chat component

Set it to the region serving your model, for example us-central1.