Skip to main content

gcp_vertex_ai_embeddings

Generates vector embeddings to represent input text, using the Vertex AI API.

# Config fields, showing default values
pipeline:
  processors:
    - label: ""
      gcp_vertex_ai_embeddings:
        project: "" # No default (required)
        credentials_json: "" # No default (optional)
        location: "us-central1"
        model: "" # No default (required)
        task_type: "RETRIEVAL_DOCUMENT"
        text: "" # No default (optional)
        output_dimensions: 0 # No default (optional)

This processor sends text strings to the Vertex AI API, which generates vector embeddings. By default, the processor submits the entire payload of each message as a string, unless you use the text configuration field to customize it.

For more information, see the Vertex AI documentation.

Fields​

project​

GCP project ID to use

Type: string

credentials_json​

An optional field to set google Service Account Credentials json.

Secret

This field contains sensitive information. Use a secret reference rather than a literal value.

Type: string

location​

The location of the model.

Type: string
Default: "us-central1"

model​

The name of the LLM to use. For a full list of models, see the Vertex AI Model Garden.

Type: string

task_type​

The way to optimize embeddings that the model generates for specific use cases.

Type: string
Default: "RETRIEVAL_DOCUMENT"

OptionSummary
CLASSIFICATIONoptimize for being able classify texts according to preset labels
CLUSTERINGoptimize for clustering texts based on their similarities
FACT_VERIFICATIONoptimize for queries that are proving or disproving a fact such as "apples grow underground"
QUESTION_ANSWERINGoptimize for search proper questions such as "Why is the sky blue?"
RETRIEVAL_DOCUMENToptimize for documents that will be searched (also known as a corpus)
RETRIEVAL_QUERYoptimize for queries such as "What is the best fish recipe?" or "best restaurant in Chicago"
SEMANTIC_SIMILARITYoptimize for text similarity

text​

The text you want to compute vector embeddings for. By default, the processor submits the entire payload as a string.

This field supports interpolation functions.

Type: string

output_dimensions​

The maximum length for the output embedding size. If set, the output embeddings will be truncated to this size.

Type: int