Skip to main content

aws_bedrock_chat

Generates responses to messages in a chat conversation, using the AWS Bedrock API.

This processor sends a prompt to a chosen large language model and returns the generated text response through the AWS Bedrock API. By default it submits the entire message payload as the prompt.

# Config fields, showing default values
processor:
  label: ""
  aws_bedrock_chat:
    model: "" # No default (required)
    prompt: ""
    system_prompt: ""
    region: ""
    endpoint: ""
    max_tokens: 0
    temperature: 0
    top_p: 0
    stop: []
    credentials:
      profile: ""
      id: ""
      secret: ""
      token: ""
      from_ec2_role: false
      role: ""
      role_external_id: ""
    tcp:
      connect_timeout: 0s
      keep_alive:
        idle: 15s
        interval: 15s
        count: 9
      tcp_user_timeout: 0s

Examples​

Send each message payload as a prompt to a Bedrock model and replace it with the generated response.

processor:
  aws_bedrock_chat:
    model: "anthropic.claude-3-5-sonnet-20240620-v1:0"
    region: us-east-1

Fields​

model​

The model ID to use. For a full list, see the AWS Bedrock documentation.

Type: string

prompt​

The prompt you want to generate a response for. By default, the processor submits the entire payload as a string.

Type: string
Default: ""

system_prompt​

The system prompt to submit to the AWS Bedrock LLM.

Type: string
Default: ""

region​

The AWS region to target.

Type: string
Default: ""

endpoint​

A custom endpoint URL for AWS API requests. Use this to connect to AWS-compatible services or local testing environments instead of standard AWS endpoints.

Type: string
Default: ""

max_tokens​

The maximum number of tokens to allow in the generated response.

Type: int
Default: 0

temperature​

The likelihood of the model selecting higher-probability options while generating a response. Lower values favor higher-probability options; higher values favor lower-probability options.

Type: float
Default: 0

top_p​

The percentage of most-likely candidates that the model considers for the next token. For example, if you choose a value of 0.8, the model selects from the top 80% of the probability distribution.

Type: float
Default: 0

stop​

A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.

Type: array
Default: []

credentials​

Optional manual configuration of AWS credentials to use. More information can be found in this document.

Type: object

credentials.profile​

A profile from ~/.aws/credentials to use.

Type: string
Default: ""

credentials.id​

The ID of credentials to use.

Type: string
Default: ""

credentials.secret​

The secret for the credentials being used. :::warning Secret This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info. :::

Type: string
Default: ""

credentials.token​

The token for the credentials being used, required when using short term credentials.

Type: string
Default: ""

credentials.from_ec2_role​

Use the credentials of a host EC2 machine configured to assume an IAM role associated with the instance.

Type: bool
Default: false

credentials.role​

A role ARN to assume.

Type: string
Default: ""

credentials.role_external_id​

An external ID to provide when assuming a role.

Type: string
Default: ""

tcp​

Configure TCP socket-level settings to optimize network performance and reliability for high-latency, long-lived, or unstable networks.

Type: object

tcp.connect_timeout​

Maximum amount of time a dial will wait for a connect to complete. Zero disables.

Type: string
Default: 0s

tcp.keep_alive​

TCP keep-alive probe configuration.

Type: object

tcp.keep_alive.count​

Maximum unanswered keep-alive probes before dropping the connection. Zero defaults to 9.

Type: int
Default: 9

tcp.keep_alive.idle​

Duration the connection must be idle before sending the first keep-alive probe. Zero defaults to 15s. Negative values disable keep-alive probes.

Type: string
Default: 15s

tcp.keep_alive.interval​

Duration between keep-alive probes. Zero defaults to 15s.

Type: string
Default: 15s

tcp.tcp_user_timeout​

Maximum time to wait for acknowledgment of transmitted data before killing the connection. Linux-only (kernel 2.6.37+), ignored on other platforms. Zero disables.

Type: string
Default: 0s