sql processor

Runs an arbitrary SQL query against a database and (optionally) returns the result as an array of objects, one for each row returned.

Deprecated

This component is deprecated and may be removed in a future release. Prefer a supported alternative where available.

Common

# Common config fields, showing default values
pipeline:
  processors:
    - label: ""
      sql:
        driver: "" # No default (required)
        data_source_name: "" # No default (required)
        query: "" # No default (required)
        args_mapping: "" # No default (optional)
        result_codec: "none"

Advanced

# All config fields, showing default values
pipeline:
  processors:
    - label: ""
      sql:
        driver: "" # No default (required)
        data_source_name: "" # No default (required)
        query: "" # No default (required)
        unsafe_dynamic_query: false
        args_mapping: "" # No default (optional)
        result_codec: "none"

If the query fails to execute then the message will remain unchanged and the error can be caught using error handling methods.

Alternatives

For basic inserts or select queries use either the sql_insert or the sql_select processor. For more complex queries use the sql_raw processor.

Fields

driver

A database driver to use.

Type: string

Options: mysql, postgres, pgx, clickhouse, mssql, sqlite, oracle, snowflake, trino, gocosmos, spanner, databricks

data_source_name

Data source name.

Type: string

query

The query to execute. The style of placeholder to use depends on the driver, some drivers require question marks (?) whereas others expect incrementing dollar signs ($1, $2, and so on) or colons (:1, :2 and so on). The style to use is outlined in this table:

Driver	Placeholder Style
clickhouse	Dollar sign
mysql	Question mark
postgres	Dollar sign
pgx	Dollar sign
mssql	Question mark
sqlite	Question mark
oracle	Colon
snowflake	Question mark
trino	Question mark
gocosmos	Colon

Type: string

unsafe_dynamic_query

Whether to enable interpolation functions in the query. Great care should be made to ensure your queries are defended against injection attacks.

Type: bool
Default: false

args_mapping

An optional Bloblang mapping which should evaluate to an array of values matching in size to the number of placeholder arguments in the field query.

Type: string

result_codec

Result codec.

Type: string
Default: "none"