Skip to main content

schema_registry

Publishes schemas to a schema registry.

Sends schema definitions to a Schema Registry service over its REST API, using the subject and compatibility level you configure. It supports basic, OAuth 1.0, and JWT authentication as well as TLS, and can backfill missing schema references and previous versions or translate schema IDs when migrating schemas between registries.

# Config fields, showing default values
output:
  label: ""
  schema_registry:
    url: "" # No default (required)
    subject: "" # No default (required)
    subject_compatibility_level: ""
    backfill_dependencies: true
    translate_ids: false
    normalize: true
    remove_metadata: true
    remove_rule_set: true
    input_resource: schema_registry_input
    max_in_flight: 64
    tls:
      enabled: false
      skip_cert_verify: false
      enable_renegotiation: false
      root_cas: ""
      root_cas_file: ""
      client_certs: []
    basic_auth:
      enabled: false
      username: ""
      password: ""
    oauth:
      enabled: false
      consumer_key: ""
      consumer_secret: ""
      access_token: ""
      access_token_secret: ""
    jwt:
      enabled: false
      private_key_file: ""
      signing_method: ""
      claims: {}
      headers: {}

Examples​

Publish schemas to a registry, reading source schemas from a schema_registry input so references and previous versions can be backfilled.

output:
  schema_registry:
    url: http://localhost:8081
    subject: ${! @schema_registry_subject }
    input_resource: schema_registry_input

Fields​

url​

The base URL of the schema registry service.

Type: string

subject​

The subject name to publish schemas under. This field supports interpolation functions.

Type: string

subject_compatibility_level​

The compatibility level to set for the subject. Can be one of BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE or NONE. This field supports interpolation functions.

Type: string

backfill_dependencies​

Whether to backfill missing schema references and previous schema versions. When set to true you must also configure a schema_registry input to read the source schemas from.

Type: bool
Default: true

translate_ids​

When set to true, automatically translates the schema ID in each message to match the corresponding schema in the destination schema registry.

Type: bool
Default: false

normalize​

Whether to normalize schemas before publishing them.

Type: bool
Default: true

remove_metadata​

Whether to remove metadata fields from the schema output, producing leaner definitions when metadata is not required.

Type: bool
Default: true

remove_rule_set​

Whether to remove rule set definitions from the schema output, simplifying schemas when rule sets are not required.

Type: bool
Default: true

input_resource​

The label of the schema_registry input from which to read source schemas.

Type: string
Default: "schema_registry_input"

max_in_flight​

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: int
Default: 64

tls​

Custom TLS settings used to secure network connections, including standard TLS and mTLS authentication.

Type: object

tls.enabled​

Whether custom TLS settings are enabled.

Type: bool
Default: false

tls.skip_cert_verify​

Whether to skip server side certificate verification.

Type: bool
Default: false

tls.enable_renegotiation​

Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you're seeing the error message local error: tls: no renegotiation.

Type: bool
Default: false

tls.root_cas​

An optional root certificate authority to use. This is a string, representing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate. :::warning Secret This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info. :::

Type: string
Default: ""

tls.root_cas_file​

An optional path of a root certificate authority file to use. This is a file, often with a .pem extension, containing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate.

Type: string
Default: ""

tls.client_certs​

A list of client certificates to use. For each certificate either the fields cert and key, or cert_file and key_file should be specified, but not both.

Type: array
Default: []

tls.client_certs.cert​

A plain text certificate to use.

Type: string
Default: ""

tls.client_certs.cert_file​

The path of a certificate to use.

Type: string
Default: ""

tls.client_certs.key​

A plain text certificate key to use. :::warning Secret This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info. :::

Type: string
Default: ""

tls.client_certs.key_file​

The path of a certificate key to use.

Type: string
Default: ""

tls.client_certs.password​

A plain text password for when the private key is password encrypted in PKCS#1 or PKCS#8 format. The obsolete pbeWithMD5AndDES-CBC algorithm is not supported for the PKCS#8 format. :::warning Secret This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info. :::

Type: string
Default: ""

basic_auth​

Allows you to configure basic authentication for requests from this component to your schema registry.

Type: object

basic_auth.enabled​

Whether to use basic authentication in requests.

Type: bool
Default: false

basic_auth.username​

The username of the account credentials to authenticate as. This field is used together with password.

Type: string
Default: ""

basic_auth.password​

The password to use for authentication. This field is used together with username. :::warning Secret This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info. :::

Type: string
Default: ""

oauth​

Allows you to configure OAuth version 1.0 to give this component authorized access to your schema registry.

Type: object

oauth.enabled​

Whether to use OAuth version 1 in requests.

Type: bool
Default: false

oauth.consumer_key​

The value used to identify this component or client to your schema registry.

Type: string
Default: ""

oauth.consumer_secret​

The secret that establishes ownership of the consumer key. :::warning Secret This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info. :::

Type: string
Default: ""

oauth.access_token​

The value that this component can use to gain access to the schema registry.

Type: string
Default: ""

oauth.access_token_secret​

The secret that establishes ownership of the oauth.access_token. :::warning Secret This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info. :::

Type: string
Default: ""

jwt​

Allows you to configure JSON Web Token (JWT) authentication for secure data transmission. This feature is in beta.

Type: object

jwt.enabled​

Whether to use JWT authentication in requests.

Type: bool
Default: false

jwt.private_key_file​

A PEM-encoded file containing a private key formatted using the PKCS1 or PKCS8 standards.

Type: string
Default: ""

jwt.signing_method​

The method used to sign the token, such as RS256, RS384, RS512 or EdDSA.

Type: string
Default: ""

jwt.claims​

The values used to pass the identity of the authenticated entity to the service provider.

Type: object
Default: {}

jwt.headers​

The key/value pairs that identify the type of token and the signing algorithm used.

Type: object
Default: {}