Skip to main content

mongodb output

Inserts items into a MongoDB collection.

# Common config fields, showing default values
output:
  label: ""
  mongodb:
    url: "" # No default (required)
    database: "" # No default (required)
    username: ""
    password: ""
    collection: "" # No default (required)
    operation: "update-one"
    write_concern:
      w: "majority"
      j: false
      w_timeout: ""
    document_map: ""
    filter_map: ""
    hint_map: ""
    upsert: false
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""

Performance​

This output benefits from sending multiple messages in flight in parallel for improved performance. You can tune the max number of in flight messages (or message batches) with the field max_in_flight.

This output benefits from sending messages as a batch for improved performance. Batches can be formed at both the input and output level. You can find out more in this doc.

Fields​

url​

The URL of the target MongoDB server.

Type: string

database​

The name of the target MongoDB database.

Type: string

username​

The username to connect to the database.

Type: string
Default: ""

password​

The password to connect to the database.

Secret

This field contains sensitive information. Use a secret reference rather than a literal value.

Type: string
Default: ""

app_name​

The client application name.

Type: string
Default: "benthos"

aws​

AWS IAM authentication using the MONGODB-AWS mechanism, for example against MongoDB Atlas. When enabled, IAM credentials are used instead of a static username and password. Role-derived session credentials are resolved when the component connects and are re-resolved whenever it reconnects. The mongodb processor and cache establish their client once at creation and cannot refresh expiring session credentials, so role, roles and session tokens are rejected for those components; use the ambient credential chain or long-lived access keys with them. For long-running pipelines, prefer the ambient credential chain (leave keys and roles unset), which the driver refreshes automatically.

Type: object

aws.enabled​

Enable AWS IAM authentication using the driver-native MONGODB-AWS mechanism. The MongoDB Atlas database user must be created with the AWS IAM authentication type, and connections require TLS. When no static credentials or roles are configured, the ambient AWS credential chain (environment variables, EC2 instance profile, EKS pod role) is used and expiring credentials are refreshed automatically.

Type: bool
Default: false

aws.region​

The AWS region used when assuming roles (for STS calls). Only used when role or roles are configured; the ambient and static-key paths ignore it. If no region is specified then the environment default is used.

Type: string

aws.session_duration​

The duration of the STS session requested when assuming roles. AWS requires at least 15 minutes and caps sessions created through role chaining at one hour. Only used when role or roles are configured. When using mongodb_cdc with role assumption, credentials are freshly resolved after the initial snapshot completes, so the streaming phase starts with a full session. The snapshot itself must still complete within a single session duration: snapshot progress is not checkpointed, so a credential expiry mid-snapshot restarts the snapshot from scratch after reconnecting. Once the snapshot completes and is fully acknowledged, its position is checkpointed, so later restarts resume the stream without re-running the snapshot. For very large snapshots prefer the ambient credential chain.

Type: string
Default: "1h"

aws.id​

The ID of credentials to use.

Type: string

aws.secret​

The secret for the credentials being used.

Secret

This field contains sensitive information. Use a secret reference rather than a literal value.

Type: string

aws.token​

The token for the credentials being used, required when using short term credentials.

Secret

This field contains sensitive information. Use a secret reference rather than a literal value.

Type: string

aws.role​

Optional AWS IAM role ARN to assume for authentication. Cannot be combined with roles; use the roles array instead when chaining multiple roles.

Type: string

aws.role_external_id​

Optional external ID for the role assumption. Only used with the role field, which cannot be combined with roles.

Type: string

aws.roles​

Optional array of AWS IAM roles to assume for authentication. Roles can be assumed in sequence, enabling chaining for purposes such as cross-account access. Each role can optionally specify an external ID. Cannot be combined with role.

Type: array of object

aws.roles[].role​

AWS IAM role ARN to assume.

Type: string
Default: ""

aws.roles[].role_external_id​

Optional external ID for the role assumption.

Type: string
Default: ""

collection​

The name of the target collection.

This field supports interpolation functions.

Type: string

operation​

The mongodb operation to perform.

Type: string
Default: "update-one"

Options: insert-one, delete-one, delete-many, replace-one, update-one

write_concern​

The write concern settings for the mongo connection.

Type: object

write_concern.w​

W requests acknowledgement that write operations propagate to the specified number of mongodb instances. Can be the string "majority" to wait for a calculated majority of nodes to acknowledge the write operation, or an integer value specifying an minimum number of nodes to acknowledge the operation, or a string specifying the name of a custom write concern configured in the cluster.

Type: string
Default: "majority"

write_concern.j​

J requests acknowledgement from MongoDB that write operations are written to the journal.

Type: bool
Default: false

write_concern.w_timeout​

The write concern timeout.

Type: string
Default: ""

document_map​

A bloblang map representing a document to store within MongoDB, expressed as extended JSON in canonical form. The document map is required for the operations insert-one, replace-one, update-one and aggregate.

Type: string
Default: ""

filter_map​

A bloblang map representing a filter for a MongoDB command, expressed as extended JSON in canonical form. The filter map is required for all operations except insert-one. It is used to find the document(s) for the operation. For example in a delete-one case, the filter map should have the fields required to locate the document to delete.

Type: string
Default: ""

hint_map​

A bloblang map representing the hint for the MongoDB command, expressed as extended JSON in canonical form. This map is optional and is used with all operations except insert-one. It is used to improve performance of finding the documents in the mongodb.

Type: string
Default: ""

upsert​

The upsert setting is optional and only applies for update-one and replace-one operations. If the filter specified in filter_map matches, the document is updated or replaced accordingly, otherwise it is created.

Type: bool
Default: false

max_in_flight​

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: int
Default: 64

batching​

Allows you to configure a batching policy.

Type: object

batching.count​

A number of messages at which the batch should be flushed. If 0 disables count based batching.

Type: int
Default: 0

batching.byte_size​

An amount of bytes at which the batch should be flushed. If 0 disables size based batching.

Type: int
Default: 0

batching.period​

A period in which an incomplete batch should be flushed regardless of its size.

Type: string
Default: ""

batching.check​

A Bloblang query that should return a boolean value indicating whether a message should end a batch.

Type: string
Default: ""

batching.processors​

A list of processors to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.

Type: array of processor