Skip to main content

signature

Signs each message's payload with the edge node's Ed25519 identity keypair.

# Config fields, showing default values
pipeline:
  processors:
    - label: ""
      signature:
        metadata_signature_key: "expanso_signature"
        metadata_keyid_key: "expanso_keyid"
        metadata_algorithm_key: "expanso_signature_alg"
        on_signing_error: "ignore"
        target: "meta"
        body_format: "nested"
        body_key: "_signature"
        _execution_id: ""

Computes a deterministic Ed25519 signature over the canonical bytes of each message and attaches the signature, keyId, and algorithm to the destination selected by target. What gets signed is always the payload as received: with target "meta" the payload is left untouched, while "body" and "both" add the configured signature fields to the JSON body.

The processor's position in the chain is the signature's meaning. Place it first to sign raw ingest; place it last to sign the fully enriched payload that hits the output. The signature, keyId, and algorithm are written to message metadata, into the JSON body, or to both, as target selects, so a verifier can recompute the canonical bytes independently and check the signature against them. When the signature travels in the body, a verifier removes those fields before re-canonicalising, so the bytes it checks are the ones that were signed.

Canonicalisation rule: if the payload parses as JSON, it is round-tripped through map[string]any with json.Decoder.UseNumber() so map keys are sorted and int64 precision is preserved. Non-JSON payloads are signed as the raw bytes received.

Fields​

metadata_signature_key​

Metadata key under which the base64-encoded signature is written.

Type: string
Default: "expanso_signature"

metadata_keyid_key​

Metadata key under which the keyId (node:<id>#sha256:<thumbprint>) is written.

Type: string
Default: "expanso_keyid"

metadata_algorithm_key​

Metadata key under which the algorithm name (always Ed25519 today) is written.

Type: string
Default: "expanso_signature_alg"

on_signing_error​

What to do when the signer returns an error. "ignore" (default) logs the failure and passes the message through WITHOUT signature metadata — the pipeline keeps running and the consumer sees an unsigned message. "fail" marks the message as failed; the pipeline's configured error handler picks it up (or the message is dropped if none is configured). Use "fail" when downstream consumers MUST receive only signed messages.

Type: string
Default: "ignore"

target​

Where to attach the signature. "meta" (default) writes three keys onto message metadata so outputs that propagate metadata as headers (HTTP, Kafka, NATS) carry the attestation. "body" splices the signature INSIDE the JSON message body so outputs that don't propagate metadata (file sinks, generic webhooks) still carry it; the body MUST be a single JSON object. "both" writes to both metadata and body — useful when the pipeline fans out to mixed outputs.

Type: string
Default: "meta"

body_format​

When target is "body" or "both", controls the layout of the signature inside the body. "nested" (default) places one key at the body root (see body_key) whose value is an object holding the JOSE-style triple {alg, kid, sig}. "flat" splices three keys at the body root, reusing the same names as the metadata fields above. Nested is the recommended default because it sidesteps collision risk with user payload fields. Ignored when target is "meta".

Type: string
Default: "nested"

body_key​

When target is "body" or "both" and body_format is "nested", the body-root key under which the signature object lives. Defaults to "_signature" (underscore prefix marks it as a framework-added field). Ignored for body_format "flat" and for target "meta".

Type: string
Default: "_signature"

_execution_id​

Internal — auto-populated by the Expanso runtime at pipeline build time. Do not set manually; user-supplied values are rejected by the submission validator.

Type: string
Default: ""

When to Use​

There is no built-in verifier processor — verification is the consumer's responsibility, and the canonicalisation rules a verifier must follow are described in the Verification section below.

Use the signature processor when you need to:

  • Attest data integrity — give downstream consumers cryptographic proof that a message has not been altered between the signing node and the sink.
  • Bind data to a node identity — every signature is tied to the producing node's identity via a keyId of the form node:<nodeID>#sha256:<thumbprint>.
  • Sign HTTP requests via metadata propagation — combined with an output that exposes metadata.include_patterns (e.g. http_client, kafka_franz, nats), the signature rides on transport headers without modifying the body.
  • Persist self-contained signed records — when writing to file or object storage, where headers don't survive, splice the signature into the body so the record verifies without any transport metadata. The verifier still obtains the node's public key separately (see Verification).

Don't use this if:

  • You need a keyed MAC (HMAC) or RSA/ECDSA signature — the processor is Ed25519-only and signs with the node's identity key. Algorithm negotiation is not supported.
  • You need replay protection — see Limitations.
  • You need to sign with a key supplied per-message — keys are bound to the node identity at service start.

Targets and body layouts​

target: meta (default)​

Three configurable keys are written to message metadata; the payload bytes are not touched. Outputs that propagate metadata as transport headers (HTTP, Kafka, NATS) carry the attestation alongside the data.

# Default behavior — propagate signature as HTTP headers
pipeline:
  processors:
    - signature: {}

output:
  http_client:
    url: https://example.com/ingest
    verb: POST
    metadata:
      include_patterns:
        - "^expanso_"   # passes the three signature keys as headers

target: body​

The signature is spliced into the JSON body. The body must be a single JSON object at its root — arrays, scalars, plaintext, and trailing content are rejected through on_signing_error. No metadata is stamped, so the message is self-contained: useful for file sinks, object storage, or any transport that doesn't carry headers.

# Nested layout (default)
pipeline:
  processors:
    - signature:
        target: body

Wire body for an input of {"temp_c":21.5,"sensor_id":42,"marker":"hello"}:

{
  "_signature": {
    "alg": "Ed25519",
    "kid": "node:a1b2c3d4-...#sha256:nVaBCZ...quU",
    "sig": "Vb0/zoMrfNlGhAaUWfvSbctp...mzRm1FnBw=="
  },
  "marker": "hello",
  "sensor_id": 42,
  "temp_c": 21.5
}

Switch to a flat layout when downstream tooling expects the metadata-style keys at the root:

pipeline:
  processors:
    - signature:
        target: body
        body_format: flat
{
  "expanso_signature": "Vb0/zoMrfNlG...",
  "expanso_keyid": "node:a1b2c3d4-...#sha256:nVaBCZ...quU",
  "expanso_signature_alg": "Ed25519",
  "marker": "hello",
  "sensor_id": 42,
  "temp_c": 21.5
}

target: both​

Writes to both metadata and body simultaneously. Useful when a pipeline fans out to mixed outputs — some that propagate metadata, some that don't. The metadata write runs first and cannot fail; the body splice runs second and is governed by on_signing_error. In the default ignore mode, a body-splice failure leaves the metadata write in place (documented partial success).

Collision handling​

The processor refuses to overwrite user-supplied fields. If the body already contains the configured body_key (nested) or any of the three expanso_* keys (flat), the splice fails and is routed through on_signing_error. Rename the keys via body_key or metadata_*_key if your payload already uses those names — but remember to configure your verifier identically.

Pipeline position is the signature's meaning​

The signature covers exactly the bytes the processor sees when the message arrives at it. Two placements give two different attestations:

# (A) Sign raw ingest — signature attests to what the producer emitted
pipeline:
  processors:
    - signature: {}
    - mapping: |
        root.enriched_at = now()
# (B) Sign enriched output — signature covers the lineage stamp as well
pipeline:
  processors:
    - metadata:
        include: [core, node, pipeline]
        target: body
        format: nested
        body_key: lineage
    - signature: {}

Both are correct; they answer different questions. Decide explicitly which one you want before deploying — once a downstream consumer trusts a signature, changing the placement silently changes what it means.

Canonicalisation rule​

The bytes that are signed are not necessarily the bytes on the wire. The processor canonicalises each payload so verifiers don't depend on accidental ordering:

  • JSON payloads (the whole input parses as one self-contained JSON value: object, array, string, number, boolean, or null) are re-encoded into a canonical form. The complete rule set:
    • Object keys are sorted by byte value, at every level of nesting. Array order is left alone.
    • Number literals are reproduced exactly as they arrived: 1.50 stays 1.50, 1e3 stays 1e3, and a 20-digit integer keeps every digit. Nothing is reformatted or rounded, so no precision is lost.
    • No whitespace between tokens.
    • <, >, and & are escaped as \u003c, \u003e, and \u0026.
    • " and \ are escaped as \" and \\. Forward slashes are not escaped.
    • Control characters U+0000 through U+001F are escaped, using \t, \n, and \r where they apply and \u00xx otherwise. U+007F is not escaped.
    • Every other character, non-ASCII included, is emitted as UTF-8 with no escaping.
  • Non-JSON payloads (binary, plaintext, anything with trailing content after a top-level JSON value) are signed as the raw bytes received.

A verifier that does not reproduce that exact byte sequence will silently fail. This is the single most common source of false negatives — see Verification.

Error handling​

on_signing_errorBehavior
ignore (default)The error is logged and the message passes through without signature attachment. The pipeline continues.
failThe processor returns a structured error. The message is routed via your configured error-handling pattern, so you can dead-letter or retry.

Three additional failure modes are surfaced at startup, not at runtime:

  1. Old edge binary (predating the signature processor) rejects the unknown processor type at stream init, so a misconfigured rollout is caught at deploy time.
  2. New edge binary without a signing identity refuses to construct the executor with a clear "no signing identity provided; this node cannot run pipelines that use the signature processor" message.
  3. Signer missing for an execution returns a runtime error from the builder — this indicates an internal bug and should be reported.

Verification​

The processor signs; it does not verify. A downstream consumer that wants to validate a signed message needs to apply the same canonicalisation rule the producer used and call an Ed25519 verifier. The protocol is the same regardless of whether the signature was carried in metadata or in the body.

Recover the canonical bytes​

Metadata mode (target: meta, or target: both when consuming via a header-propagating transport):

  1. Read the signature, key ID, and algorithm from message metadata (HTTP headers, Kafka headers, NATS headers, …).
  2. Re-encode the body per the canonical rules above, keeping number literals byte-for-byte rather than parsing them into floats. The payload is not required to be an object in metadata mode: arrays and scalars canonicalise the same way. If the body is not JSON, treat the raw bytes as the canonical form.

Body-nested mode (target: body, body_format: nested):

  1. Parse the body as JSON into a map, keeping number literals intact.
  2. Extract the alg, kid, and sig fields under the configured body_key (default _signature).
  3. Delete that root-level key from the map.
  4. Re-marshal the remaining map with sorted keys — that is the canonical input to the verifier.

Body-flat mode (target: body, body_format: flat):

  1. Parse the body as JSON into a map, keeping number literals intact.
  2. Read the three expanso_* (or renamed) keys from the root.
  3. Delete all three keys from the map.
  4. Re-marshal the remaining map with sorted keys.

Resolve the public key​

The kid is structured as node:<nodeID>#sha256:<thumbprint>, where the thumbprint is the RFC 7638 JWK thumbprint of the node's public Ed25519 key. To recompute it from a public key pub, hash the JWK template below — with no whitespace, in this exact field order — and base64url-encode the SHA-256 digest without padding:

{"crv":"Ed25519","kty":"OKP","x":"<base64url-nopad(pub)>"}

Consumers map the kid to a public key via whatever distribution channel suits the deployment (a registry endpoint, a static map, a JWKS document published by the orchestrator). The producing node's public key must be obtained out of band — the message itself does not carry it.

Verify​

// Pseudocode — every language with an Ed25519 implementation works the same way.
ok := ed25519.Verify(publicKey, canonicalBytes, signatureBytes)

Encodings are deliberate and distinct:

  • signatureBytes is the base64-std decode (RFC 4648 §4, padded) of the header or body value.
  • The kid thumbprint is base64url without padding (RFC 4648 §5).
  • The public-key x value inside the JWK template is also base64url without padding.

The canonical bytes are exactly the bytes produced by the recovery step above.

:::caution Verifier configuration must match the producer The defaults (expanso_signature, expanso_keyid, expanso_signature_alg, _signature) are convention, not wire format. If you change metadata_*_key, body_key, or body_format on the producer, your verifier must be configured with the same values — otherwise it will fail to locate the signature, or fail to delete it before canonicalising, and verification will silently fail. :::

Limitations​

  • No built-in replay protection. A captured signed message can be re-delivered at the sink. Consumers requiring replay prevention should add a transport-level nonce or timestamp and check it independently of the signature.
  • Canonicalisation must match exactly. Verifiers that sort keys differently, that lose precision by parsing numbers into floating point (what JSON.parse does in JavaScript), that pretty-print output, or that mishandle non-JSON payloads will produce false negatives.
  • Key rotation is not handled by the processor. The node's keypair is loaded once at service start. When you rotate it, records signed under the old key can only be verified if the consumer still holds the old public key — your key-distribution channel needs to retain historical keys for the lifetime of any verifiable record.
  • HTTP infrastructure may drop underscored header names. When propagating signatures via the ^expanso_ regex, some servers (nginx default, AWS ALB, some CDNs) strip headers whose names contain underscores. Rename the keys to use hyphens (e.g. x-expanso-signature) if your transport goes through such infrastructure.
  • Edge agent only. The processor runs only on edge nodes; the orchestrator is not involved in signing.

Examples​

Sign every message and push it to an HTTP endpoint, propagating the three signature metadata keys as headers:

input:
  generate:
    interval: 1s
    mapping: |
      root.temp_c    = 21.5
      root.sensor_id = 42
      root.marker    = "sig-example"

pipeline:
  processors:
    - signature: {}

output:
  http_client:
    url: https://example.com/ingest
    verb: POST
    metadata:
      include_patterns:
        - "^expanso_"

The receiver sees the body as it was emitted, plus Expanso_signature, Expanso_keyid, and Expanso_signature_alg headers.

For a complete pipeline that exercises both metadata and body modes through a switch output, see Sign Pipeline Messages.

Next steps​