Skip to main content

Sign Pipeline Messages

This example demonstrates how to sign every pipeline message with the edge node's Ed25519 identity and fan it out to two outputs at once — an HTTP endpoint that carries the signature as transport headers, and a file sink that carries the signature inside the body. It uses a single signature processor in target: both mode plus a switch output.

Download & Run​

Quick Start:

# Download and run directly
curl -sSL https://docs.expanso.io/examples/signing-and-verifying.yaml | expanso-edge run -

# Or download first, customize, then run
curl -o my-pipeline.yaml https://docs.expanso.io/examples/signing-and-verifying.yaml
expanso-edge run -f my-pipeline.yaml

Download: signing-and-verifying.yaml

What This Pipeline Does​

  1. Generates a synthetic event every second (no external dependencies)
  2. Signs each event with the edge node's Ed25519 identity, writing the signature to both metadata and the JSON body
  3. Routes each event by its destination field via a switch output
  4. Delivers matching events to an HTTP endpoint, propagating the signature as headers
  5. Persists matching events to a JSON-Lines file, where the signature lives inside the body and the record is verifiable on its own

Complete Pipeline​

input:
  generate:
    interval: 1s
    mapping: |
      root.temp_c     = 21.5
      root.sensor_id  = 42
      root.marker     = "sig-example"
      root.destination = if random_int(min: 0, max: 1) == 0 { "http" } else { "file" }

pipeline:
  processors:
    - signature:
        target: both
        body_format: nested
        body_key: _signature
        on_signing_error: fail

output:
  switch:
    cases:
      - check: this.destination == "http"
        output:
          http_client:
            url: https://example.com/ingest
            verb: POST
            metadata:
              include_patterns:
                - "^expanso_"

      - check: this.destination == "file"
        output:
          file:
            path: /var/log/signed-events.jsonl
            codec: lines

Configuration Breakdown​

Input: Generate​

input:
  generate:
    interval: 1s
    mapping: |
      root.temp_c     = 21.5
      root.sensor_id  = 42
      root.marker     = "sig-example"
      root.destination = if random_int(min: 0, max: 1) == 0 { "http" } else { "file" }

A self-contained synthetic event so the pipeline runs out of the box. The destination field exists only to demonstrate the switch output — in a real deployment your routing logic would be driven by upstream data.

See: Generate Input | Bloblang Guide

Signing: signature with target: both​

pipeline:
  processors:
    - signature:
        target: both
        body_format: nested
        body_key: _signature
        on_signing_error: fail

A single signing step covers both sinks:

  • target: both writes the signature to message metadata (three expanso_* keys) and splices it into the JSON body under _signature. Each sink picks up whichever form it can carry.
  • body_format: nested puts the signature material under a single root key (_signature) holding alg, kid, and sig. The other layout — flat — would put three keys at the JSON root instead.
  • on_signing_error: fail makes the pipeline reject unsigned messages rather than letting them through. Combined with your standard error handling, failing messages can be dead-lettered or retried.

Placement matters. The signature processor sits at the end of the pipeline, so the signature attests to the message as it leaves the node. Moving it before any further transformation would attest to the raw ingest payload instead. See Pipeline position is the signature's meaning.

See: signature processor reference

Output branch: HTTP with header propagation​

- check: this.destination == "http"
  output:
    http_client:
      url: https://example.com/ingest
      verb: POST
      metadata:
        include_patterns:
          - "^expanso_"

The http_client output ships the body as POST data and exposes selected metadata as transport headers. The ^expanso_ regex matches the three signature keys (expanso_signature, expanso_keyid, expanso_signature_alg) and propagates them as headers — the receiver sees the unmodified ingest body plus three signature headers (HTTP headers are case-insensitive, so they may arrive as expanso_signature or Expanso-Signature depending on the receiver).

This branch demonstrates why metadata mode exists: the body on the wire is the original ingest bytes, and a verifier reconstructs the canonical form on its own by parsing and re-marshalling with sorted keys before calling Ed25519.Verify.

HTTP header names with underscores

Some HTTP servers and proxies (nginx with the default underscores_in_headers off, AWS ALB, parts of the CDN ecosystem) silently strip or rewrite request headers whose names contain underscores. If you ship behind any such infrastructure, rename the keys to use hyphens — for example, metadata_signature_key: x-expanso-signature — and configure your verifier to match.

See: HTTP Client Output

Output branch: File sink with body-embedded signature​

- check: this.destination == "file"
  output:
    file:
      path: /var/log/signed-events.jsonl
      codec: lines

The file output writes one JSON object per line. File records have no headers, so the metadata-only signature would be lost. Because we used target: both, the signature also lives inside the body as the _signature key, and a downstream tool reading the JSON-Lines file can verify each record on its own with nothing but the file contents and the producing node's public key.

This branch demonstrates why body mode exists: any consumer that does not carry transport headers — file, object store, archives, replay logs — needs the signature inside the record.

See: File Output | Switch Output

What a Consumer Sees​

For an input event of:

{"temp_c":21.5,"sensor_id":42,"marker":"sig-example","destination":"http"}

The HTTP receiver gets an unchanged body plus three headers:

POST /ingest HTTP/1.1
Content-Type: application/json
Expanso_signature_alg: Ed25519
Expanso_keyid: node:a1b2c3d4-5e6f-7890-abcd-ef1234567890#sha256:EftgCusZmspbrXBf1Gx8jnomxQdxTr0O6pI9jhqex3w
Expanso_signature: uSRm83nRNPKfOG/8P4WS/Vwb9Cnty9OF1Kgk7cxgZZZyCITfRwUJXEkOXMRnrV+yO/hlGhbroF1RQzRbPVvLAw==

{"temp_c":21.5,"sensor_id":42,"marker":"sig-example","destination":"http"}

The file consumer gets self-contained JSON-Lines records:

{"_signature":{"alg":"Ed25519","kid":"node:a1b2c3d4-...#sha256:nVaBCZ...quU","sig":"Vb0/zoMrfNlG...mzRm1FnBw=="},"destination":"file","marker":"sig-example","sensor_id":42,"temp_c":21.5}

Notice that JSON object keys appear in byte-wise lexicographic order in the canonical form a verifier reconstructs (the order Go's encoding/json produces). This is intentional — the producer round-trips through a sorted-keys representation before signing, so a verifier that does the same round-trip will recover identical bytes regardless of the original field order.

Common Variations​

Use target: meta only​

When all of your outputs propagate metadata as headers (HTTP, Kafka, NATS), drop the body splice entirely:

pipeline:
  processors:
    - signature: {}   # target: meta is the default

Use target: body only​

When no sink in your fan-out carries transport headers — e.g. file, S3, GCS — keep the signature inside the body:

pipeline:
  processors:
    - signature:
        target: body

Sign the raw ingest payload instead of the enriched output​

Move the signature step before any enrichment. The signature then attests to exactly what the upstream producer emitted, which is what regulated workloads usually want:

pipeline:
  processors:
    - signature: {}
    - mapping: |
        root.enriched_at = now()
        root.region      = "eu-west-1"

Rename the metadata keys​

If your payload already uses one of the default key names, rename them — but remember the verifier must use the same names:

pipeline:
  processors:
    - signature:
        metadata_signature_key: x_sensor_signature
        metadata_keyid_key: x_sensor_keyid
        metadata_algorithm_key: x_sensor_alg
        body_key: _attestation

Verifying Signatures Downstream​

A consumer that wants to validate signed messages needs to apply the same canonicalisation rule the producer used, then call any Ed25519 verifier. Pseudocode:

1. Recover canonical bytes:
   - If signature was in metadata: parse body as JSON with UseNumber, re-marshal with sorted keys.
   - If signature was in body (nested): parse, extract alg/kid/sig from `_signature`, delete that
     root key, re-marshal the remaining map with sorted keys.

2. Resolve the public key from `kid` (format: node:<nodeID>#sha256:<RFC7638 thumbprint>).

3. ok := ed25519.Verify(publicKey, canonicalBytes, base64Decode(sig))

The single most common failure mode is canonicalisation drift — verifiers that don't sort keys, that lose int64 precision through a float64 conversion, or that don't delete the body-embedded signature before re-marshalling. See the signature reference's Verification section for the complete protocol.

Next Steps​