Verify Lineage Event Signatures

Every OpenLineage event emitted by Expanso Edge carries an Ed25519 signature under run.facets.signature. The signature proves which edge node produced the event and that the payload was not modified in transit. This page shows how to verify that signature in your downstream consumer.

The verification protocol is byte-precise. Most verifier bugs come from drift between the signer's canonical form and the verifier's reconstruction — wrong number type, wrong key ordering, wrong escape rule for <, >, or &. The reference implementations below match the signer byte-for-byte; deviate from them carefully.

What a signed event looks like

A START event with the signature facet expanded:

{
  "eventType": "START",
  "eventTime": "2026-05-22T14:23:11.123456789Z",
  "producer": "https://expanso.io/edge/v1.2.3",
  "schemaURL": "https://openlineage.io/spec/2-0-2/OpenLineage.json",
  "run": {
    "runId": "abcdef00-1111-2222-3333-444455556666",
    "facets": {
      "nominalTime": { "_producer": "...", "_schemaURL": "...", "nominalStartTime": "..." },
      "expanso_run": { "_producer": "...", "_schemaURL": "...", "fields": { "...": "..." } },
      "signature": {
        "_producer": "https://expanso.io/lineage/signature-facet/v1",
        "_schemaURL": "https://expanso.io/spec/lineage/signature.json",
        "algorithm": "Ed25519",
        "keyId": "node:7f769f72-a4d4-4a05-8082-d63262957a6f#sha256:nVaBCZ-XWaG8mO0c9VgvVx_quU",
        "payloadHash": "sha256:8a7b6c5d4e3f2g1h0i9j8k7l6m5n4o3p2q1r0s9t8u7v6w5x4y3z2",
        "signature": "Vb0/zoMrfNlG7q4mzRm1FnBwSCAJUq+wzD2Te6CrgZJrjMHC7nKHSqMOj6PgPGGevtmzRm1FnBw=="
      }
    }
  },
  "job": {
    "namespace": "production",
    "name": "log_processor"
  },
  "inputs":  [ { "namespace": "kafka://broker:9092", "name": "events.in" } ],
  "outputs": [ { "namespace": "s3://my-bucket", "name": "logs/${!timestamp_unix()}.json" } ]
}

The verification protocol

To verify the signature, reconstruct the canonical bytes that the signer signed and pass them to ed25519.Verify along with the decoded signature.

The signer signs the result of json.Marshal applied to the event-as-a-map with the signature facet removed. To reproduce that exactly:

1. Parse the event JSON into a map, preserving integer precision. In Go, use json.Decoder with UseNumber(). Without this, integer fields like byte counts above 2^53 lose precision through float64 and the canonical bytes diverge — verification will fail for any event with a large counter.

2. Delete map["run"]["facets"]["signature"] from the map. The signature facet is excluded from the bytes being signed.

3. Re-marshal the map to JSON with sorted keys and Go-style HTML escaping. Go's encoding/json sorts map keys automatically; you only need to match its escape rules. The signer escapes < as <, > as >, & as &, U+2028 as 
, and U+2029 as 
. All other non-ASCII characters pass through as UTF-8 bytes.

4. Base64-decode the signature value with standard base64 (with padding — base64.StdEncoding in Go, the default in most languages).

5. Verify with the producing node's public key: ed25519.Verify(pub, canonicalBytes, signature).

The payloadHash field on the signature facet is an audit aid only. It records sha256(canonicalBytes) so an auditor reviewing the wire payload can recompute the hash for cross-referencing. Verifiers do not use it as the signature input — the input is always the canonical bytes themselves.

Field encodings

Field	Encoding
signature	base64.StdEncoding (with = padding)
payloadHash	sha256: prefix followed by base64.RawURLEncoding (URL-safe alphabet, no padding)
keyId	node:<nodeID>#sha256:<RFC-7638-JWK-thumbprint> where the thumbprint is base64.RawURLEncoding of SHA-256(canonical-JWK)

Go reference verifier

package main

import (
	"bytes"
	"crypto/ed25519"
	"encoding/base64"
	"encoding/json"
	"errors"
	"fmt"
	"io"
)

// VerifyLineageEvent verifies the Ed25519 signature on an OpenLineage event
// emitted by Expanso Edge. eventJSON is the raw JSON bytes as received from
// the wire. pubKey is the producing edge node's Ed25519 public key, resolved
// out-of-band from the signature facet's keyId.
func VerifyLineageEvent(eventJSON []byte, pubKey ed25519.PublicKey) error {
	// 1. Parse into a map, preserving integer precision via UseNumber.
	dec := json.NewDecoder(bytes.NewReader(eventJSON))
	dec.UseNumber()
	var event map[string]any
	if err := dec.Decode(&event); err != nil {
		return fmt.Errorf("parse event: %w", err)
	}
	// Reject trailing junk that would slip past Decode silently.
	var rest any
	if err := dec.Decode(&rest); err != io.EOF {
		return errors.New("event has trailing content after JSON value")
	}

	// 2. Extract the signature facet and remove it from the map.
	run, ok := event["run"].(map[string]any)
	if !ok {
		return errors.New("event missing run object")
	}
	facets, ok := run["facets"].(map[string]any)
	if !ok {
		return errors.New("event missing run.facets")
	}
	sigFacet, ok := facets["signature"].(map[string]any)
	if !ok {
		return errors.New("event missing signature facet")
	}
	delete(facets, "signature")

	// 3. Re-marshal the map. json.Marshal on a map sorts keys and applies
	// Go's default HTML escaping — exactly what the signer did.
	canonical, err := json.Marshal(event)
	if err != nil {
		return fmt.Errorf("marshal canonical bytes: %w", err)
	}

	// 4. Decode the base64 signature with StdEncoding (padded).
	sigB64, ok := sigFacet["signature"].(string)
	if !ok {
		return errors.New("signature facet missing signature field")
	}
	sig, err := base64.StdEncoding.DecodeString(sigB64)
	if err != nil {
		return fmt.Errorf("decode signature: %w", err)
	}

	// 5. Verify.
	if !ed25519.Verify(pubKey, canonical, sig) {
		return errors.New("ed25519 signature does not verify")
	}
	return nil
}

The Go verifier matches the signer byte-for-byte because both call the same json.Marshal on a map[string]any. There is no encoding subtlety to manage.

Python reference verifier

Python verification is more delicate because Python's json.dumps does not match Go's encoding/json.Marshal out of the box. Two differences must be corrected:

• Python's json.dumps does not escape <, >, or & by default. Go does. Apply the substitution after dumping.
• Python's default ensure_ascii=True escapes all non-ASCII characters as \uXXXX. Go passes most non-ASCII through as raw UTF-8. Disable with ensure_ascii=False.

The verifier below applies the corrections in the order Go applies them.

import base64
import json
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from cryptography.exceptions import InvalidSignature


def _go_escape(payload: bytes) -> bytes:
    """
    Apply Go encoding/json's default HTML/JSONP escapes to JSON bytes that
    were produced with ensure_ascii=False. Go escapes the five characters
    below; Python does not.
    """
    return (
        payload
        .replace(b"<", b"\\u003c")
        .replace(b">", b"\\u003e")
        .replace(b"&", b"\\u0026")
        .replace(b"\xe2\x80\xa8", b"\\u2028")   # U+2028 LINE SEPARATOR
        .replace(b"\xe2\x80\xa9", b"\\u2029")   # U+2029 PARAGRAPH SEPARATOR
    )


def verify_lineage_event(event_json: bytes, public_key: Ed25519PublicKey) -> None:
    """
    Verify the Ed25519 signature on an Expanso Edge OpenLineage event.
    Raises InvalidSignature on verification failure.
    """
    # 1. Parse. Python ints are arbitrary precision, so no UseNumber-equivalent
    #    is needed — but we must reject trailing junk to match the Go contract.
    event = json.loads(event_json)
    if not isinstance(event, dict):
        raise ValueError("event is not a JSON object")

    # 2. Extract and remove the signature facet.
    sig_facet = event["run"]["facets"].pop("signature")

    # 3. Re-marshal with sorted keys, no whitespace, raw UTF-8 — then apply
    #    Go's HTML/JSONP escape pass to produce byte-identical canonical
    #    bytes.
    canonical = json.dumps(
        event,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
        allow_nan=False,
    ).encode("utf-8")
    canonical = _go_escape(canonical)

    # 4. Decode the signature (standard base64, with padding).
    sig = base64.b64decode(sig_facet["signature"])

    # 5. Verify. cryptography raises InvalidSignature on failure.
    public_key.verify(sig, canonical)

Test your verifier against a real event

Before deploying a verifier, prove byte-for-byte agreement with the signer using a real event. The single highest-value test is:

1. Capture one signed event from a running edge — write to a file with the file transport, then read one line.
2. Run both verifiers (or your verifier and the Go reference) over the same bytes.
3. Confirm both reconstruct identical canonical bytes — sha256(canonical) must match between languages, and both must match the payloadHash field on the signature facet (minus the sha256: prefix and after re-encoding with standard base64).
4. Confirm both verifiers return success against the signature.

If sha256(canonical) matches payloadHash but verification fails, the signature or public key is wrong. If sha256(canonical) does not match payloadHash, the canonicalization is wrong — re-check the escape rules and key sorting in step 3.

Discovering the public key

The keyId field on the signature facet identifies the signing node:

node:7f769f72-a4d4-4a05-8082-d63262957a6f#sha256:nVaBCZ-XWaG8mO0c9VgvVx_quU

The nodeID is between node: and #. Resolve the public key out of band — for example, by fetching the node's published JWK from a registry, or by trust-on-first-use against a key set maintained by your operator. Key distribution is a deployment concern, not part of the signature protocol itself.

The same keyId format and signing core are used by the signature processor. The Sign Pipeline Messages example covers the same key-resolution problem in the per-message context.

Common failure modes

• float64 precision loss. Verifier uses a default JSON decoder, large byte counters in expanso_run.fields parse as float64, canonical bytes diverge. In Go, set UseNumber(). In Python, use json.loads (which preserves int precision natively).
• Unsorted keys. Verifier marshals through a hash map that does not sort, or builds JSON manually. Sort every nested object's keys recursively. Go's json.Marshal on a map[string]any does this for you.
• HTML-escape drift in Python. Verifier uses json.dumps defaults and gets unescaped <, >, &. Apply the _go_escape substitution shown above.
• ensure_ascii drift in Python. Verifier uses json.dumps with ensure_ascii=True (default) and escapes all non-ASCII; Go does not. Set ensure_ascii=False.
• Wrong base64 alphabet. Verifier decodes signature with URL-safe base64. The signature uses standard base64 with padding (base64.StdEncoding). Only payloadHash uses URL-safe base64, and verifiers do not need to decode it.
• Failing to remove the signature facet before re-marshaling. The signer signs bytes that do not include the facet; the verifier must reconstruct the same bytes.

Related

• OpenLineage Emission how-to — enable the emitter on the edge.
• Lineage Configuration — full config reference.
• signature processor — per-message signing using the same Ed25519 identity and keyId format.
• Sign Pipeline Messages — sign and verify message bodies, with the same canonicalization concerns.