What is Expanso and how does it work?

Expanso is a managed platform for deploying intelligent data pipelines at the edge. It processes data where it's generated - reducing bandwidth, latency, and costs. You deploy lightweight agents on your infrastructure, build pipelines using our visual builder or YAML, and control everything from a central SaaS platform.

Can I run AI/ML models directly in my data pipelines?

Yes! Expanso supports running ONNX, TensorFlow Lite, and other models as native pipeline steps. Execute low-latency inference on streaming data, enrich events with model outputs (like risk scores), and make decisions at the edge without cloud round-trips.

How many pre-built components are available?

Expanso provides 200+ pre-built components including inputs (Kafka, HTTP, files), processors (transformations, filtering, PII masking, aggregations), and outputs (S3, Snowflake, Datadog, Splunk). Browse the complete catalog in our Component Reference.

Do I need to write code to build pipelines?

No - use our drag-and-drop visual pipeline builder to create sophisticated pipelines without code. For advanced use cases, you can also write pipelines in YAML or use the Bloblang transformation language for complex data mappings.

How does Expanso help with data governance and compliance?

Expanso includes built-in governance features: automatic PII detection and masking, policy enforcement at the edge, RBAC, SSO integration, and comprehensive audit trails. Mask sensitive data before it ever leaves your network.

Edge Node Configuration

Configuration reference for Expanso Edge nodes.

Minimal Configuration

Only one field is required to run an edge node:

# Core data directory (required)
data_dir: ~/.expanso-edge

With just data_dir configured:

The edge node runs in local mode (no orchestrator connection)
Node name defaults to system hostname
Subdirectories are created automatically
API server is disabled by default
Logging defaults to info level with json format

Common Configurations

Connect to Orchestrator

To connect to Expanso Cloud (most common):

data_dir: ~/.expanso-edge

orchestrator:
  address: nats://orchestrator.expanso.io:4222

Custom Node Name

Set a custom node name instead of using the hostname:

data_dir: ~/.expanso-edge
name: edge-production-01

Enable API Server

Enable the local management API:

data_dir: ~/.expanso-edge

api:
  listen_addr: localhost:9010

Warning: The API has no authentication by default. Only enable on trusted networks or add authentication.

Debug Logging

Enable debug logging for troubleshooting:

data_dir: ~/.expanso-edge

log:
  level: debug
  format: text  # Human-readable format

Complete Configuration Reference

All available configuration options:

# Node name - defaults to system hostname
name: edge-node-01

# Name provider for automatic name generation
# Options: "cloud", "hostname", "uuid", "machine-id"
name_provider: hostname

# Core data directory (REQUIRED)
data_dir: /var/lib/expanso-edge

# Local mode - run without orchestrator connection
local_mode: false

# Orchestrator connection settings
orchestrator:
  address: nats://orchestrator.expanso.io:4222
  require_tls: true
  # Network identifier (optional)
  # Set automatically during bootstrap, or manually for local development
  network_id: ""

# API server configuration
api:
  listen_addr: localhost:9010
  auth:
    token: your-api-token-here

# Logging configuration
log:
  level: info  # trace, debug, info, warn, error
  format: json # console, json, text

# Telemetry configuration (OpenTelemetry)
telemetry:
  endpoint: localhost:4317
  protocol: grpc  # grpc or http
  export_interval: 30s
  resource_attributes:
    environment: production
    region: us-west-2

# Graceful shutdown timeout
shutdown_timeout: 30s

# Node labels for selection and filtering
labels:
  region: us-west-2
  environment: production

# Display startup configuration banner (default: true)
display_banner: true

Configuration Sections

Node Identity

# Node name (defaults to hostname)
name: edge-node-01

# Name generation method
name_provider: hostname  # cloud, hostname, uuid, machine-id

Name Providers:

hostname: Uses system hostname (default)
cloud: Fetches from cloud metadata (AWS, GCP, Azure)
uuid: Generates random UUID
machine-id: Uses system machine ID

Data Directory

# REQUIRED: Core data directory
data_dir: /var/lib/expanso-edge

Subdirectories (created automatically):

state/ - Pipeline state and checkpoints
metrics/ - Metrics data
temp/ - Temporary files
auth/ - Credentials from bootstrap
config.d/ - Dynamic configuration

Orchestrator Connection

orchestrator:
  address: nats://orchestrator.expanso.io:4222
  require_tls: true
  network_id: ""  # Set automatically during bootstrap

Key points:

Address uses NATS protocol
Credentials populated during bootstrap
Automatic retry with exponential backoff

Network ID:

The network_id field identifies which network the node belongs to. This is set automatically during bootstrap based on the network you're registering the node with. For local development without an orchestrator, you can set this manually (e.g., network_id: "local-dev").

Bootstrap TLS Settings

If your self-hosted control plane uses self-signed certificates, you can skip TLS verification during bootstrap.

Security Notice

Only use this when connecting to a control plane you trust. Skipping TLS verification disables certificate validation, which could expose your bootstrap token if connecting to an untrusted server.

CLI flags:

Command	Flag	Description
`expanso-edge bootstrap`	`--insecure` or `-k`	Skip TLS verification during bootstrap
`expanso-edge run`	`--bootstrap-insecure` or `-k`	Skip TLS verification for automatic bootstrap on startup

Environment variable:

# Skip TLS verification during bootstrap
export EXPANSO_EDGE_BOOTSTRAP_INSECURE=true

Example usage:

# Bootstrap with self-signed certificate
expanso-edge bootstrap --token "abc123" --url "https://bootstrap.internal:8443" --insecure

# Or using environment variable
EXPANSO_EDGE_BOOTSTRAP_INSECURE=true expanso-edge bootstrap --token "abc123"

Scope

This only affects the HTTPS call to the bootstrap service. After bootstrap, the edge node connects to the orchestrator via NATS with separate TLS settings.

Custom CA Certificates for NATS

If your orchestrator uses certificates signed by a private CA, the control plane can provide custom CA certificates during bootstrap. These are automatically used for NATS connections.

How it works:

During bootstrap, the control plane returns custom CA certificates (if configured)
The edge node stores these certificates in its connection configuration
When connecting to NATS, the edge node trusts both system CAs and the custom CAs

This lets edge nodes connect to orchestrators using your private PKI while still working with public CA-signed certificates.

Server-Side Configuration

Custom CA certificates are configured on the control plane, not the edge node. The edge node receives them automatically during bootstrap. See your self-hosted deployment documentation for control plane configuration.

API Server

api:
  listen_addr: localhost:9010
  auth:
    token: your-api-token

Authentication options:

Token: auth.token: "secret"
Basic: auth.username + auth.password
JWT: auth.jwt (for identity provider integration)
None: Omit auth section (not recommended)

JWT Authentication

For browser-based access or identity provider integration (e.g., AWS Cognito):

api:
  listen_addr: ":9010"
  auth:
    jwt:
      issuer: "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_xxxxx"
      workspace_id: "ws_abc123"

Required fields:

issuer: JWT issuer URL (e.g., Cognito User Pool URL). JWKS URL derived automatically by appending /.well-known/jwks.json
workspace_id: Workspace ID for this orchestrator. Tokens must include this workspace in claims

CORS Configuration

For browser-based API access, configure allowed origins:

api:
  cors:
    allowed_origins:
      - "https://cloud.expanso.io"
      - "https://localhost:*"

Security:

Default is localhost:9010 (local-only)
Use reverse proxy for HTTPS and remote access

Logging

log:
  level: info
  format: json

Levels:

error - Only errors
warn - Warnings and errors
info - Informational messages (default)
debug - Detailed debugging
trace - Very detailed tracing

Formats:

json - Structured JSON (recommended for production)
text - Human-readable text
console - Colored console output

Telemetry

telemetry:
  endpoint: localhost:4317
  protocol: grpc
  export_interval: 30s
  resource_attributes:
    environment: production
    region: us-west-2

Protocols:

grpc - OTLP over gRPC (default, port 4317)
http - OTLP over HTTP (port 4318)

Metric Filtering

Edge nodes drop high-cardinality metrics by default to reduce telemetry costs. Control this with drop_metric_prefixes:

telemetry:
  endpoint: localhost:4317
  drop_metric_prefixes:
    - "db."
    - "ncl."
    - "ncltransport."

Defaults:

Edge nodes: Drop db.*, ncl.*, ncltransport.*
Orchestrators: Export all metrics (no filtering)

Export all metrics (for debugging):

telemetry:
  drop_metric_prefixes: []  # Empty list keeps all metrics

Add custom filters:

telemetry:
  drop_metric_prefixes:
    - "db."
    - "ncl."
    - "ncltransport."
    - "store_gc."  # Also drop garbage collection metrics

Labels

labels:
  region: us-west-2
  environment: production
  hardware: gpu

Labels are used for:

Node selection in deployment targeting
Filtering nodes in orchestrator UI
Grouping metrics and logs

On startup, the edge node displays a banner showing its configuration:

+----------------------------------------------------------------------+
|                   EDGE NODE CONFIGURATION                            |
+----------------------------------------------------------------------+
| Node Name:       edge-node-01                                        |
| Node ID:         7f769f72-a4d4-4a05-8082-d63262957a6f                |
|                                                                      |
| Target Cluster:  us1.cloud.expanso.io                                |
| Control Plane:   tls://abc123.us1.cloud.expanso.io:4222              |
|                                                                      |
| Mode:            cloud-connected                                     |
|                                                                      |
| Config Source:   /home/user/.expanso-edge/data/config.d/             |
|   - 10-name.yaml                                                     |
|   - 50-connection.yaml                                               |
|                                                                      |
| Version:         v1.0.0                                              |
+----------------------------------------------------------------------+

Use this to verify which cluster you're connecting to, which config files are loaded, and whether you're in cloud-connected or local mode.

To disable the banner:

display_banner: false

Disable this for CI/CD pipelines, containerized deployments with log aggregation, or when you prefer minimal log output.

Environment Variables

All configuration can be overridden with environment variables:

# Core settings
export EXPANSO_EDGE_NAME=edge-node-01
export EXPANSO_EDGE_DATA_DIR=/var/lib/expanso-edge

# Orchestrator
export EXPANSO_EDGE_ORCHESTRATOR_ADDRESS=nats://orchestrator.expanso.io:4222

# Bootstrap TLS (for self-signed certificates)
export EXPANSO_EDGE_BOOTSTRAP_INSECURE=true  # Skip TLS verification

# API
export EXPANSO_EDGE_API_LISTEN_ADDR=localhost:9010

# Logging
export EXPANSO_EDGE_LOG_LEVEL=debug
export EXPANSO_EDGE_LOG_FORMAT=text

# Display
export EXPANSO_EDGE_DISPLAY_BANNER=false

Environment variables take precedence over file configuration.

Configuration File Locations

The edge node looks for configuration in these locations (in order):

Path specified with --config flag
./expanso-edge.yaml (current directory)
~/.expanso-edge/config.yaml
/etc/expanso-edge/config.yaml

Example Configurations

Production Deployment

name: edge-prod-${HOSTNAME}
data_dir: /var/lib/expanso-edge

orchestrator:
  address: nats://orchestrator.expanso.io:4222
  require_tls: true

api:
  listen_addr: localhost:9010
  auth:
    token: ${API_TOKEN}

log:
  level: info
  format: json

telemetry:
  endpoint: collector.expanso.io:4317
  protocol: grpc
  resource_attributes:
    environment: production
    region: us-west-2

labels:
  environment: production
  region: us-west-2

Development/Testing

name: dev-edge
data_dir: ./dev-data

local_mode: true  # No orchestrator

api:
  listen_addr: localhost:9010

log:
  level: debug
  format: text

telemetry:
  do_not_track: true  # Disable telemetry

High-Security Environment

name: secure-edge-01
data_dir: /var/lib/expanso-edge

orchestrator:
  address: nats://orchestrator.internal:4222
  require_tls: true

api:
  listen_addr: localhost:9010  # Local-only
  auth:
    username: admin
    password: ${ADMIN_PASSWORD}

log:
  level: warn
  format: json

telemetry:
  endpoint: internal-collector:4317
  protocol: grpc

Validation

The edge node validates configuration on startup:

data_dir is required
log.level must be valid option
log.format must be valid option
telemetry.protocol must be "grpc" or "http"

Invalid configuration will cause startup failure with clear error messages.

Next Steps

Installation Guide - Install Expanso Edge
Quick Start - Build your first pipeline

Minimal Configuration​

Common Configurations​

Connect to Orchestrator​

Custom Node Name​

Enable API Server​

Debug Logging​

Complete Configuration Reference​

Configuration Sections​

Node Identity​

Data Directory​

Orchestrator Connection​

Bootstrap TLS Settings​

Custom CA Certificates for NATS​

API Server​

JWT Authentication​

CORS Configuration​

Logging​

Telemetry​

Metric Filtering​

Labels​

Startup Banner​

Environment Variables​

Configuration File Locations​

Example Configurations​

Production Deployment​

Development/Testing​

High-Security Environment​

Validation​

Next Steps​