What is Expanso and how does it work?

Expanso is a managed platform for deploying intelligent data pipelines at the edge. It processes data where it's generated - reducing bandwidth, latency, and costs. You deploy lightweight agents on your infrastructure, build pipelines using our visual builder or YAML, and control everything from a central SaaS platform.

Can I run AI/ML models directly in my data pipelines?

Yes! Expanso supports running ONNX, TensorFlow Lite, and other models as native pipeline steps. Execute low-latency inference on streaming data, enrich events with model outputs (like risk scores), and make decisions at the edge without cloud round-trips.

How many pre-built components are available?

Expanso provides 200+ pre-built components including inputs (Kafka, HTTP, files), processors (transformations, filtering, PII masking, aggregations), and outputs (S3, Snowflake, Datadog, Splunk). Browse the complete catalog in our Component Reference.

Do I need to write code to build pipelines?

No - use our drag-and-drop visual pipeline builder to create sophisticated pipelines without code. For advanced use cases, you can also write pipelines in YAML or use the Bloblang transformation language for complex data mappings.

How does Expanso help with data governance and compliance?

Expanso includes built-in governance features: automatic PII detection and masking, policy enforcement at the edge, RBAC, SSO integration, and comprehensive audit trails. Mask sensitive data before it ever leaves your network.

Edge Configuration Reference

This page documents all available configuration options for the Expanso Edge agent.

Configuration File

The edge configuration file is typically located at /etc/expanso/edge/config.yaml or can be specified via the --config flag.

Configuration Options

All configuration options are optional and have sensible defaults. You only need to specify what you want to change.

Field	Type	Description
`name`	`string`	Name is the node name that identifies this edge node in the control plane.
`name_provider`	`string`	NameProvider controls automatic name generation when name is not explicitly set.
`data_dir`	`string`	DataDir is the core data directory where all edge state is stored.
`local_mode`	`bool`	LocalMode enables standalone operation without control plane connection.
`orchestrator`	`*EdgeOrchestratorConfig`	Orchestrator contains connection settings for the Expanso control plane.
`api`	`APIConfig`	-
`log`	`LogConfig`	-
`telemetry`	`TelemetryConfig`	Simplified telemetry configuration
`store`	`StoreConfig`	-
`shutdown_timeout`	`time.Duration`	ShutdownTimeout is the maximum time to wait for graceful shutdown.
`labels`	`map[string]string`	Labels are key-value pairs used for node selection and job targeting.
`executor`	`ExecutorConfig`	Executor contains configuration for job executors.

Nested Configuration

Orchestrator Configuration (`orchestrator`)

Important: This section is automatically populated when you run expanso-edge bootstrap. You should not manually configure these settings.

The bootstrap process connects to the Expanso orchestrator, obtains authentication credentials, and saves them to config.d/99-dynamic.yaml. These settings are managed automatically.

Field	Type	Description
`node_id`	`string`	Node ID assigned during bootstrap (auto-populated)
`credentials_path`	`string`	Path to NATS credentials file (auto-populated)
`address`	`string`	Orchestrator NATS address (auto-populated)
`refresh_address`	`string`	Orchestrator refresh endpoint (auto-populated)
`require_tls`	`bool`	Require TLS for connections
`insecure`	`bool`	Enable insecure mode (development only)
`heartbeat_interval`	`duration`	Interval for sending heartbeats (can override default)

API Configuration (`api`)

Configuration for the Edge API server.

Field	Type	Description
`listen_addr`	`string`	API listen address (e.g., `localhost:9010`)
`auth.token`	`string`	API authentication token

Log Configuration (`log`)

Configuration for logging.

Field	Type	Description
`level`	`string`	Log level: `trace`, `debug`, `info`, `warn`, `error`
`format`	`string`	Log format: `console`, `json`, `text`

Telemetry Configuration (`telemetry`)

Configuration for metrics and telemetry export.

Field	Type	Description
`do_not_track`	`bool`	Disable telemetry collection
`endpoint`	`string`	Telemetry collector endpoint
`endpoint_path`	`string`	Path for telemetry endpoint
`protocol`	`string`	Export protocol: `grpc` or `http`
`insecure`	`bool`	Disable TLS verification
`export_interval`	`duration`	How often to export metrics (default: 30s)
`headers`	`map[string]string`	Optional headers for authentication
`resource_attributes`	`map[string]string`	Additional resource attributes
`include_go_metrics`	`bool`	Include Go runtime metrics
`process_metrics_interval`	`duration`	Process metrics collection interval (default: 15s)
`authentication.type`	`string`	Authentication type (e.g., `Basic`)
`authentication.token`	`string`	Authentication token
`authentication.namespace`	`string`	Telemetry namespace

Detailed Configuration Guide

This section provides in-depth guidance for key configuration options.

`name`

Type: string

Name is the node name that identifies this edge node in the control plane.

WHEN TO SET:

Explicitly set for meaningful identification (e.g., "warehouse-scanner-01")
Leave empty to auto-generate based on name_provider

DEFAULT: Auto-generated using name_provider (default: hostname)

IMPACT:

Human-readable display name used in UI and CLI
Users can list and query nodes by name (queries fail if duplicate names exist)
Optional field - separate from the auto-generated NodeID

`name_provider`

Type: string

NameProvider controls automatic name generation when name is not explicitly set. Valid values: "cloud", "hostname", "uuid", "machine-id"

WHEN TO USE EACH:

"hostname" (default): Use system hostname - good for VMs with meaningful hostnames
"uuid": Generate random UUID - good for containers/temporary nodes
"machine-id": Use system machine-id - good for persistent physical hardware
"cloud": Use cloud provider instance ID - good for AWS/GCP/Azure deployments

DEFAULT: "hostname"

IMPACT: Only used if name field is empty. Once name is set (manually or auto-generated), it persists and name_provider is no longer used.

`data_dir`

Type: string

DataDir is the core data directory where all edge state is stored. Subdirectories are managed automatically: state/, metrics/, temp/, auth/, config.d/

WHEN TO SET:

Production: Use persistent storage location (e.g., /var/lib/expanso-edge)
Development: Use local directory (e.g., ./data)
Containers: Use mounted volume (e.g., /data)

REQUIRED: Yes

SECURITY: This directory contains node credentials and sensitive data. Ensure:

Proper file permissions (600/700)
Persistent storage (survives restarts)
Regular backups for disaster recovery
Not in world-readable location

IMPACT: If this directory is lost, node must be re-bootstrapped with new credentials.

`local_mode`

Type: bool

LocalMode enables standalone operation without control plane connection.

WHEN TO USE TRUE:

Development and testing without control plane
Air-gapped environments with no network access
Standalone edge computing scenarios
Running edge independently for demos/evaluation

WHEN TO USE FALSE (default):

Production deployments with centralized management
Scenarios requiring job orchestration from central control plane
When you need remote monitoring and management

IMPACT WHEN TRUE:

No control plane connection (orchestrator section ignored)
No remote job assignments
Node runs independently
API remains available for local management

IMPACT WHEN FALSE:

Requires bootstrap credentials or existing connection
Connects to control plane for job assignments
Enables centralized management and monitoring

DEFAULT: false (control plane mode)

`shutdown_timeout`

Type: time.Duration

ShutdownTimeout is the maximum time to wait for graceful shutdown.

WHEN TO ADJUST:

Increase for long-running job cleanup (e.g., 60s, 120s)
Decrease for fast restarts in development (e.g., 10s)

DEFAULT: 30s

IMPACT:

Jobs are given this time to complete gracefully
After timeout, edge forcefully terminates
Too short: jobs may not cleanup properly
Too long: slow restarts and deployments

`labels`

Type: map[string]string

Labels are key-value pairs used for node selection and job targeting.

COMMON USE CASES:

Environment: env=production, env=staging
Location: region=us-west, datacenter=dc1, zone=a
Hardware: gpu=nvidia-t4, cpu=high-performance
Function: role=log-processor, type=edge-gateway

HOW JOBS USE LABELS: Jobs specify selectors to match nodes: selector: match_labels: env: production region: us-west

BEST PRACTICES:

Use consistent naming (lowercase, hyphens)
Keep labels stable (don't change frequently)
Document label schema across your fleet
Use hierarchical labels (region/zone, env/tier)

DYNAMIC UPDATES: Labels can be updated without restart if config watching is enabled. New jobs will target based on updated labels immediately.

Scenario-Based Examples

Production Deployment

Minimal configuration for production edge nodes. The orchestrator connection is established automatically via expanso-edge bootstrap.

# Node identification (optional - defaults to hostname)
name: ""  # Leave empty to auto-generate from hostname
name_provider: hostname

# Data storage
data_dir: /var/lib/expanso/edge

# Labels for job targeting
labels:
  env: production
  region: us-west
  role: log-processor

# Optional overrides
log:
  level: info
  format: json

shutdown_timeout: 60s  # Allow jobs time to cleanup

Note: After running expanso-edge bootstrap, orchestrator connection settings are automatically saved to config.d/99-dynamic.yaml.

Development/Testing

Configuration for local development in standalone mode (no orchestrator connection):

name: dev-edge
data_dir: ./data
local_mode: true  # Standalone mode - no orchestrator connection

log:
  level: debug
  format: console

telemetry:
  do_not_track: true  # Disable telemetry

shutdown_timeout: 10s  # Fast restarts

Container Deployment

Configuration for containerized environments:

name_provider: uuid  # Generate unique ID for each container
data_dir: /data  # Mounted volume

labels:
  env: ${ENV}  # From environment variables
  pod: ${HOSTNAME}

log:
  level: info
  format: json  # Structured logs for aggregation

Minimal Example

The simplest possible configuration - everything else uses defaults:

data_dir: /var/lib/expanso/edge

This is all you need! The edge agent will:

Auto-generate a name from the hostname
Use default log settings (info level, console format)
Listen on the default API address (localhost:9010)
After running expanso-edge bootstrap, connect to the orchestrator automatically

Configuration File​

Configuration Options​

Nested Configuration​

Orchestrator Configuration (orchestrator)​

API Configuration (api)​

Log Configuration (log)​

Telemetry Configuration (telemetry)​

Detailed Configuration Guide​

name​

name_provider​

data_dir​

local_mode​

shutdown_timeout​

labels​

Scenario-Based Examples​

Production Deployment​

Development/Testing​

Container Deployment​

Minimal Example​

Configuration File

Configuration Options

Nested Configuration

Orchestrator Configuration (`orchestrator`)

API Configuration (`api`)

Log Configuration (`log`)

Telemetry Configuration (`telemetry`)

Detailed Configuration Guide

`name`

`name_provider`

`data_dir`

`local_mode`

`shutdown_timeout`

`labels`

Scenario-Based Examples

Production Deployment

Development/Testing

Container Deployment

Minimal Example