Skip to main content

Expanso Documentation

Expanso enables you to deploy and orchestrate data pipelines from edge to destination through a unified control plane. Process, filter, and transform data - and run AI models - before it hits your backend platforms.

Deploy intelligent data pipelines anywhere

Process data and run inference where it's generated. Cut costs by filtering at the source. Deploy in minutes with 200+ pre-built components.

Cost Reduction

Filter and sample at source - only send valuable data downstream

Edge AI & ML

Call LLM and embedding models as pipeline steps, locally or via cloud providers

Built-In Governance

Mask PII and enforce policies before data leaves the edge

Platform Agnostic

Works with Kafka, Snowflake, S3, and more

Start Free Trial Quick Start Guide →

What is Expanso?​

Expanso is a managed platform for deploying intelligent data pipelines at the edge:

  • Edge-Native Architecture: Process data where it's generated - reduce bandwidth and latency.
  • AI in Your Pipelines: Call LLM and embedding models as native pipeline steps, locally via Ollama or through OpenAI, AWS Bedrock, and Vertex AI.
  • 200+ Components: Pre-built inputs, processors, and outputs for any pipeline need.
  • Visual Pipeline Builder: Build pipelines visually, or author them in YAML.
  • Managed SaaS: Central control plane with automatic agent updates.
  • Enterprise Governance: PII masking, policy enforcement, and compliance audit trails.

Quick Start​

Get your first pipeline running in 15 minutes:

  1. Sign up for Expanso Cloud (free tier available)
  2. Install an agent on your infrastructure
  3. Build a pipeline using our visual builder or YAML

Core Features​

Visual Pipeline Builder​

Create sophisticated pipelines visually or in YAML. Preview data transformations in real time as you build.

Edge AI & ML​

Call large language and embedding models directly on streaming data as a native pipeline step, locally with Ollama or through providers like OpenAI, AWS Bedrock, and Google Vertex AI.

  • Local or Cloud Inference: Run models locally with Ollama, or call managed provider APIs.
  • Enrich Data: Add model outputs (like a classification or risk score) to your events before routing.
  • Flexible Integration: Integrate with your existing ML workflows and model formats.

Powerful Transformations​

Leverage 200+ built-in components for ingesting, transforming, and routing data:

  • Parse logs (JSON, syslog, CSV, regex)
  • Filter and sample to reduce volume
  • Mask PII automatically
  • Aggregate metrics in time windows
  • Enrich with lookups, APIs, or ML models
  • Route to multiple destinations

Enterprise Ready​

  • Governance: PII detection, masking, and policy enforcement
  • Monitoring: Built-in metrics and health checks
  • Security: RBAC, SSO, audit trails
  • High Availability: Automatic failover and recovery

Example Pipeline​

Here's a pipeline that drops noisy logs and masks PII at the edge before archiving the rest to S3:

input:
  kafka:
    addresses: [kafka:9092]
    topics: [app-logs]

pipeline:
  processors:
    - mapping: |
        # Parse each log line and drop noisy debug logs at the edge
        root = this.parse_json()
        root = if root.level == "debug" { deleted() }

    - mapping: |
        # Mask PII on the logs that remain
        root = this
        root.user.email = this.user.email.hash("sha256")
        root.user.ip = deleted()

output:
  aws_s3:
    bucket: log-archive
    path: "${! timestamp_unix() }-${! uuid_v4() }.json"

See more examples →

Documentation Guide​

We've organized our documentation to help you at different stages of your journey:

🎓 Getting Started - Learn the Basics​

New to Expanso? Start here! These hands-on tutorials will get you up and running quickly.

🔧 How-To Guides - Solve Specific Problems​

Already familiar with the basics? Find step-by-step solutions to common problems.

Most Popular:

📋 Component Reference - Browse Available Components​

Need to look up a specific input, processor, or output? Browse our complete component catalog.

I want to...

Frequently Asked Questions​

What is Expanso and how does it work?

Expanso is a managed platform for deploying intelligent data pipelines at the edge. It processes data where it's generated - reducing bandwidth, latency, and costs. You deploy lightweight agents on your infrastructure, build pipelines using our visual builder or YAML, and control everything from a central SaaS platform.

Can I run AI/ML models directly in my data pipelines?

Yes. Expanso can call large language and embedding models as native pipeline steps. Run them locally with Ollama, or use managed providers like OpenAI, AWS Bedrock, and Google Vertex AI. Enrich events with model outputs (like classifications or risk scores) and route on the results.

How many pre-built components are available?

Expanso provides 200+ pre-built components including inputs (Kafka, HTTP, files), processors (transformations, filtering, PII masking, aggregations), and outputs (S3, Snowflake, Kafka, SQL databases). Browse the complete catalog in our Component Reference.

Do I need to write code to build pipelines?

No, use our visual pipeline builder to create sophisticated pipelines without code. For advanced use cases, you can also write pipelines in YAML or use the Bloblang transformation language for complex data mappings.

How does Expanso help with data governance and compliance?

Expanso includes built-in governance features: automatic PII detection and masking, policy enforcement at the edge, RBAC, SSO integration, and comprehensive audit trails. Mask sensitive data before it ever leaves your network.

Community & Support​

Ready to get started? Jump to the Quick Start guide →