Skip to main content

Quick Start

Get your first data pipeline running in 5 minutes with Expanso Cloud.

Try Without Cloud First?

Want to test pipelines locally without setting up infrastructure? Check out Local Mode - run Edge standalone with no orchestrator or cloud connection required. Perfect for learning and development!

What You'll Build​

In this guide, you'll:

  1. Create an Expanso Cloud account
  2. Set up your first network
  3. Deploy a node
  4. Build a simple log processing pipeline
  5. See data flowing through your pipeline

Prerequisites​

  • A computer running Linux, macOS, or Windows
  • Internet connection
  • Terminal access

Step 1: Sign Up for Expanso Cloud​

  1. Visit cloud.expanso.io
  2. Click Sign Up and create your account
  3. Verify your email address
  4. Log in to Expanso Cloud

You'll see an overview of your networks in the Expanso Cloud.

Step 2: Create Your First Network​

A network is a logical grouping of nodes that work together.

  1. Click Create Network on the dashboard
  2. Enter a name: my-first-network
  3. Optionally add a description: "My first Expanso network"
  4. Click Create

Wait for the network to be created. Next, you'll deploy a node to this network.

Step 3: Deploy a Node​

A node is an edge agent running on your infrastructure that executes data pipelines.

Generate Bootstrap Token​

  1. In your network view, click Add Agent
  2. Copy the bootstrap token shown
  3. Keep this window open - you'll need the install command

Install the Edge Agent​

Choose your operating system:

# Download and install
curl -fsSL https://get.expanso.io/edge/install.sh | bash

# Verify installation
expanso-edge version

# Bootstrap the agent with your token
expanso-edge bootstrap --token YOUR_BOOTSTRAP_TOKEN

Verify Node Connection​

  1. Return to Expanso Cloud.
  2. You should see your node appear in the network overview or in the Nodes tab.
  3. Note the node's name (usually the hostname).

🎉 Your node is connected!

Step 4: Build Your First Pipeline​

Now let's create a simple pipeline that reads log files and sends them to an output.

Create the Pipeline​

  1. Open the Pipelines tab and click Create Pipeline in your network.
  2. Give it a name: log-processor.
  3. Click Open Pipeline Builder.

Configure the Input​

  1. In the pipeline builder, click Add Input.
  2. Search for and select File.
  3. Configure the file input: 
    paths:
      - /var/log/app/*.log
  4. Click Save

This will create a file input that reads logs from the /var/log/app/*.log directory.

Add a Processor (Optional)​

Let's add a simple filter to process only error logs:

  1. Click to add a processor after the input.
  2. Select Mapping (Bloblang)
  3. Add this simple transformation: 
    # Only pass through if log contains "ERROR"
    root = if this.message.contains("ERROR") {
      this
    } else {
      deleted()
    }
  4. Click Save

Configure the Output​

For this example, we'll output to STDOUT (you can see logs in the agent):

  1. Click Add Output
  2. Select STDOUT
  3. Configure format: 
    codec: lines
  4. Click Save

Visual Pipeline​

Your pipeline now looks like this:

[File Input] → [Filter (ERROR only)] → [STDOUT Output]

Step 5: Deploy and Monitor​

Deploy the Pipeline​

  1. Click Save in the pipeline builder
  2. Give your pipeline a name and description
  3. Select target agents (or use label selectors)
  4. Click Confirm Deploy

The pipeline will deploy to your node(s) within seconds.

Monitor Your Pipeline​

  1. Go to Monitoring in the navigation

  2. You'll see your pipeline's metrics:

    • Messages/sec: Throughput
    • Bytes/sec: Data volume
    • Errors: Any processing errors
    • Latency: Processing time

  3. Click on your pipeline to see detailed metrics

  4. View logs to see the actual data flowing through

What's Next?​

Congratulations! You've just:

  • ✅ Created an Expanso network
  • ✅ Deployed a node
  • ✅ Built a data pipeline
  • ✅ Deployed and monitored it

Learn More​

  • Core Concepts: Understand networks, nodes, and pipelines in depth
  • Components: Explore 200+ inputs, processors, and outputs
  • Use Cases: See real-world pipeline examples

Common Next Steps​

  1. Connect to Real Data Sources: Replace the file input with Kafka, HTTP, or other sources
  2. Transform Data: Use Bloblang to parse, enrich, and transform your data
  3. Send to Destinations: Output to S3, Elasticsearch, Kafka, or other systems
  4. Add More Nodes: Deploy nodes across your infrastructure
  5. Monitor at Scale: Set up alerts and dashboards

Need Help?​

Frequently Asked Questions​

How long does it take to set up Expanso?

You can have your first pipeline running in about 5 minutes. This includes creating a cloud account, deploying an agent, and building a simple pipeline. The agent installation and bootstrap process typically completes in under a minute.

Do I need to install any software on my infrastructure?

Yes, you need to install the Expanso Edge agent on your infrastructure. It's a lightweight binary that runs on Linux, macOS, or Windows. You can also run it in Docker containers. The edge agent is the only component you need to install - everything else is managed through Expanso Cloud.

Can I test Expanso without deploying to production?

Absolutely! You can use Local Mode to run pipelines standalone without any cloud connection. This is perfect for development and testing. You can also deploy nodes to non-production environments and test pipelines there before promoting to production.

What happens if my node goes offline?

If a node goes offline, Expanso Cloud will mark it as disconnected and stop sending new jobs to it. Any running pipelines on that node will stop. When the node comes back online, it will automatically reconnect to the control plane and resume executing its assigned pipelines.

How do I update my pipelines after deployment?

You can update pipelines anytime through the Expanso Cloud UI. Simply edit the pipeline configuration and redeploy. The node will receive the updated configuration and apply it automatically, typically with zero downtime using rolling updates.

Troubleshooting​

Node Not Connecting?​

  • Verify the bootstrap token is correct
  • Check network connectivity to cloud.expanso.io
  • Review edge agent logs: expanso-edge logs

Pipeline Not Deploying?​

  • Ensure the node is online (green status)
  • Check for configuration errors in the pipeline
  • Review deployment logs in the UI

No Data Flowing?​

  • Verify the input source has data
  • Check file paths and permissions
  • Review processor logic for filtering
  • Check edge agent logs for errors