Skip to main content

Local Mode Examples

Copy-paste examples to get started with common pipeline patterns.

Looking for More?

These examples use just a few of the available components. Browse all inputs, processors, and outputs for the full list.

File Processing​

Read log files and filter for errors only.

Create logs-job.yaml:

name: log-processor
description: Process log files and extract errors
type: pipeline
config:
  input:
    file:
      paths: ["/var/log/app/*.log"]
      codec: lines

  pipeline:
    processors:
      - mapping: |
          root = this.parse_json()
          root = if this.level != "ERROR" { deleted() }

  output:
    file:
      path: ./errors.log
      codec: lines

Deploy it:

export EXPANSO_CLI_ENDPOINT=http://localhost:9010
expanso-cli job deploy logs-job.yaml

Data Transformation​

Convert a CSV file to JSON format.

Create csv-transform-job.yaml:

name: csv-to-json
description: Transform CSV users file to JSON
type: pipeline
config:
  input:
    file:
      paths: ["./users.csv"]
      codec: lines

  pipeline:
    processors:
      - mapping: |
          root = if @message_number == 0 { deleted() }

          let parts = this.split(",")
          root = {
            "name": $parts.index(0),
            "email": $parts.index(1),
            "age": $parts.index(2).number()
          }

  output:
    file:
      path: ./users.json
      codec: lines

HTTP Webhook Receiver​

Listen for incoming webhook requests and process them.

Create http-receiver-job.yaml:

name: webhook-receiver
description: Receive and process webhook data
type: pipeline
config:
  input:
    http_server:
      address: "0.0.0.0:8081"
      path: /webhook

  pipeline:
    processors:
      - mapping: |
          root = this.parse_json()
          root.processed_at = now()

  output:
    stdout:
      codec: lines

Test it:

# Terminal 1: Start edge
expanso-edge run --local

# Terminal 2: Deploy job
export EXPANSO_CLI_ENDPOINT=http://localhost:9010
expanso-cli job deploy http-receiver-job.yaml

# Terminal 3: Send test data
curl -X POST http://localhost:8081/webhook \
  -H "Content-Type: application/json" \
  -d '{"event":"test","data":"hello"}'

Common Workflows​

Development Iteration​

# Terminal 1: Start edge once
expanso-edge run --local -v

# Terminal 2: Iterate on pipelines
export EXPANSO_CLI_ENDPOINT=http://localhost:9010
expanso-cli job deploy v1-pipeline.yaml
# ... test, update file ...
expanso-cli job deploy v2-pipeline.yaml

Multiple Pipelines​

Run several pipelines at once:

expanso-cli job deploy pipeline-a.yaml
expanso-cli job deploy pipeline-b.yaml
expanso-cli job list

Next Steps​