Skip to main content

Basic K3s Log Collection

Stream logs from all pods and namespaces to S3 with batching.

Pipeline​

input:
  subprocess:
    name: kubectl
    args:
      - logs
      - --all-containers=true
      - --prefix=true
      - --follow
      - --tail=-1
      - --all-namespaces
    codec: lines
    restart_on_exit: true

pipeline:
  processors:
    - mapping: |
        root.raw_log = this
        root.timestamp = now()
        root.node_id = env("NODE_ID")
        root.cluster = env("CLUSTER_NAME").or("k3s-edge")

output:
  aws_s3:
    bucket: edge-k3s-logs
    path: 'logs/${! env("NODE_ID") }/${! timestamp_unix() }.jsonl'
    batching:
      count: 1000
      period: 1m
    processors:
      - archive:
          format: concatenate

What This Does​

  • Follows logs from all containers in all namespaces using kubectl logs --follow
  • Adds metadata: node identifier, cluster name, and timestamp to each log entry
  • Batches logs: Collects 1000 logs or waits 1 minute before writing
  • Writes to S3: Organizes logs by node and timestamp for easy retrieval
  • Auto-restarts: If kubectl process exits, it automatically restarts

Key Configuration​

--all-containers=true: Includes logs from all containers in each pod

--prefix=true: Adds [namespace/pod/container] prefix to each log line

--follow: Continuously streams new logs (like tail -f)

restart_on_exit: true: Ensures log collection continues even if kubectl crashes

Batching: Reduces S3 API calls by writing 1000 logs at once instead of individual files

Environment Variables​

Set these environment variables where Expanso runs:

  • NODE_ID: Unique identifier for this edge node (e.g., edge-site-42)
  • CLUSTER_NAME: Name of the K3s cluster (optional, defaults to k3s-edge)

Next Steps​