Example Pipelines

CSV Sales Report

Aggregate sales by region, sorted by total.

{
  "name": "sales-summary",
  "version": "1.0.0",
  "steps": [
    { "operator": "csv.parse", "params": { "header": true } },
    { "operator": "aggregate.groupBy", "params": { "key": "region", "aggregate": "sum", "aggregateKey": "amount" } },
    { "operator": "transform.sort", "params": { "key": "sum", "order": "desc" } }
  ]
}

CLI

json-toolbox run sales-summary.json < sales.csv

NDJSON Log Analysis

Filter errors and count by service.

{
  "name": "ndjson-log-analysis",
  "version": "1.0.0",
  "steps": [
    { "operator": "ndjson.parse" },
    { "operator": "transform.filter", "params": { "key": "level", "operator": "eq", "value": "error" } },
    { "operator": "aggregate.groupBy", "params": { "key": "service", "aggregate": "count" } },
    { "operator": "transform.sort", "params": { "key": "count", "order": "desc" } }
  ]
}

CLI One-liner

cat logs.ndjson | \
  json-toolbox exec ndjson.parse | \
  json-toolbox exec transform.filter --key level --operator eq --value error | \
  json-toolbox exec aggregate.groupBy --key service --aggregate count

Join Orders with Users

Left join order data with user names.

{
  "name": "join-orders-users",
  "version": "1.0.0",
  "steps": [
    { "operator": "json.parse" },
    { "operator": "join.left", "params": {
        "rightFile": "users.json",
        "leftKey": "user_id",
        "rightKey": "id"
      }
    },
    { "operator": "transform.map", "params": {
        "fields": ["order_id", "product", "name"]
      }
    }
  ]
}

CLI

json-toolbox exec join.left \
  --right users.json \
  --leftKey user_id \
  --rightKey id < orders.json

CSV to YAML Config

Convert CSV config to YAML format.

{
  "name": "csv-to-yaml",
  "version": "1.0.0",
  "steps": [
    { "operator": "csv.parse", "params": { "header": true } },
    { "operator": "yaml.stringify", "params": { "indent": 2 } }
  ]
}

Data Cleanup Pipeline

Repair JSON, remove duplicates, sort.

{
  "name": "data-cleanup",
  "version": "1.0.0",
  "steps": [
    { "operator": "fix.repair" },
    { "operator": "json.parse" },
    { "operator": "transform.unique", "params": { "key": "id" } },
    { "operator": "transform.sort", "params": { "key": "name" } }
  ]
}

Streaming Large NDJSON

Process large files with minimal memory.

# Stream mode with progress
json-toolbox exec transform.filter \
  --key status --operator eq --value active \
  --stream --chunk 5000 --progress < huge.ndjson

# Head first 100 lines only
json-toolbox exec ndjson.head --count 100 --stream < huge.ndjson

Generate JSON Schema

Infer schema from sample data.

echo '[{"id":1,"name":"Alice","active":true}]' | \
  json-toolbox exec schema.generate

Output

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "id": { "type": "integer" },
      "name": { "type": "string" },
      "active": { "type": "boolean" }
    },
    "required": ["id", "name", "active"]
  }
}