NDJSON Sink

The NDJSON sink writes generated data to a Newline Delimited JSON file — one JSON object per line. It’s useful for:

Streaming data pipelines that process records line by line
Log ingestion systems (Elasticsearch, Loki, Fluent Bit)
Tools that expect NDJSON input (BigQuery, ClickHouse JSONEachRow, jq with -c)
Preserving nested structures without flattening (unlike the CSV sink)

Configuration


{
  "sink": {
    "type": "ndjson",
    "params": {
      "path": "output.ndjson"
    }
  }
}

Configuration Options

path (required): The path where the NDJSON file will be written

Examples

Basic Usage


{
  "schema": {
    "id": "$uuid",
    "name": "$name",
    "email": "$email",
    "age": "$intrange(18,65)"
  },
  "sink": {
    "type": "ndjson",
    "params": {
      "path": "users.ndjson"
    }
  },
  "generator": {
    "num_records": 1000
  }
}

Nested Objects

Unlike the CSV sink, the NDJSON sink preserves nested structures as-is:


{
  "schema": {
    "event_id": "$uuid",
    "timestamp": "$timestamp",
    "user": {
      "id": "$uuid",
      "name": "$name",
      "email": "$email"
    },
    "metadata": {
      "ip": "$ipv4",
      "country": "$country"
    }
  },
  "sink": {
    "type": "ndjson",
    "params": {
      "path": "events.ndjson"
    }
  }
}

Output Example

Each line is a valid, self-contained JSON object:


{"id":"550e8400-e29b-41d4-a716-446655440000","name":"John Doe","email":"[email protected]","age":32}
{"id":"6ba7b810-9dad-11d1-80b4-00c04fd430c8","name":"Jane Smith","email":"[email protected]","age":28}