Getting Started

Installation

Using pip

pip install glassgen

Local Development Installation

Clone the repository:

git clone https://github.com/glassflow/glassgen.git
cd glassgen

Create and activate a virtual environment:

python -m venv venv
source venv/bin/activate  # On Windows use: venv\Scripts\activate

Install the package in development mode:

pip install -e .

Install development dependencies:

pip install -r requirements-dev.txt

Run tests to verify installation:

pytest

Basic Usage

Python SDK

Here's a simple example of using GlassGen to generate user data and save it to a CSV file:

import glassgen
 
# Define your configuration
config = {
    "schema": {
        "name": "$name",        
        "email": "$email",
        "country": "$country",
        "id": "$uuid",
        "address": "$address",
        "phone": "$phone_number",
        "job": "$job",
        "company": "$company"
    },
    "sink": {
        "type": "csv",
        "params": {
            "path": "output.csv"
        }
    },
    "generator": {
        "rps": 1500,  # records per second
        "num_records": 5000  # total number of records to generate
    }
}
 
# Start the generator
glassgen.generate(config=config)

This will:

Generate 5000 records of user data
Create records at a rate of 1500 records per second
Save the data to a file named output.csv

Configuration File

You can also load the configuration from a JSON file:

import glassgen
import json
 
# Load configuration from file
with open("config.json") as f:
    config = json.load(f)
 
# Start the generator
glassgen.generate(config=config)

The configuration file (config.json) should follow this structure:

{
    "schema": {
        "field1": "$generator_type",
        "field2": "$generator_type(param1, param2)"
    },
    "sink": {
        "type": "csv|kafka|webhook",
        "params": {
            // sink-specific parameters
        }
    },
    "generator": {
        "rps": 1000,  // records per second
        "num_records": 5000  // total number of records to generate
    }
}

Next Steps

Introduction Generators