polspec¶

Declare a Polars schema once. Generate data that matches it, and validate data against it — from the same declaration.

import polars as pl
from polspec import ColSpec, FrameSpec

class Orders(FrameSpec):
    order_id = ColSpec(pl.Int64, bounds=(1, None))
    status   = ColSpec(pl.Enum(["NEW", "PAID", "SHIPPED"]))
    total    = ColSpec(pl.Float64, bounds=(0.0, None))
    placed   = ColSpec(pl.Date, nullable=True)

df = Orders.generate(1_000_000, seed=42)   # a million rows in well under a second
Orders.validate(df)                        # raises ValidationError on any breach

The generator is written in Rust and runs the columns in parallel, so a spec that describes a realistic table produces millions of rows in the time it takes to describe one.

Why two directions from one declaration¶

Most schema tools do one or the other. A validation library tells you when production data drifted; a fixture library gives you something to test against. Keeping both behind one declaration means the fixtures and the contract cannot disagree — and where they might, polspec has a test suite whose whole job is to catch it (see Known limitations).

That is the practical payoff: the data in your tests is data your validator already accepts, so a test that passes locally is not passing on a shape production will reject.

What you can declare¶

Types and shape

Every dtype, generated and validated — integers, floats, Decimal, booleans, strings, binary, all four temporal types, Enum, Categorical, and a List, Array or Struct of any of them, nested to any depth — plus nullability, bounds, string lengths, value domains and named string formats such as uuid4 and email.

Declaring columns · String formats
Rules and invariants

Conditional values, single-column validators, multi-column checks, composite uniqueness and foreign keys between specs.

Constraints
Data on demand

Random or coverage-guaranteeing generation, reproducible seeds, and a LazyFrame that generates only the columns and rows a plan asks for — streaming straight to Parquet, CSV, Arrow IPC or NDJSON.

Generating data
Specs from elsewhere

Infer a spec by profiling an existing DataFrame, or load one from YAML so non-Python tooling can read it too.

YAML specs
What changed

Diff two versions of a spec, or a spec against data, into a report that says what moved and whether a frame that passed before could fail now.

Drift
From the command line

polspec generate writes a data file from a spec, polspec validate checks one, polspec diff and polspec drift gate a pull request on what changed, and polspec test turns a schema into a pytest round-trip.

Command line

Install¶

uv add polspec           # preferred
pip install polspec      # alternative

The generator is a compiled Rust extension, but wheels are published for Linux (x86_64, aarch64), macOS (Intel and Apple silicon) and Windows (x86_64), so installing needs no Rust toolchain. Nothing beyond Polars is needed at runtime.

Building from a checkout — which does need Rust and maturin — is covered in CONTRIBUTING.md.

Where to go next¶

Start with Getting started for the full loop — declare, generate, validate — in about five minutes. If you're weighing polspec against a hand-rolled fixture, Faker, or a data-quality framework, see Comparison to other approaches for where each one fits and the benchmark numbers behind the speed claim.

For language models¶

The documentation is published in the llms.txt format: /llms.txt indexes every page, and /llms-full.txt carries the full text of all of them -- including the API reference, expanded to signatures and docstrings -- in one file.