Skip to content

Comparison to other approaches

polspec sits at the intersection of two things usually solved by separate tools: generating test data, and validating that data against a schema. This page is about that intersection — what generating and validating from one declaration buys you that the alternatives don't, where those alternatives are still the better tool, and the actual numbers behind the speed claim.

Benchmarks

benchmarks/bench.py compare generates the same four-column frame — a non-nullable string, a nullable enum, a nullable bounded int, a nullable bounded float — three ways: polspec's Rust generator, a hand-vectorized NumPy implementation, and a pure-Python loop using random. All three produce an equivalent pl.DataFrame, so the comparison is "how fast can each approach hand back a usable frame," not raw loop speed in isolation.

n_rows polspec (Rust) NumPy Python
1,000 0.0002s 0.0006s 0.0014s
10,000 0.0003s 0.0049s 0.0140s
100,000 0.0020s 0.0482s 0.1483s
1,000,000 0.0067s 0.4874s 1.4815s
5,000,000 0.0288s 2.4138s skipped
20,000,000 0.1130s 9.7606s skipped

Measured 2026-11-16 with polspec 1.6 (before release) on an Intel 13900K with 64GB DDR5; yours will differ, and the shape matters more than the absolute numbers. Three things worth reading off it:

  • The gap widens with size, not just the ratio. At 1,000 rows all three are fast enough that the difference doesn't matter to a test suite. At 20,000,000, pure Python is impractical (skipped past a 5-second cutoff at a much smaller size) and NumPy's 1–2 million rows/second becomes a real wait in a CI loop, while polspec is still around a tenth of a second.
  • NumPy's implementation is the hard-won version. Its string column uses a fixed-width byte-array trick because NumPy has no efficient way to vectorize ragged per-row lengths — the other two implementations generate strings 5–15 characters long; NumPy's are fixed at 15 and decoded back down. That's not a knock on NumPy — it's the actual cost of writing this by hand: the fast version needs a specific trick per dtype, and someone has to know it.

  • A benchmark is a measurement of a machine, not only of code. Every number here is the fastest of several runs, each in a process of its own so that one case cannot leave the allocator warm for the next, and the run records the CPU, the thread count, the Polars version and the cargo profile beside the timings. That last one matters more than it sounds: building the extension as a single codegen unit moves the unique path by a factor of two on its own.

Reproduce it yourself, from a checkout of the repository:

uv run --group bench python benchmarks/bench.py compare

The same harness guards against regressions. record writes a baseline for the machine you are on, and check re-measures every case — each column kind, both branches of the unique draw, the cartesian and rule and foreign-key passes, the sinks — and exits non-zero if one has regressed:

uv run --group bench python benchmarks/bench.py record   # before a change
uv run --group bench python benchmarks/bench.py check    # after it

Validation

Validation compiles every check across every column into one Polars aggregation. benchmarks/validation.py validates the same two valid frames with polspec, dataframely and pandera: orders, and the customers they belong to, one for every ten orders. The claims are the same for each tool: a unique id, a key into the customers (their own id unique), a nullable column of three values, a nullable bounded float, a pattern and a bounded integer.

Each tool is written as its documentation writes it:

  • polspec: a Registry over two FrameSpecs.
  • dataframely: a Collection whose filter keeps the orders of known customers. A collection needs a primary key its members share, so an order's is its customer and its own id, and checking it is part of dataframely's time.
  • pandera: two DataFrameSchemas, with the key as a custom check.

A lazy run lasts until the validated frames are collected, since dataframely validates a lazy frame when it is collected; pandera's is asked to check the data as well as the schema. Every run was checked to fail on each claim broken. The times are the fastest of five runs, after one to warm up.

Orders polspec eager polspec lazy dataframely eager dataframely lazy pandera eager pandera lazy
1,000,000 0.0318s 0.0283s 0.0461s 0.0463s 0.0560s 0.0556s
10,000,000 0.3538s 0.3569s 0.6701s 0.6859s 0.7371s 0.6704s

Measured 2026-11-16 with polspec 1.6 (before release), Polars 1.44.2, dataframely 3.1.2 and pandera 0.36.0, on the same machine as above. All three tools hand the work to Polars, so the gap is narrow: polspec is between 1.4 and 2.1 times as fast. It is the gap between one aggregation and several passes, not between Rust and Python. A different frame — more columns, more patterns, failures to report — will move it. Rerun it:

uv run --group bench python benchmarks/validation.py
uv run --group bench python benchmarks/validation.py --rows 10000000

Compared to hand-written fixtures

The common alternative is a Python dict or list literal, copy-pasted between test files and edited by hand when the shape needs to change:

def make_customer_row(customer_id=1, tier="free"):
    return {"customer_id": customer_id, "tier": tier, "signed_up": "2023-01-01"}

This works, and for a handful of fixed cases it's often the right amount of machinery. It stops working as the schema grows: the dtype lives nowhere — tier being one of three strings is enforced by nobody until something downstream breaks — and every edge case (a null, a boundary value, a specific combination of two columns) is a row someone remembered to write by hand. There's also nothing stopping the fixture and the real schema from drifting apart; the dict doesn't know the pipeline added a column last month.

A ColSpec declaration is both the definition and the generator: the dtype, the bound, and the domain are enforced the same way whether you're generating data or checking it, and method="cartesian" covers the boundary/null cases that hand-written fixtures tend to under-cover because nobody thought to write them.

Compared to Faker and similar

Faker and libraries built on it are the right tool for semantically realistic values — names that look like names, addresses that parse like addresses, emails with plausible domains. polspec doesn't try to compete there: its strings are bounded-length ASCII, not locale-aware people or places, because it's solving a different problem — statistically-shaped data that respects a schema, not human-plausible data that respects cultural conventions.

The two combine rather than compete. A Faker-generated pool of realistic values becomes a ColSpec.choices list; polspec supplies the bounds, nullability, cross-column rules, and cross-table referential integrity that sit around it:

import polars as pl
from faker import Faker
from polspec import ColSpec, FrameSpec

fake = Faker()
first_names = list({fake.first_name() for _ in range(500)})  # choices must be distinct

class Customers(FrameSpec):
    name = ColSpec(pl.String, choices=first_names)
    signup_bonus = ColSpec(pl.Float64, bounds=(0.0, 50.0))

What Faker doesn't do on its own is hand back a typed pl.DataFrame, enforce a bound, or keep a foreign key consistent across two generated tables — those are the parts of the problem polspec is actually for.

Compared to NumPy or a bespoke script

The benchmark above is this comparison: a hand-written NumPy implementation is faster than pure Python and can be made fast enough for most purposes, but someone has to write it, and it has to be rewritten — bounds, nullability, dtype casts — for every new column and every schema change. There's also nothing left over afterward: the script that generated the data has no relationship to a validator that checks it, because there was never a shared declaration for the two to share.

polspec's Rust generator is faster than a hand-written NumPy version because it doesn't pay Python's per-call overhead and fills columns in parallel — but the bigger difference for day-to-day use is that the declaration doesn't have to be rewritten by hand for each column, and the same one both generates and validates.

Compared to property-based testing (Hypothesis)

Hypothesis explores the space of possible inputs and, when one fails, shrinks it to the smallest failing case. polspec does not search on its own: method="cartesian" is a fixed, deterministic set of known-important combinations (every enum value, every numeric sign, null), not an open-ended exploration.

The two combine. polspec.strategy(Orders) is a Hypothesis strategy of frames the spec accepts: Hypothesis draws a row count and a seed, and shrinks a failure to the fewest rows that still fail. The values themselves come from the seed and do not shrink one by one.

Compared to dataframely, pandera and Pointblank

These are the nearest tools: each validates Polars frames, and dataframely and Pointblank generate data from a schema too. dataframely samples a frame that meets its schema and rules, and related tables as a collection. Pointblank generates realistic names and addresses for 100 countries, and strings from a regex, and reports validation with thresholds and actions. pandera's data synthesis is not yet available for Polars.

polspec's ground among them is generation at volume, in Rust, with cross-table keys, lazy output and file sinks, and the spec's life after it is written: profiling it from data, and drift between it and the data or another version of it. It has no typed frames, and leaves reporting to the caller. Coming from another tool has a feature table, a concept map and one schema translated from each; it was checked against their documentation on 2026-11-15.

Compared to data-quality platforms (Great Expectations, ...)

These are built around a different center of gravity: validating, profiling, and monitoring data that already exists — often production tables, with scheduling, history and dashboards. That is a larger and more operational surface than polspec's validate() and drift(), which answer when asked.

FrameSpec is meant to be small enough to declare once and use for both jobs in a test suite, not to replace a data-quality platform watching a production warehouse.

Patito is close in spirit: a Polars frame declared as a Pydantic model, validated against it, and able to fill in example rows for a test from the values you give. polspec's declaration is not a Pydantic model, and its generation goes further — whole frames at volume, from distributions, weights and rules, with foreign keys kept across tables — while generate(fixed=) covers the give-some-values, fill-the-rest case.

What polspec doesn't try to be

Worth being direct about, in the same spirit as the known limitations and stability pages:

  • Not a realistic-fake-data library. No locales, no plausible names or addresses out of the box — pair it with Faker for that.
  • Not a data-quality or monitoring platform. drift() measures a frame against its declaration and says what moved, but there are no dashboards, no scheduling, no history kept between runs and no anomaly scoring — polspec answers when asked.
  • Not a property-based testing engine. method="cartesian" is a fixed set of known-important cases; for search and shrinking, hand strategy to Hypothesis.
  • Not everything that validates generates. __checks__ and ColSpec.validators are validated but not generated, by design — see Known limitations.

Where it fits

Polars-native pipelines that need fast, schema-shaped synthetic data and matching validation from one declaration — especially across several related tables via ForeignKey, at volumes where a pure-Python or pandas generator starts to cost real CI time, in tests that need to stay hermetic. See Testing pipelines for that in practice.