Comparison to other approaches¶
polspec sits at the intersection of two things usually solved by separate tools: generating test data, and validating that data against a schema. This page is about that intersection — what generating and validating from one declaration buys you that the alternatives don't, where those alternatives are still the better tool, and the actual numbers behind the speed claim.
Benchmarks¶
benchmarks/bench.py compare generates the same four-column frame — a
non-nullable string, a nullable enum, a nullable bounded int, a nullable
bounded float — three ways: polspec's Rust generator, a hand-vectorized NumPy
implementation, and a pure-Python loop using random. All three produce an
equivalent pl.DataFrame, so the comparison is "how fast can each approach
hand back a usable frame," not raw loop speed in isolation.
| n_rows | polspec (Rust) | NumPy | Python |
|---|---|---|---|
| 1,000 | 0.0002s | 0.0006s | 0.0014s |
| 10,000 | 0.0003s | 0.0049s | 0.0140s |
| 100,000 | 0.0020s | 0.0482s | 0.1483s |
| 1,000,000 | 0.0067s | 0.4874s | 1.4815s |
| 5,000,000 | 0.0288s | 2.4138s | skipped |
| 20,000,000 | 0.1130s | 9.7606s | skipped |
Measured 2026-11-16 with polspec 1.6 (before release) on an Intel 13900K with 64GB DDR5; yours will differ, and the shape matters more than the absolute numbers. Three things worth reading off it:
- The gap widens with size, not just the ratio. At 1,000 rows all three are fast enough that the difference doesn't matter to a test suite. At 20,000,000, pure Python is impractical (skipped past a 5-second cutoff at a much smaller size) and NumPy's 1–2 million rows/second becomes a real wait in a CI loop, while polspec is still around a tenth of a second.
-
NumPy's implementation is the hard-won version. Its string column uses a fixed-width byte-array trick because NumPy has no efficient way to vectorize ragged per-row lengths — the other two implementations generate strings 5–15 characters long; NumPy's are fixed at 15 and decoded back down. That's not a knock on NumPy — it's the actual cost of writing this by hand: the fast version needs a specific trick per dtype, and someone has to know it.
-
A benchmark is a measurement of a machine, not only of code. Every number here is the fastest of several runs, each in a process of its own so that one case cannot leave the allocator warm for the next, and the run records the CPU, the thread count, the Polars version and the cargo profile beside the timings. That last one matters more than it sounds: building the extension as a single codegen unit moves the
uniquepath by a factor of two on its own.
Reproduce it yourself, from a checkout of the repository:
The same harness guards against regressions. record writes a baseline for
the machine you are on, and check re-measures every case — each column kind,
both branches of the unique draw, the cartesian and rule and foreign-key
passes, the sinks — and exits non-zero if one has regressed:
uv run --group bench python benchmarks/bench.py record # before a change
uv run --group bench python benchmarks/bench.py check # after it
Validation¶
Validation compiles every check across every
column into one Polars aggregation. benchmarks/validation.py validates the
same two valid frames with polspec, dataframely and pandera: orders, and the
customers they belong to, one for every ten orders. The claims are the same
for each tool: a unique id, a key into the customers (their own id unique),
a nullable column of three values, a nullable bounded float, a pattern and
a bounded integer.
Each tool is written as its documentation writes it:
- polspec: a
Registryover twoFrameSpecs. - dataframely: a
Collectionwhose filter keeps the orders of known customers. A collection needs a primary key its members share, so an order's is its customer and its own id, and checking it is part of dataframely's time. - pandera: two
DataFrameSchemas, with the key as a custom check.
A lazy run lasts until the validated frames are collected, since dataframely validates a lazy frame when it is collected; pandera's is asked to check the data as well as the schema. Every run was checked to fail on each claim broken. The times are the fastest of five runs, after one to warm up.
| Orders | polspec eager | polspec lazy | dataframely eager | dataframely lazy | pandera eager | pandera lazy |
|---|---|---|---|---|---|---|
| 1,000,000 | 0.0318s | 0.0283s | 0.0461s | 0.0463s | 0.0560s | 0.0556s |
| 10,000,000 | 0.3538s | 0.3569s | 0.6701s | 0.6859s | 0.7371s | 0.6704s |
Measured 2026-11-16 with polspec 1.6 (before release), Polars 1.44.2, dataframely 3.1.2 and pandera 0.36.0, on the same machine as above. All three tools hand the work to Polars, so the gap is narrow: polspec is between 1.4 and 2.1 times as fast. It is the gap between one aggregation and several passes, not between Rust and Python. A different frame — more columns, more patterns, failures to report — will move it. Rerun it:
uv run --group bench python benchmarks/validation.py
uv run --group bench python benchmarks/validation.py --rows 10000000
Compared to hand-written fixtures¶
The common alternative is a Python dict or list literal, copy-pasted between test files and edited by hand when the shape needs to change:
def make_customer_row(customer_id=1, tier="free"):
return {"customer_id": customer_id, "tier": tier, "signed_up": "2023-01-01"}
This works, and for a handful of fixed cases it's often the right amount of
machinery. It stops working as the schema grows: the dtype lives nowhere —
tier being one of three strings is enforced by nobody until something
downstream breaks — and every edge case (a null, a boundary value, a specific
combination of two columns) is a row someone remembered to write by hand.
There's also nothing stopping the fixture and the real schema from drifting
apart; the dict doesn't know the pipeline added a column last month.
A ColSpec declaration is both the definition and the generator: the dtype,
the bound, and the domain are enforced the same way whether you're generating
data or checking it, and method="cartesian"
covers the boundary/null cases that hand-written fixtures tend to under-cover
because nobody thought to write them.
Compared to Faker and similar¶
Faker and libraries built on it are the right tool for semantically realistic values — names that look like names, addresses that parse like addresses, emails with plausible domains. polspec doesn't try to compete there: its strings are bounded-length ASCII, not locale-aware people or places, because it's solving a different problem — statistically-shaped data that respects a schema, not human-plausible data that respects cultural conventions.
The two combine rather than compete. A Faker-generated pool of realistic
values becomes a ColSpec.choices list; polspec supplies the bounds,
nullability, cross-column rules, and cross-table referential integrity that
sit around it:
import polars as pl
from faker import Faker
from polspec import ColSpec, FrameSpec
fake = Faker()
first_names = list({fake.first_name() for _ in range(500)}) # choices must be distinct
class Customers(FrameSpec):
name = ColSpec(pl.String, choices=first_names)
signup_bonus = ColSpec(pl.Float64, bounds=(0.0, 50.0))
What Faker doesn't do on its own is hand back a typed pl.DataFrame, enforce
a bound, or keep a foreign key consistent across two generated tables — those
are the parts of the problem polspec is actually for.
Compared to NumPy or a bespoke script¶
The benchmark above is this comparison: a hand-written NumPy implementation is faster than pure Python and can be made fast enough for most purposes, but someone has to write it, and it has to be rewritten — bounds, nullability, dtype casts — for every new column and every schema change. There's also nothing left over afterward: the script that generated the data has no relationship to a validator that checks it, because there was never a shared declaration for the two to share.
polspec's Rust generator is faster than a hand-written NumPy version because it doesn't pay Python's per-call overhead and fills columns in parallel — but the bigger difference for day-to-day use is that the declaration doesn't have to be rewritten by hand for each column, and the same one both generates and validates.
Compared to property-based testing (Hypothesis)¶
Hypothesis explores the space of
possible inputs and, when one fails, shrinks it to the smallest failing
case. polspec does not search on its own: method="cartesian" is a fixed,
deterministic set of known-important combinations (every enum value, every
numeric sign, null), not an open-ended exploration.
The two combine. polspec.strategy(Orders)
is a Hypothesis strategy of frames the spec accepts: Hypothesis draws a row
count and a seed, and shrinks a failure to the fewest rows that still fail.
The values themselves come from the seed and do not shrink one by one.
Compared to dataframely, pandera and Pointblank¶
These are the nearest tools: each validates Polars frames, and dataframely and Pointblank generate data from a schema too. dataframely samples a frame that meets its schema and rules, and related tables as a collection. Pointblank generates realistic names and addresses for 100 countries, and strings from a regex, and reports validation with thresholds and actions. pandera's data synthesis is not yet available for Polars.
polspec's ground among them is generation at volume, in Rust, with cross-table keys, lazy output and file sinks, and the spec's life after it is written: profiling it from data, and drift between it and the data or another version of it. It has no typed frames, and leaves reporting to the caller. Coming from another tool has a feature table, a concept map and one schema translated from each; it was checked against their documentation on 2026-11-15.
Compared to data-quality platforms (Great Expectations, ...)¶
These are built around a different center of gravity: validating,
profiling, and monitoring data that already exists — often production
tables, with scheduling, history and dashboards. That is a larger and more
operational surface than polspec's validate() and drift(), which answer
when asked.
FrameSpec is meant to be small enough to declare once and use for both
jobs in a test suite, not to replace a data-quality platform watching a
production warehouse.
Patito is close in spirit: a
Polars frame declared as a Pydantic model, validated against it, and able to
fill in example rows for a test from the values you give. polspec's
declaration is not a Pydantic model, and its generation goes further — whole
frames at volume, from distributions, weights and rules, with foreign keys
kept across tables — while
generate(fixed=) covers
the give-some-values, fill-the-rest case.
What polspec doesn't try to be¶
Worth being direct about, in the same spirit as the known limitations and stability pages:
- Not a realistic-fake-data library. No locales, no plausible names or addresses out of the box — pair it with Faker for that.
- Not a data-quality or monitoring platform.
drift()measures a frame against its declaration and says what moved, but there are no dashboards, no scheduling, no history kept between runs and no anomaly scoring — polspec answers when asked. - Not a property-based testing engine.
method="cartesian"is a fixed set of known-important cases; for search and shrinking, handstrategyto Hypothesis. - Not everything that validates generates.
__checks__andColSpec.validatorsare validated but not generated, by design — see Known limitations.
Where it fits¶
Polars-native pipelines that need fast, schema-shaped synthetic data and
matching validation from one declaration — especially across several related
tables via ForeignKey, at volumes where a pure-Python or pandas generator
starts to cost real CI time, in tests that need to stay hermetic. See
Testing pipelines for that in practice.