Skip to content

Changelog

All notable changes to polspec are recorded here. The format follows Keep a Changelog. From 1.0, the public API, the spec-file format, seeded output, the command line and the finding and drift codes change only as the Stability page promises; before 1.0, a minor version could break any of them.

Unreleased

1.7.0 - 2026-11-17

Generated data that keeps its checks, and validation that tolerates a little. generate(checks=True) keeps only the rows a spec's checks pass, so fake data no longer fails its own spec; @check declares a check as a function. tolerate= lets a finding cover a share of the rows and still pass, from Python or polspec validate --tolerate. polspec.formats() lists every string format, synthesize can return the spec it generated from, and a TableSpec asked for a verb names the function to call. Text of whole numbers followed by decimals now reads as Float64. The docs gain Performance and How the pieces fit.

Everything is additive under the 1.0 promises. No seeded output moves: checks=True is off by default, and a frame whose rows all pass is the frame 1.6 generated. Profiles change where a column of numbers written as text is now read as Float64, which Stability leaves open. The JSON report gains a tolerated key on each finding. Nothing is deprecated. The spec-file format stays at version 4, and 1.6 reads 1.7's files.

Added

  • checks=True keeps a spec's checks in the data it generates. __checks__ and column validators were validation-only, so a generated frame could fail its own spec. With checks=True, generate, generate_batches, scan, every sink_* and a registry's generate_all, generate_related and scan_all keep the first rows of the seed's frame that pass every check, reading on for more when some fail. A unique column stays unique, across batches too, and a frame whose rows all pass is the frame checks=False gives. method="cartesian", a hierarchy, a key into the spec's own rows and a pin of all n values are refused, and a check that rejects nearly every row is a GenerationError naming it. Off by default, so no seeded output moves.
  • @check declares a check as a function on a FrameSpec class, named after the function: it returns a col() predicate or a Polars expression, and joins __checks__.
  • tolerate= accepts a little bad data. validate, inspect and their facades take a share of the rows that a finding may cover and still pass: one for every finding, or by finding code, column, or (column, code). A tolerated finding stays in the report, marked Finding.tolerated; ValidationReport.failures and .tolerated split them, and passed counts failures only. polspec validate --tolerate [KEY=]SHARE takes the same, with 0.1% for a share. The JSON report gains a tolerated key on each finding.
  • polspec.formats() lists every string format as a frame: its name, a summary, whether it is a finite list, an example value, and a finite format's values.
  • synthesize_all(return_registry=True) and synthesize(return_spec=True) return what the fake data was generated from. A larger fake set widens its key ranges, so profile_all's registry refused it on bounds; the one returned accepts it.

Changed

  • A TableSpec asked for a verb names the function. polspec.profile(df) .validate(df) raised a bare AttributeError; a spec is a value and its verbs are functions over it, so it now says to call polspec.validate(spec, ...), or wrap it with FrameSpec.from_spec.
  • Docs: Performance, a new how-to page with what each choice costs — generating, streaming, keeping checks, validating a lazy frame, profiling a sample — measured. Generate data and Validate data show what their examples print, and Multiple specs declares the key its example relied on.
  • Docs: How the pieces fit, a new explanation page with one diagram: the declarations, verbs as functions, what generation keeps, why schema infer and profile() have different defaults, and seeds.
  • schema infer and synthesize describe their shared flags alike. --max-unique-enum, --sample and --replace read the same on both commands' --help, and synthesize lists them in schema infer's order. What each flag does is unchanged.

Fixed

  • Text of whole numbers then decimals reads as Float64. Text is tried on its first 1,000 values first, and a column whose first values were whole and whose decimals came later stayed text, saying its values were too large for an Int64 and quoting the decimals. Every later type is now tried on the whole column, and that reason is given only for whole numbers an Int64 cannot hold.

1.6.1 - 2026-11-16

Every choice and its share, in the data dictionary. to_markdown can now document a long list of choices whole, in an appendix or in its cell, for a table of many Enum columns whose weights the summary cut short.

Additive under the 1.0 promises: the default writes exactly what 1.6.0 wrote, no seeded output moves, and the spec-file format is unchanged.

Added

  • to_markdown(choices=) shows every choice and its share. A column of more than four choices still shows its first three and its count; choices="appendix" adds a Choices section listing every value and share of each such column, in one table, and choices="full" lists them all in the cell. The default, "summary", writes what 1.6.0 wrote. FrameSpec.to_markdown takes it too.

1.6.0 - 2026-11-16

Faster where it was slow, and dates as they are written. Small batches, scans and sinks stop paying for a whole engine chunk each: a million rows in batches of 1,000 take 12 ms, not 897 ms. List, Map and Decimal columns are built in the engine, a unique parent is read as it stands, validation skips a check an Enum cannot fail, and profiling runs its columns at once. profile(parse_text=True) reads dates written as 30 Sep 2031 or 30/09/2031, keeps the format as text_format, and writes fake files in it; a column that nearly parses says why it stays text. to_markdown shows an Enum's choices and shares, and Registry.replace swaps one spec.

Everything is additive under the 1.0 promises. No seeded output moves: batched and generated frames hold exactly the values they held in 1.5, checked case by case across chunk boundaries. Profiles change where a column of dates written as text is now read, or now says why it stays text, which Stability leaves open, as it does a repr and a rendered document. Nothing is deprecated. The spec-file format stays at version 4: a text_format key is written only when used, and 1.5 refuses it by name.

Added

  • Dates, datetimes and times written as text are read in the form they are written. profile(parse_text=True) reads 30 Sep 2031, 30 September 2031, 30-Sep-2031, Sep 30, 2031, 30/09/2031, 09/30/2031, 2031/09/30 and 30.09.2031, unpadded too, any of them with a time, and a time alone (14:05, 2:05 PM), where only ISO was read before. A format fits when every value parses with it and writes back as itself, so 30 Feb 2031 fits nothing. When two formats fit and read the values differently, as day-first and month-first do for 01/02/2031, the column stays text and says which: polspec does not guess.
  • ColSpec(text_format=...): how a Date, Datetime or Time column is written as text, as a strftime string ("%d %b %Y"). Profiling writes it for a column read that way; validate(parse_text=True) and read() read the column with it, and a value written otherwise is a dtype finding that names it; a CSV or NDJSON file polspec writes holds the column in it, from the sink_* verbs and from polspec generate, synthesize and validate --output. In memory, and in Parquet and IPC, a date stays a date. A directive the column cannot be written with is refused where it is declared. A new optional key on spec-file version 4, written only when set; 1.5 refuses a file that uses it by name.
  • A column that nearly parses says why it stays text. With parse_text=True, a column of nearly numbers or nearly dates is described as Stays text: ...: the values that do not parse ('1,500,000.00', 'N/A'), a value written with a leading zero, a whole number past an Int64, or the two date formats that both fit. schema infer prints the same line. Without parse_text, a column that would parse says in which format, and takes no shape pattern. Numbers with thousands separators or decimal commas are left to the caller to clean, and the docs show how.
  • Registry.replace(spec): a new registry with one spec swapped for the one of its name, in the same place and with the same categories. Editing a profiled registry no longer means rebuilding it by hand.

Changed

  • to_markdown writes an Enum as its repr does: Enum in the Type column and its categories in Domain / Choices, each beside its share when weighted (SEN 60%, SUB 40%); weighted choices on any column the same. A nullable column shows its null rate, Yes (5%). An Enum inside a List or Struct keeps the dtype's own form.
  • A ForeignKey's repr names its target rather than printing it. It printed the whole parent spec, over 600 characters for one key of the shop tutorial's registry.
  • TableSpec.select and drop take a list, as Polars' do, as well as names one by one; a list raised TypeError: unhashable type before.
  • from_yaml given YAML text says so. to_yaml() with no path returns text, and passing it back raised a raw OSError; from_yaml, Registry.from_yaml and CatSpec.from_yaml now raise a SerializationError naming the call that reads text.

  • Batches, scans and sinks are faster where they were slow. The engine fills a column in 65,536-row chunks, so a batch that started inside one filled it from its start and dropped the head, and a batch of one chunk filled it on one thread. A stream now draws its columns many chunks at a time, up to about 64 MiB, in parallel, and slices each batch from them. A million rows of four columns in batches of 10,000 take 7 ms, not 118 ms; in batches of 1,000, 12 ms, not 897 ms. sink_parquet is a third faster, and batches with a foreign key five times. Seeded output is unchanged: a batch holds the rows it held. A List, Array or Map column is still drawn batch by batch.

  • List, Array, Map and Decimal columns generate several times faster. A list's cells are built in the engine from each cell's length and the flat elements, where a group-by over every element built them before: a million rows of List(Int64) take 5 ms, not 105 ms, and of List(String) 20 ms, not 140 ms. A Decimal is built from its physical integers, not divided into: 5 ms a million, not 20 ms. A unique column's permutation runs in 64-bit arithmetic where its domain fits, a tenth faster. Seeded output is unchanged.
  • A foreign key reads a unique parent without deduplicating it. A parent's key is usually unique, so its keys are the column as it stands; one count of distinct values says so. A 1,000-row child of a 1M-row parent takes 3 ms, not 33 ms, and a self-referencing key a quarter of the time. Seeded output is unchanged.
  • Validation leaves out a check that cannot fail. Values that arrive in the very Enum a column declares, with no choices narrowing it, are all its categories; testing each against them took most of the time of validating a frame with an Enum in it. A four-column frame of a million rows validates in 1 ms, not 20 ms.
  • Profiling is faster. A frame of 50,000 rows or more has its columns profiled at once, on a pool of threads; a column every value of which has a format (an email, a uuid4) is checked in one pass, not counted value by value. An 11-column frame of a million rows profiles in 204 ms, not 509 ms; a UUID column in 96 ms, not 182 ms. Every profile is the same as before, warnings and their order included.

Documentation

  • The README and Command line catch up with 1.5: related tables profiled and faked together, several files on the command line, --foreign-key, --no-foreign-keys, --scale, --format and --no-patterns. The README's timings are re-measured.
  • A weighted column is taught as choices={...}, the spelling its repr prints and profile reports; weights= is shown as the same spec given apart, and for a Boolean.
  • Dates written as text, and why a column stays text, have their own sections in Infer a spec and Validate data, a recipe, and rows in ColSpec fields and Spec files. Generate data says small batches cost little now.
  • Comparison is re-measured on 1.6: generation, last measured on 0.21.0, and validation, where polspec is 1.4 to 2.1 times as fast as dataframely and pandera.

1.5.0 - 2026-11-15

Real data in. profile_all profiles a set of related tables into a Registry, with the foreign keys between them found from their values, and synthesize_all makes a fake copy of the set, a tenth of the size or three times it, with every key held by a fake parent. The command line takes several files for both. profile writes a code's shape as its pattern, so fake order references and SKUs look like the real ones, and a weighted Enum prints as ColSpec(Enum, choices={'A': 0.5, ...}). Validation is benchmarked beside dataframely and pandera, and a third tutorial starts from data you already have.

Everything is additive under the 1.0 promises. Seeded generation is unchanged. Profiles change where a code now gets its pattern, which Stability leaves open, as it does a repr. Nothing is deprecated. The spec-file format stays at version 4.

Added

  • polspec.profile_all(frames) profiles a set of related tables into a Registry, with the foreign keys between them found from their values. A column becomes a key into a unique integer, text or date column -- of another table or its own -- when every value it holds is one of the parent's, in all the rows, and either the names link the two (customer_id or customer into Customers.id, parent_id into its own table's id) or it is text with at least 20 distinct values. A child with orphans, or with two parents to choose between, gets no key; the second warns. The key's column takes its parent's claims and keeps its own null rate, so the registry validates its sources and generates frames that pass. keys= declares keys instead, checked against the data, and keys=False infers none.
  • polspec.synthesize_all(frames, n=, scale=) is generate_all() over profile_all(): a fake frame for each table, in its source's dtypes, every key between them held by a fake parent. scale=0.1 takes a tenth of each table, keeping the ratios between them; n is a count for every table or a mapping. A parent's key range widens for a larger set, and its children with it.
  • polspec schema infer and polspec synthesize take several files. schema infer a.csv b.csv -o shop.yaml writes a registry (.yaml, or .py with a Registry after the classes), and synthesize a.csv b.csv -o fake/ writes one fake file per source into a directory, in its own format or --format's. Each spec is named after its file. --foreign-key Orders.buyer=Customers.id declares a key and --no-foreign-keys finds none. synthesize --scale X sizes the output as a multiple of the source's, for one file too.
  • profile writes a code's shape as its pattern. A text column of at least 20 distinct values that no format fits and that is not an Enum -- an order reference, a SKU, a postcode -- gets the pattern its values share when they have at most three shapes: each run of digits, capitals or small letters becomes its class with the lengths seen, and a part every value shares stays as written (^ORD-[0-9]{6}$). Every source value matches, and the fake codes have the shape. Not for a replaced column, for prose, or for numbers and dates written as text. A column of distinct values gets one only when it holds ten times as many. patterns=False, or --no-patterns, turns it off.

Changed

  • A column's repr writes an Enum's categories, and weighted choices, as choices: ColSpec(Enum, choices={'A': 0.5, 'B': 0.3, 'C': 0.2}) rather than ColSpec(Enum(categories=['A', 'B', 'C']), weights=(0.5, 0.3, 0.2)), each category beside its share. It is a spelling ColSpec takes. An Enum inside a List or Struct keeps the dtype's own. Spec files are unchanged.

Fixed

  • --all finds a data file by the spec name its stem gives. validate --all, drift --all and the other --all verbs looked for <Spec>.<suffix> and left the match to the file system, so orders.csv was found for Orders on Windows and macOS but not on Linux. A file matches when its stem is the spec's name or gives it, as several-source schema infer names a spec after its file (order_lines.csv for OrderLines), on every platform. Two files for one spec are refused, as before.

Documentation

  • Validation is benchmarked. benchmarks/validation.py validates the same million orders and their customers with polspec, dataframely and pandera, each written as its documentation writes it, eager and lazy. Comparison has the numbers: polspec is 1.3 to 2 times as fast, the gap between one aggregation and several passes. dataframely and pandera join the bench dependency group only.
  • A third tutorial, A spec for data you already have: three small CSV files of a shop, in docs/data/shop/, profiled into a registry with its keys, edited, validated, watched for drift and synthesized at a tenth for tests. Its commands run in the docs tests, as its Python does: a shell block marked <!-- docs: run --> runs each polspec line through the CLI.
  • Fake data from real data gains Related tables: synthesize_all, its sizes, how keys are found and how to declare them. Infer a spec from data gains Several tables. Known limitations says what survives between tables, and Recipes gains a fake copy of related tables and a key the profile missed.

1.4.0 - 2026-11-15

Generate what you validate. A pattern column is generated from its pattern, through the automaton Polars' own regex engine builds, so a spec with order references, postcodes or SKUs passes its own round trip with every check on; unique=True works on one. A column can be derived from others (derive=col("qty") * col("price")), computed by generation and held to its expression by validation. fixed= reaches every way of generating and the command line. schema_only=True checks a lazy frame's shape without running it, each finding carries its share of the rows, and uuid7, ulid, url and iso_country3 join the formats.

Everything is additive under the 1.0 promises, but for one change to seeded output the Stability page allows: a pattern column whose plain text broke its own pattern now draws from the pattern. A pattern every plain string passes, and one the generator cannot follow, keep their output exactly. Nothing is deprecated. The spec-file format stays at version 4: a derive: key is written only when used, and 1.3 refuses it by name.

Added

  • Strings from a pattern. A String column with a pattern is generated from it: each value is a full match, drawn by a walk through the DFA that regex-automata -- the engine under the regex crate Polars validates with -- builds for the pattern, so every value passes validation. ASCII wherever the pattern allows it; within string_length when declared; an unbounded repeat runs to 16 characters past the shortest match. A list of strings with a pattern generates its elements the same way. unique=True draws each of the pattern's values once: they are counted at each length, so an index decodes to one value, and the column is a keyed permutation of them like any other unique column. A million values take about 8 ms.
  • Derived columns: ColSpec(pl.Float64, derive=col("qty") * col("price")). A column computed from others with the col() expressions rules and checks use. Generation computes it after every column it reads -- rules, foreign keys, fixed= pins and other derived columns -- and casts it to the column's dtype; validation holds the column to it, as a new derived finding: equal, or for a float within a relative 1e-9, and null exactly where the expression is. derived=False (and polspec validate --skip derived) switches the check off. The expression decides the values, so a derived column declares no bounds, choices, rules, distribution, format, pattern, length, unique or default, is nullable when an input is, and is written by nothing else; a cycle of derived columns is refused. Spec files write it as a derive: key, which 1.3 refuses by name; diff reports one added or changed as breaking and one removed as compatible; rename rewrites it, and join drops it, as it drops rules. The data dictionary and the ER diagram show the expression.
  • fixed= everywhere: generate_batches, scan, every sink_*, their FrameSpec forms, and a registry's generate_all, generate_related and scan_all, which take pins spec by spec ({Customers: {"tier": "pro"}}). The rules are generate()'s. A batch is a window onto the pinned frame: a scalar fills every batch, a sequence of all n values gives each batch its stretch, and the batches are row for row what generate() makes. A scan checks its pins when it is built. polspec generate --fixed COLUMN=VALUE pins from the shell, each value read as its column's dtype (--fixed SPEC.COLUMN=VALUE with --all).
  • inspect(frame, schema_only=True) -- and validate and assert_valid -- checks what the schema says (missing, extra and optional columns, each column's dtype) and reads no rows: a LazyFrame's query is planned, never run. It does not combine with parse_text=True.
  • Finding.share: a counted finding's count as a fraction of the frame's rows, None where count is. The text report adds (2.5% of rows) to each such finding, the Markdown report shows it beside the count, and the JSON report has a share key (schema_version stays 1: a key added, none changed). The rows are counted in the aggregation that already runs.
  • Formats uuid7, ulid, url and iso_country3. A version-7 UUID and a ULID by their syntax (their timestamps are not read); an http or https URL with a hostname, optional port, path, query and fragment; and the ISO 3166-1 alpha-3 codes of the same 249 countries as iso_country. Each is generated, validated and recognised by profile(). A profile of data holding them names the format where it named none before; three codes (BTN, MKD, SLE) are both countries and currencies, too few for either format to be taken for the other.
  • A warning for a pattern the generator cannot follow -- a Unicode word boundary, an automaton over its size limit, no match within the declared string_length -- naming the column and the reason. The column is filled with plain random text, as before.
  • polspec test checks patterns. A generated test no longer passes pattern=False, except for a pattern the generator cannot follow, with a comment naming it and why.

Changed

  • Seeded output, for pattern columns only. A pattern column used to be filled with random alphanumeric text that its own pattern refused, and a minor release may change output that broke its own spec: those columns now draw from their pattern. A pattern that every such string already passes (^[A-Za-z0-9]+$) is decided on its automaton and keeps exactly the output it had. So does a pattern the generator cannot follow: a Unicode word boundary, an automaton over its size limit, no match within the declared lengths. Stability lists the change with the pre-1.0 ones, and the seeded-output catalogue now holds a pattern column, a unique one, one that keeps its plain text, a derived column and the four new formats, so their output is promised from here on.
  • Docs. Nested types (List, Array, Struct, Map) have a page of their own, Nested columns, out of Declaring columns; patterns moved to String formats. Recipes gains an order reference or postcode, a computed total, and a lazy pipeline's shape checked without running it. Pages that called patterns validation-only no longer do.

1.3.0 - 2026-11-15

Tests that say what broke. assert_valid checks a frame at the end of a test and, when it fails, names each finding, shows the failing rows and the seed to rerun with. check_pipeline tests a whole pipeline in one call. A pytest plugin gives each test frames from a seed of its own, so adding a test changes no other test's data, and strategy hands a spec to Hypothesis. polspec test writes its tests with them. The docs gain Recipes and a page for readers coming from dataframely, pandera or Pointblank. Everything is additive under the 1.0 promises, and no seeded output changes; nothing is deprecated.

The spec-file format stays at version 4, and 1.3 writes nothing 1.2 cannot read. The plugin loads whenever pytest runs with polspec installed; -p no:polspec turns it off.

Added

  • assert_valid(df, spec, **options): validate() as an assertion, for the end of a test. It returns the validated frame, so it chains, and on a failure raises AssertionError -- a failed test in any runner, not an error -- whose message names the spec, each finding by its key, and up to ten failing rows as a table.
  • check_pipeline(fn, inputs, output, *, n=, seed=): a pipeline test in one call. Each input is a spec to generate or a frame to pass; the specs are generated together, so a foreign key between inputs lines up, from a seed that defaults to 0. It calls fn with them, checks the result against output with assert_valid, and returns the result and the inputs.
  • A pytest plugin with a polspec_frames fixture. It loads whenever pytest runs with polspec installed (-p no:polspec turns it off). polspec_frames(Orders, 500) is generate() with a seed of the test's own, derived from the session seed and the test's id, so adding, removing or reordering tests changes no other test's frames. --polspec-seed=N sets the session seed, 0 by default, and --polspec-seed=random draws one; the report header names it, and a failing test that polspec took part in says how to rerun it. assert_valid names the seed in its message, and check_pipeline generates from the test's seed under pytest.
  • strategy(spec, *, min_rows=, max_rows=): frames a spec accepts, as a Hypothesis strategy, for property tests. Each is generate() on a drawn row count and seed, so it meets the spec by construction and follows its distributions; a failure shrinks to the fewest rows that fail. Hypothesis is not a dependency: it is imported when a strategy is asked for.
  • polspec schema infer SOURCE -o - prints the YAML spec instead of writing a file; the summary and any notes go to stderr, so redirecting stdout captures only the spec.
  • Docs: Recipes, short answers to "how do I…" (a primary key, ids 1..n, sorted timestamps, an empty frame, quarantining bad rows, a drift gate in CI, a nullable foreign key, a CSV read as text), each linking to the page that covers it.
  • Docs: Coming from dataframely, pandera or Pointblank, a feature table, a concept map and one schema translated from each, checked against their documentation on 2026-11-15. Comparison names the three, and says how polspec and Hypothesis now combine.

Changed

  • CI waits less. A pull request runs the oldest and newest Python on each OS, with the floor and newest-Polars jobs: eight test jobs, where it ran sixteen. main and a weekly run keep every Python on every OS, and coverage is measured on Python 3.12, a leg every pull request runs. A release no longer runs the matrix again: it waits for the Test run on main for the tagged commit and builds only if it passed. A test job compiles the Rust extension only when its sources changed: the built module is cached on them, one per OS, saving 5 to 14 minutes a job.
  • polspec test writes its tests with the new helpers. Each test draws from the polspec_frames fixture and checks with assert_valid, so a failure shows the rows and the seed to rerun it with. A spec whose foreign keys reach other specs in the same file generates those first, and its test runs where it used to be skipped; one into a spec the file does not declare is still skipped, saying which. --seed now pins every frame to one seed, and is unset by default, where it was 42.
  • CI measures coverage with coverage run -m pytest, not pytest --cov. pytest loads polspec's own plugin at start-up, which imports the package before pytest-cov begins, so every module's top-level lines counted as missed and coverage read 86% instead of 94%.

1.2.0 - 2026-11-14

Quicker to write, quicker to run. Data read as text profiles and validates as the types it holds. A spec is shorter to write: a bare dtype is a column, an Enum's weights sit beside its categories, dates in bounds can be text, and a class's docstring is its description. Declarations that never worked now fail where they are written, saying why. Specs and reports pickle, a spec reads as an Arrow schema, and every public name's reference shows a runnable example. Batched foreign keys, profiling and import polspec are 1.7× to 10× faster. Everything is additive under the 1.0 promises, and no seeded output changes; nothing is deprecated.

The spec-file format stays at version 4, and polspec writes nothing 1.1 cannot read. A spec whose class has a docstring now writes it as a description: key, which 1.0 refuses by name, as it does any 1.1 key.

Added

  • profile(parse_text=True) reads a text column as the type its values hold, for a frame read with its types switched off (pl.read_csv(..., infer_schema=False)): Boolean, Int64, Float64, Date or Datetime, the first every value parses as exactly. Empty text is a null; a value written with a leading zero, or an integer past Int64, keeps its column text. synthesize, FrameSpec.from_dataframe and ProfileOptions take it, and polspec schema infer and polspec synthesize take --parse-text. Off by default: a text column that would parse stays text, and its description says what it would parse as, which the two commands also print.
  • validate(parse_text=True) reads each column that arrived as text, where the spec declares a type text can hold, as that type before checking anything, so a text frame validates against its own profile(parse_text=True) and comes back typed. inspect, ValidationOptions, validate_all and inspect_all take it. A column with a value that does not parse stays text, and its dtype finding now counts those values and gives some as samples; polspec validate always reports them so.
  • A bare dtype is a column. opened = pl.Date in a class body, a dtype in __columns__ or in TableSpec(columns=...), and opened: Date in a spec file each mean ColSpec of that dtype. Spec files are still written in the long form, which 1.0 and 1.1 read.
  • Dates and times in bounds may be ISO text: bounds=("2020-01-01", None) on a Date, Datetime or Time column is read as the date it spells, where it is declared.

  • A bare pl.Enum takes its categories from choices. ColSpec(pl.Enum, choices={"A": 0.5, "B": 0.3, "C": 0.2}) declares pl.Enum(["A", "B", "C"]) with those weights, and is the same spec as that spelling -- equal, the same seeded frame, the same file. to_python writes a weighted Enum this way, each category beside its share; spec files keep the Enum: [...] and weights: form 1.0 and 1.1 read, and also read a bare Enum with choices. A bare pl.Enum with no choices is refused, saying to list its categories.

  • TableSpec.from_schema(schema, name=): a spec of a pl.Schema, a mapping of column name to dtype, or an Arrow schema, each column declaring only its dtype.
  • ColSpec.replace(**changes), for a column with a field changed.
  • Orders.names.total: each column's name by the attribute it is declared as, honouring col_name, so code can name a column without spelling it as text; on TableSpec too.
  • Reports show as Markdown in a notebook, through _repr_markdown_ on ValidationReport and DriftReport.
  • Specs read as Arrow schemas. A TableSpec and a FrameSpec class implement __arrow_c_schema__, so pl.Schema(Orders), pyarrow.schema(Orders) and any other reader of the Arrow PyCapsule interface take a spec where they take a schema.

Changed

  • Faster where it was slow, measured with benchmarks/bench.py against 1.1.0 on the same machine, with every other case unchanged:
  • Batches, scans and sinks with a foreign key into a large parent: a million rows in 10k batches from a 1M-row parent, 3.72 s to 0.36 s. Each parent's keys are read once per call, not once per batch. Seeded output is unchanged.
  • profile(): 235 ms to 93 ms on a million rows, 411 ms to 243 ms on the same rows as text. A column is no longer hashed again for each thing asked of it. Every profile is the same as before.
  • import polspec: 200 ms to under 60 ms on top of Polars. Generating, validating, drift, profiling, spec files and rendering load on first use.
  • A class's docstring is its description. A FrameSpec with a docstring and no __description__ now has that docstring as its description, which to_markdown shows and to_yaml writes as a description: key -- one 1.1 reads, and 1.0 refuses by name. __description__ wins when a class has both. Not inherited.
  • A spec prints as it was declared. A ColSpec shows its dtype and the fields that differ from their defaults, ColSpec(Int64, bounds=(0, 10)), and a TableSpec one column to a line, where both used to print every field.
  • Choices that are an Enum's categories, in its order, are dropped at declaration, so pl.Enum(cats) with choices= the same is the spec weights= alone makes. They generated the same frame already.
  • read() types numbers and booleans too. A column the spec declares as a Boolean, integer or float that arrived as text is parsed, as dates, decimals and durations already were, so read(spec, path, infer_schema=False) hands back a typed frame. A column of its own is stripped first and empty text read as a null. A column with a value that does not parse stays text, as before. polspec validate reads files the same way.

Fixed

Four declarations that never worked now fail where they are written, with a message saying why. A class or spec that hits one did not declare what it said, so each now raises SpecError at import where before it failed later, or silently declared less:

  • A dtype that is not a Polars one. ColSpec(int) or ColSpec("Int64") failed on first use with an unrelated AttributeError. It is refused with the Polars spelling: use pl.Int64.
  • Text in the bounds of a date, datetime or time column was kept as text, and validating against it raised on Polars 2, which no longer casts text to a date. It is read as the value it spells, and text that is not ISO, or text on any other column but a Decimal, is refused. A datetime on a Date column is refused as one, not as outside the range a Date holds.
  • Class-body slips. A trailing comma after ColSpec(...), a Check, ForeignKey, Hierarchy or ColRule under a name polspec does not read, and a misspelt declaration dunder (__unique_togther__, __check__) each built a spec without what they declared. Each is refused, a misspelt dunder with the one meant.
  • A range that is not a pair. bounds=(5,) raised ValueError: not enough values to unpack; bounds, string_length and list_length now say they must be a (min, max) pair.

An unknown dtype name in a spec file names the closest known one, or lists them.

  • Specs and reports could not be pickled. A TableSpec, a struct column, a format's extra_values, a Registry, a ValidationReport and a DriftReport each raised cannot pickle 'mappingproxy' object (or failed on a closure), so none could go to another process or be deep-copied. Each now pickles and deep-copies, and a report still names its rows afterwards. Their read-only mappings print as dicts.

  • Validating a number too large for a Decimal with choices crashed. Asking whether a float or integer standing in for a Decimal(p, s) was one of its choices, or a rule's, cast it to the Decimal; Polars raises on a cast past the precision, so inspect() and validate() raised ComputeError where they should have reported the value. It is now the dtype finding it is without choices.

  • profile() could refuse its own hostname column. A dot is what makes a column hostnames to the profiler, so a dotless value such as localhost or 0 was kept as an extra_values stand-in. But the hostname format accepts those, and ColSpec refuses a stand-in its format already accepts, so profile(), synthesize() and polspec schema infer raised SpecError on the column. Stand-ins are now only values the format refuses.
  • profile() took decimals held as text for hostnames. 0.45 passes the hostname check, as host 0 in domain 45, so a numeric column read as text (a CSV read with infer_schema=False) was profiled with format: hostname. A value ending in an all-digit label is no longer evidence of a hostname, since no real one ends that way; validating a declared format="hostname" is unchanged.

1.1.0 - 2026-11-08

A spec says more. A table or column can say what it means, how many rows a table holds, what fills a missing value and which columns may be absent; a generated frame can be pinned to the values a test needs; and a join of two specs is a spec. Everything is additive: code that runs on 1.0 runs unchanged, no seeded output changes, and nothing is deprecated.

The spec-file format stays at version 4. A file that uses none of the new keys reads in 1.0 as before; one that uses description, rows, default or optional is refused by 1.0, which names the key it does not know.

Added

  • description= on ColSpec and TableSpec, __description__ on a FrameSpec, and TableSpec.with_description(): what a table or column means. to_markdown shows it, spec files and to_python keep it, and diff reports a change as compatible. Nothing else reads it.
  • rows= on TableSpec, __rows__ on a FrameSpec, and TableSpec.with_rows(): how many rows a table may hold, such as (1, None) for a table that must not be empty. Outside it, validation reports the new structural finding row_count; rows=False and --skip rows turn it off. generate() still makes the rows it is asked for. polspec test skips the check in the tests it writes.
  • default= and optional= on ColSpec, and fill_defaults=True (--fill-defaults) on validation. A default is a value its column accepts, checked where it is declared; fill_defaults fills a defaulted column's nulls, and adds it when absent, before any claim is checked. An optional column may be absent without a missing_columns finding. missing_cols="add" adds a column holding its default. diff reports a column made optional, or an optional column added, as compatible; made required, breaking. generate() reads neither.
  • generate(fixed=) and FrameSpec.generate(fixed=): pin columns to given values, a scalar or exactly n of them, and generate the rest. A rule keyed on a pinned column follows it, and no other column's values change. A pin its column would refuse, or one on a column a pass writes, is a GenerationError naming the column. Not yet taken by generate_batches, scan, the sinks or the command line.
  • TableSpec.join(): the spec of a join of two frames, for every how, suffix and coalesce Polars takes, its columns named as the installed Polars names them. Claims about values are kept, a side that can go unmatched becomes nullable, and claims about rows are dropped, except a unique the other side's unique key preserves. semi and anti keep the left spec but for its row count and order.
  • TableSpec.prefix() and TableSpec.suffix(): every column renamed, as rename renames.
  • The finding code row_count, added in a minor release as the Stability page allows.

Changed

  • The documentation was reviewed page by page: sample outputs match the specs each page declares, the new fields are covered where a reader looks for them, and the site renders a dash as a dash.

1.0.0 - 2026-10-22

Stable. The public API, the spec-file format, seeded output, the command line and the finding and drift codes now change only in a major release, after a deprecation: the Stability page states each promise and the test that holds it. 1.0 removes what 0.x deprecated, below; its other changes were made in 0.23, so code that runs on 0.23 without deprecation warnings runs on 1.0. No seeded output changes.

Removed

What 0.x deprecated, each of which warned for at least one minor release:

Removed Deprecated in Use instead
polspec.profile_dataframe(df, ...) 0.18.0 polspec.profile(df, weights=False, shape=False, detect_unique=False, detect_order=False).columns
polspec.serialization.COLSPEC_FIELDS, TABLESPEC_FIELDS, COLRULE_FIELDS, CHECK_FIELDS, FK_FIELDS 0.18.0 the keys the Spec files reference lists
FrameSpec.checks() 0.21.0 FrameSpec.spec.checks
FrameSpec.foreign_keys() 0.21.0 FrameSpec.spec.foreign_keys
FrameSpec.unique_together() 0.21.0 FrameSpec.spec.unique_together

A column may now be called checks, foreign_keys or unique_together without shadowing anything. From 1.0, a deprecated name is removed only at the next major release.

Documentation

  • The site is slimmer: the changelog (on GitHub), roadmap and generation-and-validation pages are gone -- the seeded-output history now sits under Stability's promise, and the registry's edges and the two-sides argument in Known limitations -- and the architecture page is ARCHITECTURE.md, for contributors. Inferring a spec sits beside declaring one, and the command-line guide links to its generated reference rather than copying its flag tables.

0.23.0 - 2026-10-22

The surface 1.0 freezes, released first. A review before 1.0 read every public name and signature, since 1.0 makes them permanent, and this release makes every change it found: one namespace, polspec; constructors that take their options by keyword; one spelling for each option and parameter; typed options on every entry point, FrameSpec.drift among them, which took four of its seven; and the internals out of the API reference. They are breaking, made outright with no deprecation period, and listed below with the replacement for each. No seeded output changes.

Breaking

0.23 makes every change to the public surface before 1.0 freezes it, so 1.0 freezes one that has been released. Each is made outright, without a deprecation period; this list, with its replacements, is the migration guide.

  • One namespace: everything public is imported from polspec. polspec.drift, polspec.validation and polspec.serialization are internal now, as every other module is.
Before Now
from polspec.drift import drift, diff from polspec import drift, diff (or polspec.drift(spec, df))
polspec.drift.DriftCode, polspec.drift.Severity polspec.DriftCode, polspec.Severity
polspec.validation.FINDING_COLUMN, polspec.validation.FindingCode polspec.FINDING_COLUMN, polspec.FindingCode
polspec.serialization.to_dict, from_dict, FORMAT_VERSION polspec.to_dict, polspec.from_dict, polspec.FORMAT_VERSION
polspec.serialization.catspec_to_dict(cats), catspec_to_yaml(cats, path) cats.to_dict(), cats.to_yaml(path)
polspec.serialization.catspec_from_dict(data), catspec_from_yaml(path) CatSpec.from_dict(data), CatSpec.from_yaml(path)
polspec.serialization.registry_to_dict(r), registry_to_yaml(r, path) r.to_dict(), r.to_yaml(path)
polspec.serialization.registry_from_dict(data), registry_from_yaml(path) Registry.from_dict(data), Registry.from_yaml(path)

Every name already at the top level stays where it was. - CatSpec reads an entry three ways, and its parts from three mappings. cats["X"], cats.X and cats.get("X") give the dtype, matching a name's case as before; cats.enums, cats.categoricals and cats.choices hold what is underneath, keyed exactly.

Before Now
cats.dtype_of("X") cats.get("X")
cats.enum.X, cats.enum["X"], cats.enum("X") cats.X or cats["X"]
cats.categorical.X (and its item and call forms) cats.X or cats["X"]
cats.get_enum("X") cats.enums["X"], or cats["x"].categories.to_list() for any case
cats.get_categorical("X") cats.categoricals["X"], or cats["x"].categories for any case
cats.get_choices("X") cats.choices.get("X")
  • Constructors take everything past what identifies the thing by keyword. ColSpec(dtype, *, ...), TableSpec(name, columns, *, ...), Check(expr, *, ...) and ForeignKey(columns, *, references=..., ...); ColRule, Hierarchy, ValidationOptions, DriftOptions, ProfileOptions, Finding and DriftFinding by keyword throughout. ColSpec(pl.Int64, "x") used to set col_name without saying so. Bound(min, max) stays positional.
Before Now
ColSpec(pl.Int64, None, None, True) ColSpec(pl.Int64, nullable=True)
ForeignKey("customer_id", Customers) ForeignKey("customer_id", references=Customers)
Hierarchy("node", "parent") Hierarchy(child="node", parent="parent")
ValidationOptions("drop") ValidationOptions(extra_cols="drop")
  • One spelling for each validation switch: the field's name. validate(df, checks=False), as ValidationOptions(checks=False) and --skip checks already said it. The old spelling is refused, naming the new one ('validate_checks' (did you mean 'checks'?)).
Before Now
validate_rules=, validate_validators=, validate_unique=, validate_checks=, validate_foreign_keys=, validate_hierarchy=, validate_order=, validate_pattern=, validate_bounds= rules=, validators=, unique=, checks=, foreign_keys=, hierarchy=, order=, pattern=, bounds=
  • Writers take path; a reader's source stays. to_yaml, to_python, FrameSpec.to_yaml/to_python, CatSpec.to_yaml and Registry.to_yaml call their destination path, as to_markdown, to_mermaid and every sink_* already did. from_dict's label for its errors is where.
Before Now
to_yaml(spec, source="orders.yaml") (and the other writers) to_yaml(spec, path="orders.yaml"), or positionally as before
from_dict(data, source="orders.yaml") from_dict(data, where="orders.yaml")
  • Every drift and validation entry point takes options= or typed keywords. FrameSpec.drift(df, ...) takes all seven DriftOptions fields -- it named four, so significance= was a TypeError there and worked in polspec.drift() -- and drift(), diff() and FrameSpec.diff() type them (DriftKeywords). Registry.inspect_all and validate_all take options= or the validation keywords, typed, where they took untyped **options.
  • FrameSpec.from_dataframe() names its spec "Profiled", as profile() does, not "ProfiledFrameSpec".
  • Internals out of the public API. Pred is what builds, evaluates, writes and renames a predicate: to_expr, to_data, to_source, rename, equals, and the operators. Its tree walks -- children(), rebuild(), root_names(), literals() -- are internal, as is TableSpec.resolve_target(). Check and ForeignKey no longer take pred= and target=, which they fill in themselves; check.pred and fk.target are still there to read.
Before Now
pred.root_names() set(pred.to_expr().meta.root_names())
spec.resolve_target(fk) fk.target (or the spec itself, for references="self")
  • TableSpec.estimated_size(n) is polspec.estimated_size(spec, n); FrameSpec.estimated_size(n) stays.
  • Finding.rows(frame) is internal; report.rows(finding) is how a report's rows are read.
  • CliError is no longer exported. The command line never let one reach a caller: main() turns every polspec error into an exit status.

0.22.0 - 2026-10-22

Beta: the contract, written down and tested. The Stability page states what polspec promises now and what 1.0 will -- the public API, spec files, seeded output, the command line, the finding and drift codes -- and a test holds each promise to a golden copy, so a release cannot break one without a test failing first. What 1.0 removes is listed there and warns today. Two small changes come with it: every --json document leads with a schema_version, and the --all documents key their reports under reports; and schema infer no longer writes a null rate on a column with no nulls, and rounds category frequencies. No seeded output changes.

Changed

  • Beta. The Development Status classifier says so, and the API is frozen pending 1.0: no public name changes in place. What 1.0 removes is listed on the Stability page, Removed at 1.0, and each one warns today: profile_dataframe, serialization's five *_FIELDS tables, and FrameSpec.checks(), .foreign_keys() and .unique_together().

  • Every --json document carries a schema_version, and so does ValidationReport.to_dict() and DriftReport.to_dict(). Its keys and their types are the promise, and the version rises if one is ever renamed, removed or changes type.

  • validate --all --json and drift --all --json key their reports under reports: {"schema_version": 1, "reports": {"Orders": {...}}}. They were keyed by spec name at the top level, where a spec called schema_version would have collided. A script reading doc["Orders"] reads doc["reports"]["Orders"] now.
  • schema infer writes what a person would. A column with no nulls carries no null rate -- it is not nullable, so the rate meant nothing, and every such column used to say null_probability: 0.0. Under --round-rates (on by default) category frequencies are written at the rates' precision too, each to its nearest step and never to 0: weights=[0.4, 0.3, 0.3], not [0.4023872466554514, ...]. The same holds for profile() and FrameSpec.from_dataframe() given round_rates. What a spec generates and accepts is unchanged.

Documentation

  • The finding codes table gains the three hierarchy codes it lacked: hierarchy_multi_parent, hierarchy_cycle and hierarchy_depth.
  • Errors said an exception other than a PolspecError keeps its traceback on the command line; it is printed as one line, with its class, and exits 1.
  • Stability, a new page: what polspec promises now, in the beta, and what 1.0 promises -- the public API, spec files, seeded output, the command line and its exit codes, finding and drift codes -- with the test that holds each, what is not promised, and the supported Python, Polars and operating systems. The roadmap keeps where polspec is going, and the list of releases that changed seeded output.
  • The command-line guide says a command line that is itself wrong exits 2.

0.21.1 - 2026-10-22

The documentation, current and usable. No code changes.

Documentation

  • Infer a spec from data, a new how-to: profile and FrameSpec.from_dataframe side by side -- one imitates the data, one describes it -- with every option, ProfileOptions, rounding, and the schema infer equivalent. Until now this was split across three pages, and rounding from Python was only in the API reference.
  • Every page declares what its examples use. Nine pages used specs only the docs tests declared, and eleven used names they never imported, so copying an example could fail with a NameError. Two foreign-key examples passed references= to a spec with no foreign key; they now use one that has.
  • Testing pipelines gains a fixture per spec, a stage that must reject bad rows (asserted against inspect() and passing_rows()), and fixtures that look like production.
  • Corrected: schema infer's example output, which predated rounding; the cartesian example's row count (twelve, not sixteen); three error comments naming ValueError for SpecError; four error messages shown shorter than they are; "early alpha" on two pages, against the README's "pre-1.0".
  • Roadmap and stability: which releases changed seeded output is a table, current to 0.21. The benchmark table is re-measured on 0.21.0.
  • The home page lists row order among what a spec can declare, and links to inferring a spec. Known limitations no longer opens on internal module names; the architecture page says it is for contributors.
  • The docs tests now check what a page shows, not only that its code runs: each error comment against the error raised, and each output block against what the block prints.

0.21.0 - 2026-10-22

A consolidation release: what 0.20 added, made sound. passing_rows() keeps its promise under order_by and a key into the frame's own rows, profile() is fast again, and an error in a spec file says where it is. Profiling takes one options object, ProfileOptions, as validation takes ValidationOptions, and synthesize takes all of it. The rest is groundwork for the 0.22 contract: three FrameSpec accessors deprecated in favour of .spec, a weekly deep run of the properties, and a test suite that tests through the public API only.

Added

  • ProfileOptions: one definition of what profiling takes. profile, FrameSpec.from_dataframe and synthesize take the options as an options= object or as the keywords themselves -- as validate takes ValidationOptions -- and the keywords are typed (ProfileKeywords), so an editor completes and checks them. synthesize takes every one now, not four of them: it can be told not to keep the source's order, not to fit shapes, or to round. FrameSpec.from_dataframe takes sample too.

Changed

  • The validation keywords are one typed list, ValidationKeywords. validate, inspect and the FrameSpec facades take **keywords typed by it rather than spelling every option out four times, so an editor still sees each one. A misspelt option is refused as validate refuses it, naming the option meant ('validate_uniqe' (did you mean 'validate_unique'?)), where the facades said only "unexpected keyword argument". Still a TypeError.

Deprecated

  • FrameSpec.checks(), .foreign_keys() and .unique_together(), removed in 1.0. Each call warns, and still works. Use .spec.checks, .spec.foreign_keys and .spec.unique_together: everything a declaration says is on .spec, which no column can shadow, and an order or a hierarchy needs no accessor of its own.

Documentation

  • Known limitations says that an ordered spec can't be batched or streamed, as a hierarchy can't.
  • The drift how-to lists row order among the pass/fail claims that drift() leaves to validate().
  • The constraints how-to gains a recipe for a sequence, 1, 2, …, n: a unique column bounded (1, n) and ordered by itself.

Fixed

  • passing_rows() could keep rows the spec rejects. Taking the failing rows away could break a claim relating rows to each other in the rows left: with order_by, [3, 1, 2, 5, 4, 6] passed [3, 2, 5, 6], still out of order; with a key into the frame's own rows, a row whose parent failed passed as an orphan. What is left is now validated again until nothing more fails, so the passing rows are always a frame the spec accepts, and failing_rows(once=True) names the rows taken later by the claim they broke -- the two still split the frame exactly. A spec with no such claim validates once, as before.
  • profile() was about 20 times slower in 0.20, finding the order a frame comes in by ranking every column whole, both ways. It now reads each column's first rows first, which rules out nearly every unsorted column, and checks the rest whole: on 1,000,000 rows by 20 columns, order detection takes 8 ms, not 1.2 s. It finds the same order.
  • An error in a spec file says where it is. A declaration refused while reading a file -- bounds: [5, 1], an order_by naming a missing column -- named neither the file nor the column. It now does (orders.yaml: columns.total: Bound min (5) must be <= max (1)), and a registry file's names the spec too (specs.orders: ...). The error keeps its type.

0.20.0 - 2026-10-21

Rows that come in order. A spec can declare the order its rows come in -- __order_by__ = ["user_id", "ts"] for each user's events in time order -- which event logs, time series, window functions and join_asof all depend on. generate() sorts whole rows, so every other claim still holds; validate() reports rows out of order; profile() finds an order a frame already has, so synthesize() keeps it.

Inferred specs that read like written ones. polspec schema infer now writes bounds rounded outward and rates rounded up -- [0, 9900], not [3.17, 9871.23] -- at a precision you can choose, always keeping the data valid; dates as dates rather than day counts; and an unbounded unique ID column no longer stops at two million rows. A spec also comes back as text from to_yaml() and to_python() without a path.

Python 3.15 is supported, with Polars 2, and tested in CI on every OS.

No seeded output changes: every catalogue case generates what 0.19.0 does, checked against the published 0.19.0 across 463 cases. No spec file needs migrating: order_by and descending are new optional keys. schema infer's files change by default (--round-bounds exact --round-rates exact writes what 0.19.0 wrote), and so do profile()'s temporal bounds and the order of a sampled frame's rows.

Added

  • Python 3.15. The wheels are built against Python's stable ABI, so the same wheel runs on 3.12 to 3.15; CI now tests 3.15 on Linux, macOS and Windows. On 3.15, polspec requires Polars 2: Polars 1 does not run on Python 3.15 -- some of its Series methods return None there, and the process can crash after -- so pip installs Polars 2 on 3.15, and Polars 1.39 or later on 3.12 to 3.14 as before. PyYAML publishes no 3.15 wheels yet, so on 3.15 pip installs it from source -- its pure-Python version, with no compiler needed.
  • order_by: rows that come sorted. A spec declares the columns its rows are sorted by, as DataFrame.sort takes them -- __order_by__ = ["user_id", "ts"], with __descending__ for the direction -- for event logs, time series and anything a window function, join_asof or a keep-the-latest dedup reads in order. generate() sorts whole rows last, so every rule, check, composite key, foreign key and hierarchy still holds; validate() reports each row that belongs before the one above it (finding code order, skipped with validate_order=False or --skip order). Nulls come last. A spec file carries it as order_by: and descending:, to_python as the class attributes, and drift reports an order added as breaking. The streaming verbs -- generate_batches, scan() and the sinks -- refuse an ordered spec, naming why, as they refuse a hierarchy: an order is a property of the whole frame. The Markdown render names the order.
  • profile() declares the order a frame comes in: the first column of single values -- at least 20 of them distinct, so not by chance -- whose rows come sorted either way, nulls last. So synthesize()'s fake rows come in the source's order. detect_order=False turns it off; FrameSpec.from_dataframe(detect_order=True) and polspec schema infer --order turn it on, as detect_unique and --keys are. An order within each key is the user's to declare.
  • Inferences at a person's precision. polspec schema infer writes each numeric column's bounds rounded outward and each null and NaN rate rounded up -- [3.17, 9871.23] as [0, 9900], 0.023651 as 0.024 -- so a draft spec reads like a written one. --round-bounds and --round-rates take a step (1000, 1k, 1M, 0.01), auto (the default: two significant figures, and whole seconds for a timestamp) or exact. Outward and up, within each dtype's range, so the data still passes the spec it was inferred from; a rate of 0 stays 0. profile() and FrameSpec.from_dataframe() take round_bounds= and round_rates= too, and write exactly what they observed unless asked.
  • A spec as text. to_yaml(spec) and to_python(spec) -- and registry_to_yaml, and the FrameSpec and Registry methods -- return the text when given no path, as to_markdown and catspec_to_yaml already did.

Changed

  • polspec schema infer's bounds and rates are rounded by default (auto, above). --round-bounds exact --round-rates exact writes what 0.19.0 wrote.
  • A profiled date, time, datetime or duration column's bounds are dates and times, as a person writes them -- [2024-01-01, 2024-12-31], not [19723, 20088] -- in the column's time zone. A nanosecond column's bounds round outward to the microsecond, Python's limit, so every value stays inside; a bound Python cannot hold stays the integer. The data generated is the same.
  • A unique=True Int64 or UInt64 (or 128-bit) column with no bounds makes room for the frame. Its default range, +-1,000,000, refused a frame of more than 2,000,001 rows; it now widens upward to fit. Only where it refused before, so nothing that generated changes.

  • A sample keeps its source's row order. profile(sample=...) and the command line's --sample draw the same rows as before, for the same seed, but in the order the file has them, not shuffled: a sorted file's sample is sorted too, and reads in file order.

0.19.0 - 2026-10-13

The same seed, the same data, on every operating system. A column drawn from a non-uniform distribution could differ between Windows, Linux and macOS, because its sampler used each platform's math library. 0.19.0 samples with maths of its own, the same bits everywhere, and CI now checks every seeded case against one digest on all three. The Known limitation is gone, ahead of 1.0's promise that a seed's output is stable.

Seeded output changes, once, for some distribution columns -- on Windows, a lognormal column and a batched beta column; everything else draws what 0.18.0 drew. No spec file needs migrating, and no API changes.

And faster: distribution and uniform columns generate as fast as 0.18.0 or faster, all but one case.

Changed

  • A seed draws the same data on every operating system. A column drawn from a non-uniform distribution (normal, lognormal, exponential, gamma, beta, poisson) could differ between Windows, Linux and macOS: its sampler took exp, ln and pow from each platform's math library, which round the last bit differently on a few inputs. The samplers are now polspec's own: rand_distr's algorithms, ported draw for draw, with exp, ln and pow from a port of Arm's optimized-routines -- the algorithms glibc uses, kept to plain IEEE arithmetic -- which round the same way everywhere and are as fast as a platform's own; a lognormal column draws faster than before. Seeded output changes for such columns wherever the platform's library and the port disagree. On Windows that is a lognormal column and a batched beta column only: normal, exponential, gamma and poisson columns draw what 0.18.0 drew. On Linux and macOS, where 0.18.0 kept no record to compare against, the same kinds of column may change. A sampler draws the same random numbers as before, so where a value changes, it changes from the first draw the libraries disagree on.
  • Distribution columns generate faster. A column's values are drawn by a loop specialized to its distribution, rather than one loop deciding per value, so each holds only the code it needs. Against 0.18.0 on Windows, 20 million values each: lognormal 8% faster, gamma 5-9%, beta 6%, exponential 12%, a uniform column 6-12%; normal the same, and a poisson with a small lambda 5% slower.
  • profile() rounds a fitted distribution's parameters to 12 significant digits -- mean: 1.08778398481, not 1.0877839848116615. They are measured with the platform's float maths (a log-normal's through ln), which may differ in the last bit between operating systems; rounded, a profiled spec, and what synthesize() draws from it, is the same on each in practice. A spec file reads more cleanly too. A spec declared or loaded from a file is unaffected.

0.18.0 - 2026-10-09

What you can import is what polspec promises. The public API is now written down: polspec's __all__, which gains the seven names that lived only in submodules, plus three public submodules, polspec.drift, polspec.validation and polspec.serialization. Every other module says it is internal, and a test holds the docs, the docstrings and the command line to that line. There is one profiler, profile(), and a public quarantine view, failing_rows(once=True).

Nothing that worked breaks. This is polspec's first release with deprecations: profile_dataframe and serialization's field tables still work, warn, and go at 1.0. No seeded output changes: every catalogue case generates what 0.17.0 does, checked against the published 0.17.0 across 463 cases. No spec file needs migrating.

Added

  • The public names that lived only in submodules are at the top level: to_yaml, from_yaml, to_python, to_markdown, to_mermaid -- each taking a TableSpec or a FrameSpec class, as every verb does, and each the function of the same name FrameSpec has a method for -- lit, beside col, and estimated_size. The submodule paths still work: they are the same functions. The API reference has a Files and documents page.
  • ValidationReport.failing_rows(once=True): each failing row once, naming every claim it broke -- the quarantine view --failing writes, which with passing_rows() splits a frame exactly. The default is unchanged.
  • profile() takes the switches profile_dataframe had: weights, shape, detect_unique and calculate_bounds, each on by default -- what profile() already did -- so one profiler covers both uses.
  • The public API is written down, and tested. It is polspec's __all__, and the __all__ of polspec.drift, polspec.validation and polspec.serialization. Every other module says in its docstring that it is internal, and may change in any release (Roadmap and stability, The public API is what polspec exports). A test holds the line: the docs import only public names, every public class and function has a docstring, and no module takes another's private names -- the command line included, which now uses only what reading, validation and catspec expose for it. The serialization functions for a CatSpec and a Registry gained the docstrings they lacked.

Deprecated

  • profile_dataframe, removed in 1.0. Each call warns, and still works. The same columns are profile(df, weights=False, shape=False, detect_unique=False).columns. This is polspec's first deprecation: a public name now warns for at least one minor release before it goes, rather than changing in place (CONTRIBUTING, Deprecations).
  • polspec.serialization's field tables (COLSPEC_FIELDS, TABLESPEC_FIELDS, COLRULE_FIELDS, CHECK_FIELDS, FK_FIELDS), removed in 1.0. They describe polspec's own codec and were never documented; the Spec files reference lists the keys a file holds. They are out of the module's __all__, and each still works, and warns.

0.17.0 - 2026-10-08

Every Polars from 1.39 to 2.x. polspec declared polars>=1.44.1,<2; it declares >=1.39,<3 now, and makes it true. Below 1.43 it had 24 failing tests: Polars features newer than 1.39, and Polars bugs fixed since, each worked around in the one place it touched. CI tests the floor, the newest 1.x and the newest 2.x on every pull request, and the same seed generates the same data on all of them -- every seeded catalogue case is compared against a stored digest of its values, on each. Comparing them across operating systems too found one thing that never held: values drawn from a non-uniform distribution may differ between Windows, Linux and macOS (Known limitations). They are the same on every Polars on one OS.

No seeded output changes. Every spec that constructs under 0.16.0 generates what it did -- checked against the published 0.16.0 across 463 cases, and on every Polars from 1.39.0 to 2.0.0 across 356 -- and no spec file needs migrating.

Performance is flat on Polars 1.44 and 2.0: validation, drift, profiling, synthesis and cartesian generation time the same as 0.16.0, interleaved run for run. On 1.43 and later, an Array's elements are checked exactly as before; below it, through a list, which older Polars needs.

Fixed

  • polspec works on Polars 1.39 to 1.43. Every test passes on the latest patch of each, from 1.39.0, where 24 failed before. Each cause was a Polars feature newer than 1.39, or a Polars bug fixed since:
  • cartesian generation joined its columns with concat(how="horizontal_extend"), which Polars has from 1.42 -- while 1.42 deprecates how="horizontal"; it uses DataFrame.hstack now;
  • an Array's elements were checked with arr.eval, which before 1.42 refuses an is_in or a filter and, before 1.43, panicked filtering an Array of Decimal or Duration; on those versions they are checked as a list's are, and from 1.43 exactly as before;
  • Polars before 1.43 writes an Array with null rows to JSON wrongly -- a null row as [null, null], the rows after it shifted -- so sink_ndjson and the command line write an Array as the list it is in JSON;
  • read() rebuilt an array's null row as an array of nulls on Polars before 1.43; it stays null.

Changed

  • polspec supports Polars 1.39 to 2.x: polars>=1.39,<3, where it declared >=1.44.1,<2. CI tests the floor (1.39.0), the newest 1.x (the lock) and the newest 2.x on every pull request. A seed generates the same data on every one: the seeded catalogue's values are compared, case for case, against a stored reference on each -- a column drawn from a non-uniform distribution on the operating system that recorded it. On Polars 1.39 alone, a nullable struct with no fields holds no nulls (Known limitations).

Internal

  • tests/contracts/test_seeded.py holds the seeded output to a reference, seeded.json: a digest of the values each catalogue case generates -- every column case whole, batched and scanned, every dtype, the table-level features. It digests the values rather than hash_rows, whose hash differs between Polars majors. POLSPEC_WRITE_SEEDED=1 rewrites it, for a deliberate change.
  • CI: the newest-polars job resolves Polars 2.x and is the required Polars 2 run; the pinned 2.0.0 job and the pre-release job are gone.

0.16.0 - 2026-10-07

Any column name. The review of 0.15.1, on the day Polars 2.0.0 was released, found one class of defect: a column polspec could not reach. A column named * or ^…$ is a pattern to Polars; a column named __row, __polspec_row or count collided with a helper column of polspec's or Polars' own; and failing_rows() wrote over a column named __polspec_finding. Every name now works, and that one is refused by name. The property tests draw all of them, and a test keeps a bare pl.col of a column name out of the package.

Polars 2.0.0. The suite passes on it unchanged, and CI runs it on every pull request. A panic in Polars 2.0.0's streaming engine, on validation's aggregation over a list column, is avoided by validating on the in-memory engine. polspec still declares polars<2; 0.17.0 widens that to <3, and lowers the floor to 1.39.

Changed, on purpose: col("*") in a check means the column named *.

No seeded output changes. Every spec that constructs under 0.15.1 generates what it did -- checked against the published 0.15.1 across 463 cases -- and no spec file needs migrating.

Performance is flat. Validation, drift, profiling and synthesis time the same as 0.15.1, interleaved run for run on a million rows; an ordinary column name compiles to exactly the expressions it did.

Fixed

  • A column named *, or ^…$, works like any other. Polars reads pl.col("*") as every column and pl.col("^id$") as a regular expression, and so do select, group_by, pl.struct, drop and a join's keys given the name as a string. Every place polspec names a column now refers to it exactly -- generation, validation, foreign keys, profiling, drift, read() -- through one helper, frames.column, which is the same pl.col as before for every other name. Many Polars Series methods also select their Series by its own name, so a Series named ^c$, a pattern that does not match itself, fails fill_null, is_null, unique or explode in Polars 1 and 2. polspec measures such a column under a plain name, and draws it under a plain name seeded by its own, so its values are what any name would draw. Known limitation since 0.15.1; the test that pinned it now passes.
  • polspec's helper columns no longer take a name the data has. A list column named __row could not be generated; a column named __polspec_row made passing_rows(), and --output with --failing, raise DuplicateError; a column named count made profile() raise, since Polars' value_counts names its own column that. Each helper column now takes a name absent from the frame, or polspec measures the data under names of its own.
  • failing_rows() no longer writes over a column named __polspec_finding. It wrote the finding keys over the data's own column of that name, silently; it raises ValueError now, naming the column, and polspec validate --failing refuses before writing either file.
  • Validating a list column no longer panics on Polars 2.0.0. Polars 2.0.0 runs a lazy query on its streaming engine by default, and that engine panics on validation's single aggregation over a list column -- list.eval beside a filter -- for some frames: one concatenated from batches, whose columns are chunked differently, and a two-row frame of a list of categories beside a column of nulls. Found by the batched and arrival properties, about one draw in 25,000, and reproduced without polspec. Validation runs on the in-memory engine unless streaming=True is asked for -- the engine a lazy query already ran on in Polars 1, and faster on 2.0.0 too (26 ms against 34 ms for a million rows).

Changed

  • col("*") in a check means the column named *, as col() means a column everywhere else -- not every column. Likewise col("^id$").

Internal

  • The lock moves to Polars 1.44.2, and the required Polars 2 CI job from the 2.0.0rc2 release candidate to 2.0.0.
  • POLSPEC_DEEP_EXAMPLES=3000 uv run pytest tests/contracts/test_properties.py runs every property 3,000 times; CONTRIBUTING has a Deep runs section. A new property holds generated data to 0.14.0's promise: a spec does not drift from its own output.
  • scripts/deep/render/ parses drawn specs' ER diagrams with Mermaid 11 and 10 and their data dictionaries with GitHub-flavoured Markdown rules; scripts/deep/polars2_eval_chunks_repro.py reproduces the Polars 2.0.0 panic without polspec, for the upstream report.

0.15.1 - 2026-10-05

What polspec writes, it reads back -- and a person can open. A patch release from the 0.15.0 review, about the edges rather than the core: the documents, diagrams and spec files polspec writes for people, the NDJSON it writes for itself, and the quarantine file --failing writes. Each fix makes something that failed work, or refuses it by name, and is pinned by a test that fails on 0.15.0. The property tests now draw awkward names -- pipes, spaces, keywords, the empty string -- and found two more defects of their own.

Changed, on purpose: --failing writes each failing row once, naming every claim it broke. A Binary column is refused when writing JSON, and a column name holding a NUL when the spec is declared; both panicked inside Polars before.

No seeded output changes. Every spec that constructs under 0.15.0 generates what it did -- checked against the published 0.15.0 across 463 cases -- and no spec file needs migrating.

Fixed

  • to_markdown() keeps its tables whole, whatever the names. A | in a column name, category, choice, tag or pattern split its cell -- inside backticks too -- and a newline ended the row; both are escaped now, in the data dictionary, the drift report and the category registry. A name holding a backtick is fenced so its code span holds, and an empty name is an empty code span rather than a stray "``".
  • to_mermaid() draws a diagram Mermaid can parse, whatever the names. An attribute is drawn under a name Mermaid reads as one word (ASCII letters, digits, _, -, not a leading digit, not PK, FK or UK), with its real name first in the note when that changed it; two that would draw alike are told apart by a suffix. An entity name no longer starts with a digit, holds a non-ASCII letter Mermaid 10 refuses, or is a word Mermaid reserves (class, end, one, ...). A ~ in a note -- a choice, a pattern, a name -- is drawn as the look-alike ∼, since Mermaid 10's parser hung on it. Names that already read are drawn as before.
  • to_python refuses a spec name it cannot declare, instead of writing a module that does not parse. A spec's name is the class name, and a TableSpec may be named anything: Odd Name, class, or a name Python would read as another (file as file) wrote class Odd Name(FrameSpec):. It raises SerializationError now, before anything is written, naming a spec.with_name(...) that works and to_yaml(), which keeps any name.
  • What sink_ndjson writes, read() reads back. Polars writes a Decimal to JSON as text ("1.25"), a Duration as ISO 8601 seconds ("PT3600S"), a fixed-size Array as a list, a date inside a list or struct as text, and a column of nothing but nulls with no type -- so a spec's own NDJSON output failed its own validation. read(), and the command line through it, now reads each back as the spec declares it, when every value reads exactly: a decimal from its digits and never rounded, a duration with integer arithmetic to the nanosecond, an array only when every list has its width. Anything else stays as it arrived and is reported. Every case in the test catalogue JSON can hold now reads back equal to what was generated; NaN, which JSON writes as null, cannot.
  • Writing JSON no longer panics on categories. A list of a few hundred Enum or Categorical values panicked Polars' JSON writer (1.x and 2.0 alike), in sink_ndjson and in polspec generate, --output and --failing to .ndjson, .jsonl or .json. Categories are written as their text -- the same file -- and read back as declared.
  • A list or array column named "" generates. It raised a SpecError about its element's seed_name; its values are what a column of any other name would draw.

Changed

  • polspec validate --failing writes each failing row once. A row breaking several claims was written once per claim, so --output and --failing held more rows than the file and a count of rejected rows overcounted. Each row is written once now, in the file's order, its __polspec_finding naming every claim it broke, comma-separated: the two files split the input exactly. ValidationReport.failing_rows() is unchanged -- once per claim, for grouping by claim.
  • A Binary column is refused by name when writing JSON, which has no bytes, with a GenerationError from sink_ndjson and an error from the command line, before anything is written. It panicked inside Polars' writer before -- an exception except Exception does not catch.
  • A column name holding a NUL character is refused when the spec is declared, with a SpecError. It constructed before, then panicked in Arrow's C interface the first time anything generated.

Internal

  • The property tests draw awkward names -- punctuation Markdown and Mermaid read, whitespace, keywords, leading digits, non-ASCII, the empty string and arbitrary text -- and hold generation, validation, YAML, the data dictionary and the ER diagram to them. They found the two defects above.
  • The README's example runs in the test suite, on its own, as someone copying it would run it.
  • Every case in the test catalogue is written with sink_ndjson and read back with read(), and must equal what was generated.
  • A newly known limitation, found by the same properties: a column named * or ^…$ cannot be generated or validated, since Polars reads pl.col() of either as a pattern. It is listed in Known limitations and pinned by a strict xfail, so the day it works the suite says so.

0.15.0 - 2026-10-04

A profile never contradicts its own data. 0.14.0's promise -- a spec does not drift from data it describes -- broke in two places: a country or currency column profiled from a file drifted from that file, and a hierarchy's nulls and a rule's NaN drifted from their own generated output. Both are fixed, each pinned by a test that fails on 0.14.0, and a realistic source is now checked against its own profile. A few repeated country codes also keep their frequencies through synthesize again, which 0.14.0 lost to the format.

Changed, on purpose: a few repeated country or currency codes profile as an Enum again; --sample N is a random sample of the file rather than its first N rows; a composite key's column cannot declare nan_probability.

No seeded output changes. Every spec that constructs under 0.14.0 generates what it did -- checked against the published 0.14.0 across 463 cases -- and no spec file needs migrating.

Performance is unchanged -- generation, validation, drift, profiling and synthesis measured against 0.14.0 on one build of the extension, all within noise.

Fixed

  • A spec drifted from data it describes, three ways. A column with a finite format -- iso_country, iso_currency -- counted every code as a declared value, so a profiled country column drifted from its own source (cardinality_moved: 224 of 249 never appear); a format promises a valid code, not every code, and only choices, Enum categories and extra_values count now. A nullable Hierarchy column drifted from its own output on its null rate, and a column a rule rewrites on its NaN share: drift now skips, on each column a pass writes, exactly the claims that pass overrides -- measured per pass, so a foreign key's null rate is still watched.
  • polspec drift can set every drift option. 0.14.0 added significance, frequency_tolerance and distribution_tolerance to DriftOptions and no flags for them; --significance, --frequency-tolerance and --distribution-tolerance now exist, every flag's default is the option's own -- said once, on DriftOptions -- and a test holds the command to the options. An option out of range is an error: line, not a traceback.

Changed

  • A few repeated country or currency codes profile as an Enum again. 0.14.0 named the iso_country or iso_currency format before a column could narrow, so a column nine tenths GB profiled as every country evenly, and synthesize spread it over 249. A finite format is now named only where the column would not narrow to an Enum -- more distinct codes than max_unique_enum, or a replaced column -- and an Enum keeps the codes' frequencies. Template formats (email, UUID, IP, MAC, hostname) are named first, as in 0.14.0.
  • --sample N is a random sample. polspec schema infer and polspec drift read a file's first N rows, which for a file sorted by date or by amount is not a sample of it: bounds that reject the rest, a distribution that has "moved". Both now draw N rows at random from the whole file, seeded so the same file gives the same result, as polspec synthesize already did; a sample of zero or fewer rows is refused.
  • A composite unique key's column cannot declare nan_probability, as a unique=True one cannot: no NaN equals another. The repair that separates repeated combinations hashed NaN as one value and generated none of a declared share.

Internal

  • docs/llms.txt and docs/llms-full.txt are built by the docs workflow, not committed. Every page or docstring edit regenerated 400 KB of tracked text -- 78 of the repository's first 148 commits touched it. The site serves the same files, built from the same source. So that the build can import polspec without installing it, polspec.__version__ is "0+unknown" in a checkout that was never installed, where it used to raise.

0.14.0 - 2026-09-29

A profiled spec says what the data is, and drift notices when that stops being true. A float declares how often it is NaN (nan_probability); the profiler records that share, names a text column's format -- emails, UUIDs, IPs, country codes -- and makes a text column an Enum only when its values repeat; and drift compares weighted frequencies and declared distributions, reporting a move only when it is significant at the rows seen and large enough to matter.

Breaking, on purpose: - a NaN in a float column that does not declare one is a nan finding; - spec files are format version 4, and polspec 0.13 and earlier refuse every file 0.14.0 writes; - from_dataframe() and schema infer infer differently: formats named, Enums only of repeated values; - drift reports fewer, better-founded findings: no small-sample rate alarms, no dtype that validates, and new codes for frequencies and distributions.

No seeded output changes. Every spec that constructs under 0.13.1 generates what it did -- checked against the published 0.13.1 across 435 cases -- and a version 3 spec file reads as before.

Generation and validation run as fast as they did -- measured against 0.13.1 on one build of the extension. Drift and profiling do more than they did, and take longer for it: on a million rows, drift with two declared distributions 44 ms to 79 ms, profiling with a text column checked for formats 0.10 s to 0.22 s (formats=False skips that).

Added

  • ColSpec.nan_probability: NaN in a float value, declared. How often a present value is NaN, from 0 to 1, on a float column, a list's float elements or a struct's float field. Generation draws NaN on that share of the present rows -- whole and batched alike, from a draw of its own, so a column that declares none generates exactly what it did -- and spec files carry it as nan_probability. diff calls allowing NaN compatible and forbidding it breaking; drift reports a NaN share that has moved, significantly and by at least null_rate_tolerance, as nan_rate_moved. Refused on any other dtype and with unique=True.
  • Profiling records a float's NaN share. profile(), from_dataframe(), synthesize() and both commands record how often a float column -- or a list's float elements, or a struct's float field -- is NaN as its nan_probability, so a source holding NaN validates against its own profile (pinned as a strict xfail since 0.13.1, now passing) and its fake data holds NaN on the same share. The 0.13.1 warning now names infinities alone, which stay a matter of bounds.
  • Profiling names a text column's format. profile(), from_dataframe(), synthesize(), polspec schema infer and polspec synthesize name the format a text column's values have -- uuid4, ipv6, ipv4, mac, iso_country, iso_currency, email, hostname, tried in that order -- when at least twenty distinct values have it and every value does, or 99% of rows do and the rest are at most five repeated stand-ins, which become extra_values at their observed shares. A hostname has to hold a dot. A synthesized email column is email-shaped now, where 0.13.0 made text of an email's length. On by default; formats=False or --no-formats turns it off. A replaced column is named a format only when every value has it.
  • Drift on frequencies and distributions. frequencies_moved compares a weighted column -- an Enum, a Boolean, choices -- with its weights; distribution_moved compares a column that declares a distribution with a sample drawn from the declaration itself, so the clamping at its bounds and an integer's rounding are what generation does. DriftOptions gains significance, frequency_tolerance and distribution_tolerance.
  • A foreign key from a parent that holds NaN is refused unless its column declares NaN too -- the parent's NaN keys would fill it, and fail its own claim.
  • The data dictionary shows a NaN share in the Nullable column (Yes; NaN 5%), and the diagram as a comment.

Changed

  • A NaN a float column does not declare is a finding. The default, nan_probability=0, claims a column holds numbers, so validation now reports a NaN in it as a row-level nan finding, and drift as new_values -- where before a NaN in a column with no bounds passed unremarked. And NaN is no longer judged by bounds or choices: in a bounded column it used to be a bounds finding, since it compares greater than every number; it is now the nan finding alone, and nothing at all where it is declared.
  • Spec files are format version 4. nan_probability is the reason: a float that does not declare it now claims to hold no NaN, which an older reader would not check, so the version has a 0.13 reader refuse the file by name rather than read it more loosely than it was written. Every file 0.14.0 writes carries version: 4 -- polspec 0.13 and earlier refuse all of them, whether or not they use the key. Version 3 files read as before, through a migration that changes nothing.
  • A profiled text column becomes an Enum only when its values repeat. from_dataframe(), profile(), synthesize(), polspec schema infer and polspec synthesize narrowed any text column with at most max_unique_enum (50) distinct values to an Enum of them -- so forty distinct names in a forty-row sample became an Enum of those forty, and the spec rejected every other name. The values now have to repeat, each at least twice on average (polspec.profiler.ENUM_MIN_REPEATS); a column of distinct values stays String, with its lengths, and a Categorical stays Categorical. A spec inferred before is unchanged unless it came from such a column; the same holds for a list's elements and a struct's fields.
  • Drift reports a move only when it is both significant and large. A null rate, an element null rate or a NaN share is reported as moved only when it is at least null_rate_tolerance from the declared rate and unlikely -- at significance, 0.001 by default -- to be sampling noise over the rows seen: fifty rows of a column declared 30% null holding 22% used to be a finding, and a spec's own output could drift from it. The same rule holds for the new frequency and distribution findings, and a declared value missing from the data is cardinality_moved only when, at its declared share, missing it is itself unlikely. A column a pass rewrites -- a rule, a foreign key, a hierarchy, a composite key -- is not held to its weights, distribution or unseen values, none of which its values are drawn by. A spec's own seeded output does not drift from it at 50, 5,000 or 5,000,000 rows.
  • A dtype that validates is no drift finding. String for an Enum, Int64 for an Int32, Datetime for a Date -- what a CSV hands back -- used to be a compatible dtype_changed against data that validate() accepts; it is now reported only under strict_dtypes=True, where it is breaking. diff between two declarations still reports every change.

Fixed

  • A test that runs pytest on a generated file could fail now and then on Windows: left to find its own root, the subprocess scanned directories up into the system temp folder, where another program's file vanishing mid-scan failed collection. It is now rooted in the file's own folder.

0.13.1 - 2026-09-28

Fixes, for data nobody generated. 0.13.0 pointed polspec at real tables; this release makes it answer for them. inspect() promises never to raise for a frame that fails, and three ways a frame arriving in another dtype broke that are closed -- most found by a new property test that carries generated frames through the dtypes real data arrives as. Profiling survives NaN and infinities, a Null column generates, and every function that takes a spec takes a FrameSpec class.

Validation is stricter in two places, on purpose: a value the declared dtype cannot hold (an Int8 column arriving as Int64 with 1000 in it) is now a dtype finding, and a naive Datetime no longer stands in for a zoned one, or a Time or Duration for a Date. Each was a frame that was not what its spec declared, and could end in a raw Polars error one step later; a frame with no findings now always survives validate(cast=True).

No seeded output changes. Every spec that constructs under 0.13.0 generates what it did -- checked against the published 0.13.0 across 435 cases -- and no spec file needs migrating.

Fixed

  • A temporal stands in only for its own kind. Validation's default, permissive dtype check accepted any temporal for any declared temporal, so a Time or a Duration passed for a Date column with no bounds, and made inspect() raise Polars' InvalidOperationError -- and polspec validate end in a traceback -- for one with bounds. A Date and a Datetime still stand in for each other, as text formats need; a Time only for a Time and a Duration only for a Duration, any unit. Anything else is a dtype finding. Stricter: a naive and a zoned Datetime no longer stand in for each other -- one is a wall-clock reading, the other an instant; two zones still do. Drift's dtype_changed severity follows, decided by the same function.
  • A value the declared dtype cannot hold is a finding. A wider dtype stands in for a declared integer or Decimal, and its values were never asked to fit: an Int8 column arriving as Int64 with 1000 in it, a UInt8 with -5, passed -- and validate(cast=True), which only runs on a frame that passed, then raised Polars' conversion error. Stricter: such a value is now a row-level dtype finding keyed <column>__dtype_range, with samples and rows, in list elements, struct fields and map values too. A float or integer too large for a Decimal's integer digits, after rounding to its scale, is one as well. A declared bound has to fit its dtype, so on a side a checked bound closes, such a value is out of bounds too and reported there alone -- which keeps the check free for the usual case, a CSV's Int64 for a bounded Int32. A frame with no findings now always survives cast=True.
  • A claim over a column that arrived as another dtype no longer takes inspect() down. Found by a new property test that carries generated frames through the dtypes data really arrives as. A rule or check reading a column of the wrong dtype -- a Boolean flag read as text -- was compiled anyway and Polars refused it; it is now skipped, as one reading a missing column is, the column being a dtype finding already. A check's is_in over dates refused a Datetime column standing in for a Date. And a column's or rule's choices against a dtype standing in for the declared one -- datetimes against a Date -- are now asked of the column as cast=True would hand it back, so a clean report stays clean once cast. Temporal bounds are measured the same way: a bound a microsecond past midnight, on a Datetime column arriving as a Date, rounded to the day and passed a value the cast then put outside it.
  • A float column holding NaN or an infinity can be profiled. An infinity made profile(), synthesize(), from_dataframe(), polspec schema infer and polspec synthesize fail -- a bound must be finite -- and a NaN was left outside the inferred bounds without a word, so the spec rejected the file it came from. The bounds are now the finite values', and a UserWarning (a warning: line on the command line) names each column holding either, with how many. A spec still cannot declare NaN, so validating the source against its profile reports them; see Known limitations.
  • A Null column generates. A column of nothing but nulls -- what an empty column is in Parquet -- could be profiled but not generated, so synthesize failed on it, and it could not be written to a spec file. ColSpec(pl.Null, nullable=True) now generates all nulls, writes as Null, and validates any dtype holding only nulls, a value in it being a row-level dtype finding. It must be declared nullable, and its null rate is 1.
  • A column named "" came back from generation named column_0, so a spec -- or a synthesized file -- with an empty column name failed.
  • Every function taking a spec takes a FrameSpec class. polspec.generate(Orders, 10) -- and generate_batches, scan, the four sink_* functions, inspect, validate and polspec.sizing.estimated_size -- failed with AttributeError: type object 'Orders' has no attribute 'columns'; each now reads the class as its .spec, as drift, read and the renderers already did. A class that is not a spec is refused naming the class.

0.13.0 - 2026-09-27

Two features: a fake dataset from a real one, and a format with values of your own beside it. polspec.synthesize turns a DataFrame, a LazyFrame or a data file into a stand-in with the same schema, shapes, frequencies and keys, none of its rows, and no value of a column named in replace=; extra_values widens a format -- ISO countries plus UK (ISO), IPv4 or NOT AVAILABLE -- for validation and generation alike.

Three breaking changes, each a refusal of something that already did not work: a choice its dtype cannot hold, and a check naming a column the spec does not declare, are refused at declaration -- both used to fail at generation or validation -- and TableSpec == compares columns in order.

No seeded output changes. Every spec that constructs under 0.12.0 and still constructs generates what it did -- checked against the published 0.12.0 across 519 cases -- and no spec file needs migrating.

Alongside them, a round of fixes found by fuzzing specs nobody wrote down: Time, Duration and Decimal values in spec files, checks Polars refused to evaluate, a skipped rule's claims, and to_python imports.

Added

  • polspec.synthesize: a fake dataset from a real one. From a DataFrame, a LazyFrame or a data file, a frame with the same schema whose columns follow the source's shapes, category frequencies and null rates, whose keys stay distinct, and which holds none of its rows. replace= names columns whose values must not be carried over; text in them is generated from its lengths, never as an Enum of the values the source holds. sample= profiles a random sample of a large source. polspec synthesize does the same from the shell, and --spec writes the spec the data was generated from. Columns are profiled one at a time, so relationships between them are not kept -- see Fake data from real data.

  • ColSpec.extra_values: a format with values of your own beside it. ColSpec(pl.String, format="iso_country", extra_values=["UK (ISO)", "UK (ISLANDS)"]), or format="ipv4" with extra_values={"NOT AVAILABLE": 0.05, "INVALID": 0.01}. Validation accepts a value that has the format or is an extra; generation draws each extra on its share of the present rows -- 1% each for a list, the given share for a mapping -- for a finite format and a template one alike, whole and batched. It is written to spec files (a list, or a mapping when a share is not the default), drifts as a domain (an extra removed is breaking), and takes part in a foreign key's domain check. format beside choices is still refused, and the message now points here.

  • polspec.profile, the half of synthesize that describes the source as a TableSpec, to review, edit or keep before generating.

  • Profiling can fit a shape, find keys, and leave values behind. from_dataframe() and profile_dataframe() take shape=True -- each numeric or temporal column's distribution, fitted against every one the engine draws and kept if it beats drawing evenly (polspec.shape) -- detect_unique=True, and replace=; polspec schema infer takes --shape, --keys and --replace. All three are off by default: a profile with none of them is exactly what it was.

Changed

  • A choice its dtype cannot hold is refused at declaration. ColSpec(pl.Int8, choices=[1000]) constructed, and generation gathered the choice through a lenient cast: it came out as a null, in a column that need not allow one, and the spec failed its own validation. 1.5 on an Int8 came out as 1, a domain nobody declared, and text on a Date column failed inside Polars at generation. A column's choices and each rule's are now held to the column's dtype and refused, naming them, when it cannot hold them as written. Text on a String column, an integral float on an integer one, a date on a Datetime and the nearest float on a float column are all held, as before.

  • A check naming a column the spec does not declare is refused. It constructed, and failed only when validated. pl.all() and a regex selector name no column, and pass.

  • TableSpec == compares columns in order. columns was compared as a mapping, so two specs holding the same columns in a different order were equal though their schema()s -- and every frame generated or validated against them -- differ. They are now two specs.

Fixed

  • Validating a frame that lacks a column a check needs no longer crashes. The check's mask was built regardless, and Polars raised ColumnNotFoundError out of inspect() and validate() -- and out of polspec validate as a traceback. The check is now skipped, as a rule or composite key already was: the absent column is the missing_columns finding, or was allowed to be absent.

  • drop() and select() remove the checks they strand, as they remove a composite or foreign key that loses a member.

  • A rule that cannot be evaluated no longer hands its rows to the next. Rules are first-match-wins; with an earlier rule's column absent (missing_cols="allow"), a later rule was checked against rows the earlier one had rewritten, and reported them as failing. Once a rule cannot be evaluated, the rules after it on that column are not checked either.

  • Time and Duration values can be written to a spec file. YAML has no form for a time of day or a duration, so to_yaml() crashed with PyYAML's RepresenterError on a Time or Duration bound, a choice, a rule's choice, or a check's literal -- a spec could be declared in Python that no file could hold. Both are now written in a tagged form, {time: "12:30:00"} and {duration: {days: 1, seconds: 0, microseconds: 0}}, and read back as themselves. Not a new format version: a file without them reads as before.

  • Decimal choices can be written to a spec file. A Decimal bound was always written as the exact string it prints as; a Decimal choice was written raw, and to_yaml() crashed on it. Choices now follow bounds, and a text choice on a Decimal column is read back as a Decimal.

  • to_python() imports what it writes. Its imports were guessed from the bounds alone, so a Decimal choice wrote Decimal('0.5') into a module that never imported it. They now follow from the source written.

  • Anything else YAML cannot hold is a SerializationError, naming where in the file it sits (columns.price.choices[0]), and raised before the file is opened, so no half-written file is left behind.

  • A check naming a column twice no longer crashes validation. A range check -- (col("a") > 0) & (col("a") < 10) -- names a twice, and the struct its samples were drawn into had two fields of one name: Polars raised DuplicateError. Each column is now named once.

  • is_in over datetimes, times and durations works on a coarser unit. Python holds them to the microsecond, and Polars' is_in refuses a list finer than the column's unit: col("d").is_in([timedelta(0)]) on a Duration("ms") column raised, in a check and in a rule's condition -- and so at generate(). Such a membership is now evaluated as the equalities it stands for.

Internal

  • The Hypothesis round trip draws what it did not: choices from every dtype's own value space, Time and Duration bounds, a rule on a column, and checks whose literals have the column's type -- including a range check that names its column twice. Its first runs found the last three fixes above. A second property draws declarations that cannot mean anything and requires each to be refused as a SpecError.

Documentation

  • Specs as files showed a check comparing against subtotal, a column its Orders does not declare -- the new refusal found it -- and quoted the unrepresentable-check warning as polspec worded it before col().

  • Roadmap and stability no longer says drift takes one spec: polspec drift --all runs over a directory, as generate and validate do.

0.12.0 - 2026-09-26

One seeded change, once, and a fix rather than a choice: the elements of a List, Array or Map column in every batch after the first of generate_batches, scan() or a sink. Every batch used to draw them from the first batch's seed, so a stream repeated its opening elements over and over; each batch now draws its own. generate(), and a stream's first batch, produce exactly what 0.11.0 did, and no spec file needs migrating.

One breaking change: a report is falsy when it is empty. if report: now means something was found; write report.passed or report.unchanged for the opposite.

Alongside them, three more fixes -- a projected cartesian scan, a chunked Array on Polars 1, TableSpec.rename and checks -- reference pages for ColSpec's fields, the spec-file format and the command line, and CI that tests Map, the lowest dependency versions and every release wheel.

Changed

  • A report is falsy when it is empty. ValidationReport and DriftReport defined bool() as "passed" or "unchanged" while len() counted their findings, so an empty report was truthy -- the reverse of every other collection. __bool__ is gone: a report is truthy exactly when it holds findings, so if report: now means something was found. Code that wrote if report: for "passed" must write if report.passed: (or report.unchanged), which has always worked.

Fixed

  • A nested column no longer repeats itself batch after batch. The elements of a List, Array or Map column were drawn from the same seed in every batch of generate_batches, scan() and the sinks, so a stream of any size repeated its first batch's elements over and over. Each batch after the first now draws them from a seed of its own. This changes the seeded output of those batches; generate(), and the first batch, are unchanged.

  • Projecting a method="cartesian" scan no longer changes its values. A scan generated only the columns a projection asked for, but a coverage set is the product of every column, so dropping one changed the rows of the others: scan(..., method="cartesian").select("c") was not the full scan's c. A cartesian scan now generates every column and projects afterwards.

  • Validating an Array column in several chunks no longer panics on Polars 1. Taking distinct samples of an Array column that arrived in more than one chunk -- concatenated, or read from a file in row groups -- panicked inside Polars 1 whenever no row failed, which is every frame that passes. Samples of an Array are now taken as the lists they are; they come back as the same values. Polars 2 was not affected.

  • TableSpec.rename renames checks and validators too. It rewrote rules, keys and the hierarchy but left checks alone, so a check naming a renamed column kept naming the old one, and validating the renamed spec failed inside Polars with a ColumnNotFoundError. A check or validator written with col() is now renamed with its column -- its name following if it was the default -- and one written as a raw pl.Expr that names a renamed column is refused at rename() with a SpecError saying why. A validator written with col() no longer stops its column being renamed.

Documentation

  • Three reference pages, each held to the code by a test: ColSpec fields (every field, what it applies to, and what generation and validation do with it), Spec files (the YAML format key by key, the dtype forms, the predicate operations and the format versions), and Command line (every command and argument, generated from the parser).
  • The how-to guides are grouped -- declaring a spec, using one, specs as values and files, the command line -- rather than listed as fourteen.
  • The home page names Map, patterns, hierarchies, the Registry and read(), and no longer says Polars is the only runtime dependency. The README points into the docs instead of copying part of their nav, and CONTRIBUTING's checks match what CI runs.

Internal

  • The round-trip contract covers the streaming verbs and drawn specs: tests/contracts/test_streaming.py runs the case catalogue (tests/cases.py) through generate_batches and scan, and tests/contracts/test_properties.py holds specs Hypothesis draws to the round trip, whole, batched and through a spec file.
  • Spec files from every older format version are pinned as golden fixtures under tests/serialization/fixtures/, each read against the spec it has to become.
  • CI: a required job on a pinned Polars 2 release candidate, so Map is tested where it gates a merge; a job on the lowest version of every direct dependency; a coverage floor of 90% (branch) on one leg of the matrix; and each release wheel installed alone and smoke-tested before it is published. The docs rebuild when CHANGELOG.md changes.
  • The dev group's pyarrow floor rises to 16, the first that imports beside numpy 2.
  • The package is laid out by side. Declarations (ColRule, ForeignKey, Hierarchy) no longer carry the generation passes that apply them, which live in polspec.generation.passes; the engine, the pipeline and the passes are all under polspec.generation. polspec.constraints is polspec.domain and polspec.pass_order, and polspec.report -- rendering -- is polspec.render, so no two modules share a name with a different meaning. Helpers used across modules lose their leading underscore, and a handful of duplicates are one copy. None of these modules is public API, and no seeded output changes.
  • An import-linter contract (lint-imports, in CI) holds the layering: declarations import no verb, file format or engine; generation and validation import nothing of each other; only generation reaches the Rust extension.

0.11.0 - 2026-09-26

One seeded change, once: a unique=True column filled by a foreign key generates different parent keys for the same seed -- in generate() too, not only when batched -- because it is now a permutation of the parent's keys. That is what makes it unique across a whole frame however it is produced: until now, generate_batches, scan() and the sinks drew it afresh in every batch, so two batches could hand out the same parent key. Nothing else's seeded output changes, and no spec file needs migrating.

Alongside it, Polars 2's Map(key, value) generates, so every Polars dtype that holds data now does.

Changed

  • A unique foreign key is a permutation of its parent's keys, so it is unique across the whole frame. Row r takes the parent key at π(r), for a permutation keyed by the frame's seed -- the one behind every other unique column since 0.10.0 -- so a batch takes the keys the whole frame takes there. This covers a key into another spec and a unique key into the spec's own rows; a many-to-one key is still drawn with replacement, and a __unique_together__ group is still distinct within a batch only.

  • A parent too small for a unique foreign key is refused before the first batch. generate_batches, scan() and the sinks check that the parent holds a distinct key for every one of the n rows up front. Before, each batch needed only a batch's worth, so a frame larger than its parent was produced -- with keys repeating across batches -- rather than refused.

Added

  • Map columns generate, on Polars 2. A Map(K, V) is the list of {key, value} entries Polars stores it as, and polspec declares it as one: list_length is how many entries a map holds, and fields says what its key and value are. Every map is drawn with its keys present and distinct -- a key that repeats within its map is redrawn before the cast, which would otherwise fold the two entries into one. With no list_length, a map holds up to five entries, or as many as its keys can take: two, for Boolean keys.

A nullable key, a declared list_length longer than the keys can fill, and choices, weights or element_null_probability on the map itself are refused at declaration.

  • A Map validates, drifts, profiles and is sized as its entries. A finding about a key or a value is named m.key or m.value, the length m__list_len, and a map holding one key twice -- which a map read from Arrow can -- is m.key__unique. A map whose key or value is wider than declared stands in for it unless strict_dtypes=True. from_dataframe re-declares a map by its key and value, and the generated docs list its entries.

Internal

  • The keyed permutation is reachable from Python as _ffi.permuted_indices(domain, seed, start, n): a window of it is the whole range's slice.
  • The dtype census covers Map where Polars has one, and its list of dtypes not generated yet is empty.
  • A docs example marked <!-- docs: polars2 --> runs only where Polars 2's dtypes exist.

0.10.1 - 2026-09-25

Ready for Polars 2. polspec passes its whole suite on the Polars 2.0 release candidates, and CI now runs it there on every change; the bound stays polars>=1.44.1,<2 until 2.0 itself is released and the same suite is green against it. Along the way, testing against Polars 2 turned up two bugs that Polars 1 users hit today, both fixed here. No seeded output changes, and no spec file needs migrating.

Fixed

  • choices on a Decimal, or a Datetime not in microseconds. Drift failed on a Decimal column with choices; validation raised an error on a Datetime("ms") column with them, and on a Datetime("ns") column rejected the column's own generated data. The choices reached Polars' is_in as a Python list, which arrives at full precision or in microseconds. Validation, a rule's choices and drift now compare against the choices as a Series of the column's own dtype -- which is also what Polars 2 requires of an integer choice on a float column.

Added

  • scan() names itself in explain() on Polars 2. A scan's node reads PYTHON[polspec: Orders] SCAN, with an INFO: line giving the rows, the seed and the batches. Whether to pass the labels is read off register_io_source's signature, so on Polars 1 the call is unchanged and the header stays PYTHON SCAN.

  • Polars 2's Map dtype has a spec-file form, {Map: {key: <dtype>, value: <dtype>}}, so a profiled map column can be saved -- polspec schema infer on a file with one no longer fails. A Map column declares, validates, profiles and drifts by its dtype; generate() refuses it by name until drawing its values arrives.

Internal

  • A polars-next CI job runs the suite on the newest Polars 2 pre-release, informational until 2.0 is final.

0.10.0 - 2026-09-25

One seeded change, once: every unique=True column generates different values for the same seed -- in generate() too, not only when batched -- because a unique column is now a permutation of its value space. That is what makes it unique across a whole frame however it is produced: until now, generate_batches, scan() and the sinks drew a unique column afresh in every batch and the same way each time, so a key repeated every batch (0.9.2 warned; this release fixes it). Nothing else's seeded output changes, and no spec file needs migrating.

Around it: Orders.read(path) reads a file in the spec's terms, report.passing_rows() and polspec validate --output / --failing split a file into what passed and what did not, a list's elements can be null, and the test suite is organised by what each file covers.

Changed

  • A unique column is a permutation, so it is unique across the whole frame. Its value at row r is the π(r)-th value of its space, where π is a keyed permutation -- a Feistel network walking its cycle back into range -- so distinct rows get distinct values and a row's value depends only on the row and the seed. Every unique kind: integers, booleans, choices and Enum (an offset into the domain), floats (an evenly spaced grid over the bounds, four ulps apart at the widest magnitude so a cast cannot merge two points), dates, Decimal, Int128, Float16, and strings and formats, whose values decode from an index one digit per template part. A batch of generate_batches, a scan() or a sink is now the whole frame's slice for a unique column as for any other, and the 0.9.2 warning is gone with what it warned about. __unique_together__ and a foreign key drawn without replacement are still distinct only within a batch.

Two consequences to know. A unique string is drawn uniformly from every string its lengths allow, so it favours its longest length -- 98% of a string_length=(5, 15) column is fifteen characters. And a null row still takes its place in the permutation, so a nullable unique column's domain has to hold every row, nulls included.

  • Much faster, and bounded in memory. The unique draw no longer holds a set of the values it has drawn, and fills chunk by chunk in parallel like any column: 10 million rows from a crowded integer domain take 0.016s where they took 1.42s, and a million unique strings 0.014s where they took 0.185s.

Added

  • Orders.read(path) (and polspec.read(spec, path)): a data file in the spec's terms. The extension picks the reader -- the set polspec validate reads -- Polars' options pass through (separator=";"), and each column the spec declares as a Date, Datetime or Time that arrived as text is parsed when every value parses, leaving date-shaped text in a String column alone. Nothing else is cast: the frame goes to inspect() or validate(cast=True). "Checking a file you were given" starts with it, and the CLI reads through the same code.

  • report.passing_rows(): every row no finding touched, lazily and in order -- the complement of failing_rows(), so the two split a frame into what to keep and what to send back. A structural finding judges the whole frame, so it refuses until that is fixed.

  • polspec validate --output PATH --failing PATH: the passing rows, validated again with cast=True so the file is one the spec accepts and typed as it declares, and the failing rows with the claim each broke. The exit status is unchanged, so a pipeline can quarantine the bad rows and still fail the step; notes go to stderr so --json stays one document.

  • element_null_probability: how often an element inside a list is null. 0 by default -- elements are never null, as before -- and any rate up to 1 accepts them, on a List or an Array, a struct or list element null as a whole. Generation, validation, the spec file (an added optional key), diff and drift (which now measures element nulls), the profiler, the data dictionary and estimated_size all read it.

Internal

  • The test suite in folders, by what each file covers: contracts/ (the round trip, and every "each field has an entry" test in one test_parity.py), declaration/, generation/, validation/, drift/, serialization/, features/ (one file per feature across every verb), cli/ and docs/. The same 1135 tests collected before and after the move; a test now fails on a test file at the top level or without a docstring.

0.9.2 - 2026-09-24

The fixes from a review of 0.9.1, and the three numeric dtypes 0.9.0's "every dtype generates" missed: Int128, UInt128 and Float16 now generate, and a census test fails the day Polars adds one that does not. The one to know about: a unique=True column repeats its values in every batch of generate_batches, scan() and the sinks -- that has been true since batching arrived, and the docs understated it. This release warns when it happens and says so plainly; 0.10.0 makes the draw batch-stable. No seeded output of an existing spec changes, and no spec file needs migrating.

Added

  • Int128, UInt128 and Float16 generate. They declared and validated before, but generate() failed with an engine error and to_yaml could not write them. Each is drawn through a type the engine has and cast back, as a Decimal is: a 128-bit integer through 64 bits -- bounds past that are refused at generate(), naming the column, while validation checks the full range -- and a half as a single, rounded to the nearest half. The draw stays between the halves nearest each bound from inside, so rounding never carries a value past one; bounds too close together to hold any half are refused where they are written, and a unique half is drawn from the finite set of halves between its bounds. YAML, to_python, estimated_size and from_dataframe know all three.

"Every dtype" now reads every dtype that holds data: Null, Object, Unknown and extension types declare and validate by dtype, and generate() refuses each by name. A census test walks every dtype class Polars exports and fails on one that is neither.

Changed

  • A unique column that repeats across batches says so. Under generate_batches, scan() and every sink_*, a unique=True column's draw starts over in each batch the same way, so every batch holds the same values -- the docs called it "drawn per batch", which read as independent draws. polspec now warns once, as the second batch is drawn, naming the columns and pointing at generate() for a frame that must be unique throughout; a run that fits in one batch, or a scan that never collects the column, stays silent. The limitations and generating pages describe what happens.

Fixed

  • A finding on a zoned Datetime no longer panics on Windows. A finding's samples are Python values, and a zoned datetime needs the IANA database to become one; Windows ships none, so Polars panicked -- past any except Exception -- on the first finding on such a column. tzdata is now a dependency on Windows.

  • ColSpec(pl.Struct({})) generates a Struct({}) column, where it generated an untyped null one.

  • Docs: comparison.md no longer says polspec has no drift detection, and architecture.md's module table gains the seven modules it had fallen behind on, pinned by a test.

Internal

  • Warnings are errors in the test suite, so a warning is asserted where it is expected -- with pytest.warns, or the doc runner's new <!-- docs: warns --> marker -- and a failure everywhere else.

0.9.1 - 2026-09-24

A patch release around one question: someone hands you a CSV, and there is a spec it should meet -- what do you do? The CLI now reads a text file's dates the way the spec declares them, where before every CSV with a date column failed polspec validate; the docs walk the flow end to end; and bounds can be set aside while you find out what the ranges really are. No seeded output changes, and no spec file needs migrating.

Added

  • Checking a file you were given, a section of Validating data: read the file with try_parse_dates=True rather than the spec's schema (which stops at the first bad value), inspect() it for every problem at once, decide whether the file or the spec is wrong, then validate(cast=True) for the typed frame. The example runs in the suite.

  • polspec validate --skip CHECK, repeatable, for any of the validate_* switches -- --skip bounds --skip checks. Its choices come from the switch list itself, so a switch added later reaches the CLI.

  • polspec.__version__, the installed version.

  • validate_bounds=False turns off the bounds checks, beside the other validate_* switches and as ValidationOptions(bounds=False). The other switches cover what generation cannot yet satisfy; this one is for real data, whose ranges you may want to look at before holding it to them while every other claim is still checked. It covers every bounds, a List's elements and a struct's fields included; string_length and list_length stay on.

Fixed

  • The CLI reads a text file's dates in the spec's terms. A CSV or JSON file has no date type, so a Date column arrived as text and validation reported its dtype and checked nothing else -- every CSV with a date column failed polspec validate, including one polspec generate had just written. validate, drift and both --all modes now parse each column the spec declares as a Date, Datetime or Time that arrived as text, keeping the parse only when every value parses: a column with a bad value stays text, so its dtype finding stays true, and a String column of date-shaped text is left alone. schema infer, with no spec to go by, reads a CSV with try_parse_dates and declares a date column as a Date.

  • estimated_size no longer charges a null row for content it does not hold. A nullable text or List column was estimated as if every row held a value -- 23% high for a string column at a 30% null rate, 32-38% for a list. A string's bytes past its view and a list's elements now scale by the share of rows expected to be present; views, offsets, fixed-width values, an Array's slots and a struct's fields are paid on a null row too and are unchanged.

  • The API reference for FrameSpec.validate, diff and drift rendered prose after their parameter lists as parameters named A, that and Uniqueness. It is a Notes section now, and a test parses every docstring in the package as the reference does, failing on a warning.

0.9.0 - 2026-09-24

Two things are removed, as 0.8.0 announced: generate(lazy=True), which is now a TypeError -- scan() is the lazy frame -- and the polspec[arrow] extra, which no longer resolves. Both are under Removed below. Everything else in the release is additive.

The headline is Struct, the last dtype polspec declared and could not generate. ColSpec(fields={name: ColSpec(...)}) says what is claimed about each field's values, so a field is described exactly as a column of its dtype would be, and every part of polspec reads it: generation, validation, the spec file, diff and drift, from_dataframe, the data dictionary and estimated_size. The same recursion generates a List(Struct), a List(List) and an Array(Struct), so every dtype now generates, nested to any depth. No seeded output of an existing spec changes, and no spec file needs migrating.

Added

  • Every dtype generates. A Struct column is generated from its fields -- one column per field, gathered -- and the same recursion generates a List(Struct), a List(List), an Array(Struct) and a struct of either, nested as deeply as the dtype goes. There is no dtype left that polspec declares and cannot generate, and test_roundtrip.py pins that claim rather than the list of exceptions it used to hold.

A field is drawn by the code that draws a column of its dtype, and is seeded under its parent by name: renaming a struct column with seed_name keeps every field, adding a field beside one moves nothing, and a struct column is a window under generate_batches and scan() like any other. The column's nullable is the cell -- a null struct -- and a field's own nullable says whether it may be null inside a struct that is present.

Validation checks each field's claims in place, and a finding names the field: key point.lat__bounds, message Column 'point.lat', while its columns stay ("point",) so report.rows() returns the offending rows. Inside a list the samples are the offending lists, and a list of lists tells its levels apart (c__element_null, c[]__element_null). A struct in the data matches a declared one by field name rather than order, each field compatible by the usual rules -- an Int32 field stands in for a declared Int64 outside strict_dtypes.

  • ColSpec(fields=…): what is claimed about a struct's field values. A Struct column's dtype is its schema -- every field's name and type comes from it -- and fields is a ColSpec per field saying what its values are, so a field is described exactly as a column of the same dtype would be:
ColSpec(
    pl.Struct({"lat": pl.Float64, "lon": pl.Float64}),
    fields={"lat": ColSpec(pl.Float64, bounds=(-90, 90))},
    nullable=True,
)

It is partial: a struct of twenty fields where one needs bounds spells one field. A name the dtype does not declare, or a field spec whose dtype disagrees with the struct's, is refused where it is written, as are unique, rules, validators, seed_name and col_name -- each a claim about a column among columns, which a value inside a struct is not. A List of a Struct takes fields too, and a field may itself be a struct, so a declaration nests as deeply as the dtype does.

A Struct dtype now has a spec-file form ({Struct: {name: <dtype>}}), so a struct column round-trips through YAML and to_python -- and so do List(Struct) and List(List), which had no written form before. The format version stays at 3: fields: is an added optional key.

  • The rest of polspec reads a struct too. diff and drift compare a struct's fields as they compare columns -- every comparator, keyed by path, so describing point.lat with bounds is domain_narrowed on point.lat (breaking, as on a column) and data escaping them is bounds_exceeded there; a field's null rate is measured inside the structs that are present. from_dataframe re-declares a struct column by its fields, a List of structs by its element's, and a List of lists by its outer length, where it used to record the dtype alone. The data dictionary gives each field a row (point.lat) under its column, and estimated_size costs a struct as the sum of its fields.

Removed

  • generate(lazy=True), deprecated in 0.8.0. It built the whole frame and called .lazy() on it, so it looked like the lazy verb while being the eager one. scan() is the frame that has not been built; .lazy() on a generated frame is a handle on one that has. Passing lazy= is now a TypeError.
  • The polspec[arrow] extra, redundant since 0.8.0 made the sinks Polars' own. Nothing in polspec needs PyArrow, so polspec[arrow] no longer resolves -- install polspec and, if you want PyArrow for your own code, PyArrow.

0.8.0 - 2026-09-22

The headline is scan(): a LazyFrame that generates rows as they are collected, so a plan that wants five rows of one column out of fifty million generates five rows of one column. Projection is exact rather than approximate, because 0.7.0 keyed every column and every pass by name. Around it, a batch becomes a window onto one frame -- the same rows whatever the batch size -- the sinks become scan() written out and stop needing PyArrow, estimated_size() says how large a frame will be before it is allocated, and generate(lazy=True) is deprecated in favour of the verb that means it. Every seeded batched or sunk output changes, once; generate() is unchanged, and no spec file needs migrating.

Added

  • estimated_size(n), and a word before a large frame is allocated. Orders.estimated_size(50_000_000) reads the answer off the declaration -- the width of each dtype, the lengths the spec declares -- so it costs nothing and needs no data. generate() now estimates before it allocates: past four gibibytes it warns, naming the figure and pointing at scan() and generate_batches(); max_bytes= makes it a refusal instead, for a CI job that should fail rather than swap, and max_bytes=0 silences both. polspec generate prints the same note. The estimate is the frame, not the process peak -- generation holds working buffers on top, most visibly for Decimal and List -- and for a frame of scalar columns it is within a percent of measured.

  • scan(): a LazyFrame that generates rows as they are collected. Orders.scan(50_000_000, seed=1) builds nothing; the plan decides what is drawn. .sink_parquet(...) streams in bounded memory, .select("total").head(5).collect() generates five rows of one column, and a predicate filters rows that were drawn rather than narrowing the draw. Projection is exact: every column is seeded by its name and every pass by what it is for, so lf.select(cols).collect() is always lf.collect().select(cols) -- where a column depends on others (a rule's when columns, a composite key's members) the closure is generated and dropped again on the way out. A scan is batched, so it carries generate_batches' terms: a __hierarchy__ is refused, and uniqueness holds within a batch. Registry.scan_all() does the same for a set of specs, generating parents eagerly -- a foreign key needs the whole parent column -- and the children lazily. Built on polars' register_io_source, which polars marks unstable; the tests pin behaviour rather than the API surface.

  • polspec drift --all specs/ data/, completing the --all trio: every spec under a directory measured against the data file named after it, specs without one listed and skipped, --fail-on deciding the exit status across all of them, and --json printing one report per spec. diff compares two declarations rather than a declaration and data, so it takes no --all.

Changed

  • The four sink_* functions are scan() written out. Each is now scan(...) handed to the matching LazyFrame.sink_*, so PyArrow is no longer needed for the Parquet and IPC sinks -- nothing beyond Polars is needed at runtime, and the arrow extra is redundant. Signatures, defaults and the n=0 behaviour are unchanged; what a sink's **kwargs reach is now polars' own sink rather than pyarrow.parquet.ParquetWriter or pyarrow.ipc.new_file, so a call passing a PyArrow-only writer option needs the polars spelling instead. compression= is typed as the literal polars accepts rather than str, so a misspelling is caught where it is written. The polspec[arrow] extra still resolves and is removed in 0.9.
  • generate(lazy=True) is deprecated, and removed in 0.9. It builds the whole frame and calls .lazy() on it, so the memory is already spent: use scan() for a frame that generates as it is collected, or .lazy() on the result for a handle on an eager one.
  • A batch is a window onto one frame. generate_batches and the sink_* functions used to seed batch k from the k-th draw of the caller's seed, so the same seed with a different batch_size was a different frame and no batch could be made without the ones before it. The engine now takes a row offset and numbers its chunks from it, so for a column no pass rewrites pl.concat(generate_batches(n, batch_size=b, seed=s)) equals generate(n, seed=s) for every b; a List column's lengths are a window too. Rules, foreign keys, composite keys and List elements are still drawn per batch -- deterministic, keyed by the batch's offset, but not row for row the whole frame's -- and uniqueness still holds within a batch. Every seeded batched or sunk output changes, once; generate() is unchanged. A batch smaller than the engine's 65,536-row chunk costs up to one chunk of extra draws.

Internal

  • A link to a heading that does not exist fails the test suite, not just zensical build --strict. tests/test_docs.py checked that a linked page existed but never its anchor, so a wrong slug reached CI before anyone saw it.
  • polspec.validation.constraints is a package, one module per kind of claim: _values (what one value must be, lifted over a List's elements), _rules, _table (composite keys, checks) and _relations (foreign keys, the hierarchy), with the _Constraint base and the dtype rule in _base. No import outside the package changed.
  • A generated column is dropped as it is finished, so a temporal, Binary or gathered column never exists twice while the frame is assembled.

0.7.0 - 2026-09-22

The headline is nested and decimal dtypes: List, Array and Decimal columns generate, validate and round-trip, described by the same fields a scalar column takes. Around it, one change to what a seed produces -- the passes that run after the columns are filled are now keyed by name, so inserting a column never changes its neighbours -- and, alongside it, the type checker moves to ty with nothing suppressed, the CLI gains --all, and four long-standing limitations close, the last strict xfail among them. Specs with rules, foreign keys, a hierarchy or a composite key produce different values for the same seed than 0.6 did, once; see Changed. No spec file needs migrating.

Added

  • pl.List and pl.Array columns generate, validate, and round-trip. A nested column is described by the same fields as a scalar one, read as claims about each element: ColSpec(pl.List(pl.Int64), bounds=(0, 10), list_length=(1, 5)). bounds, choices, weights, format, pattern, string_length and distribution describe the values inside the list; the new list_length describes the list (an Array takes its length from the dtype); nullable describes the cell, and an element is never null. Generation draws the lengths and the elements as two columns of the inner dtype and wraps one by the other, so an element is made by the same code that makes a scalar of its dtype -- every generatable dtype, Enum, Decimal and a format included -- and a List column keeps its data across a rename through seed_name. Validation runs every element claim inside the list and fails a list where any element does; list_length is a new finding code and a null element a nullability one. diff compares list_length like string_length; drift measures a List column as its elements and its lengths; from_dataframe declares one by its elements; {List: <dtype>} and {Array: {inner: <dtype>, width: N}} are the spec-file forms. unique and rules are refused on a nested column, and a List of a List or Struct declares and validates by dtype only. What generation cannot fill is now Struct and those.
  • pl.Decimal columns generate. A Decimal(precision, scale) is an integer and a scale, so it is drawn as the integer through the engine's 64-bit kind and scaled back to the declared type -- with bounds, a distribution, choices, nullability and cartesian coverage like any other numeric column, and a default range of the float default or the widest the precision allows. Bounds are held exactly (int, decimal.Decimal, or a string read exactly, which is how a spec file writes one); an endpoint finer than the scale is refused rather than rounded. Validation compares values whatever the frame holds them as (a CSV hands a Decimal back as a float), drift measures the extent in the column's own type, from_dataframe declares one with its extent, and {Decimal: {precision: P, scale: S}} is its spec-file form. The ungeneratable dtypes are now the three nested ones.
  • ColSpec(seed_name=...): rename a column without changing the data it generates. Each column is seeded from the frame seed and its name, so a rename had always changed a column's values -- which matters when generated frames are snapshots other things are compared against. A column declared with seed_name="old" is seeded as "old" and keeps producing what it did. Declared, not guessed: rename() does not set it, and two columns of one spec cannot share one (they would draw identical values). It holds across a rename and nothing else: the passes that run after the columns are filled draw their seeds in declaration order, so an inserted rules column still reshuffles the ones after it, and that boundary is pinned and stated in Known limitations.

Documentation

  • Generating data said each column derives its seed "from its position". It is from its name -- the reason seed_name is needed at all.

  • polspec generate --all and polspec validate --all. With --all, SPEC is a directory of specs: generate writes one <name>.<format> per spec into -o DIR, parents first with their keys threaded into their children, as Registry.generate_all does; validate checks each spec against DATA/<name>.<suffix>, every spec seeing the others' files as parents, as Registry.inspect_all does, exiting 1 when any fails and printing one report per spec with --json.

Changed

  • A rule leaves a null a null. A nullable column with rules used to lose nulls on the rows a rule matched, ending up below its declared null_probability. The column's nullability is decided when it is drawn; a rule says what a value on a matched row is, which is also all validation checks it against.
  • to_mermaid marks one primary key. A lone unique=True column is the entity's PK; when several columns are unique each is a UK, as a __unique_together__ member already was.
  • A CatSpec refuses two entries differing only in case (STATUS and status), naming both: lookup is case-insensitive, so they were one name with two answers.
  • Pass seeds are keyed by name, not drawn in order. The passes that run after the columns are filled -- rules, the hierarchy, foreign keys, composite uniqueness, a bounded categorical's pool, cartesian coverage representatives -- used to take their seeds from one random.Random in declaration order, so inserting a rules column shifted every later pass and seed_name could not hold a ruled column's data. Each now mixes the frame seed with a key naming what it is for, by the same construction the engine uses for columns. A spec with any of those produces different values for the same seed than 0.6 did, once; a spec with plain columns is unchanged, and so is every column's own draw. The limitations.md bullet that described the old behaviour is gone.

Fixed

  • generate(0, method="cartesian") returns the empty typed frame generate(0) and the sinks do, rather than the whole coverage set. The last strict xfail in tests/test_roundtrip.py is gone with it.

Internal

  • ty replaces mypy, and nothing is suppressed. The 23-module ignore list was one pattern: ColSpec, Check and ForeignKey annotated their fields with what the constructor accepts (pl.Int64 or pl.Int64(), a tuple or a Bound, one validator or several) while __post_init__ narrows every one to what the instance holds. The fields now say what they hold, and each class declares the accepted signature in an __init__ that exists only for type checkers, held to the fields by a test. Every spec.dtype.is_integer() in the library type-checks; the cast() accessors drift needed are gone; the API reference and llms-full.txt show the accepted signature. CI runs uv run ty check.
  • polspec.cli is a package, one module per verb, with the readers, writers and spec loaders every verb shares in _io.
  • validate()/inspect() and diff()/drift() shared 25 lines of options handling twice; it is one polspec._options.options_from.
  • Bound.closed(): both endpoints of a bound that has both.

0.6.0 - 2026-09-14

The headline is drift: a report of what changed between two specs, or between a spec and the data it describes, with a severity per finding under one mechanical rule -- and three CLI verbs so that report can gate a pull request or a nightly load with no Python. Alongside it, pattern= completes the split format= began. Nothing in the engine changes; no seed produces a different frame, and no spec file needs migrating.

Added

  • Drift as a report. polspec.drift.diff(old, new) says what changed between two declarations; polspec.drift.drift(spec, df) says how data has moved relative to what its spec declares -- values outside the domain, a bound exceeded and by how much, a format no longer matched, a null rate that moved, declared values never seen. Both return one DriftReport, also reachable as Orders.diff(Other) and Orders.drift(df):
report = OrdersV1.diff(OrdersV2)
report.breaking          # a narrowed bound, an added column, a new constraint
report.compatible        # a widened domain, a removed check
report.to_markdown()     # the shape of a pull-request comment

Every finding carries a severity under one mechanical rule: breaking when a frame that satisfied the old side could fail the new one -- decided, for dtypes, by the same function validation uses. Two tests hold the two sides to it: what generate() produces never drifts breakingly from its own spec, and every breaking finding against data is a column validate() reports. Sixteen closed finding codes, a DriftOptions with explicit tolerances, and a comparator per ColSpec and TableSpec field with a parity test, so a field added to a declaration is one entry or one red test. DriftReport, DriftFinding and DriftOptions are exported from polspec.

  • ColSpec(pattern=...): a regular expression every value of a String column must match, checked by validation only. Generation does not read it -- the column is filled with ordinary random text, and the round trip holds only with validate_pattern=False, exactly as for validators. That is the honest half of the split format= made: any regex can be checked, the curated set can be generated. A pattern finding code, a validate_pattern switch on ValidationOptions, one pattern: key in spec files (the format version stays at 3: an added optional key is not a new version, and migrations.py now says so). Cannot be combined with format; compiled by Polars' own engine at declaration, so a pattern Polars cannot run is refused with its message.

  • Three CLI verbs. polspec generate SPEC -n N -o FILE writes generated rows to any format the CLI reads (--seed, --method, --references as for validate); polspec diff OLD NEW and polspec drift SPEC DATA print a drift report as text, --json or --markdown, and gate on it with --fail-on breaking|any|none -- so a schema change in a pull request, or a nightly load, can fail CI with no Python:

polspec generate orders.yaml -n 1000 -o orders.parquet --seed 1
polspec drift orders.yaml orders.parquet
polspec diff main/orders.yaml pr/orders.yaml --markdown --fail-on breaking

Documentation

  • Five claims the docs had stopped being true about are fixed, and one of them is pinned: unique=True and __unique_together__ had been listed as work generation does not attempt (generated since 0.2.0); generate() on an unsupported dtype raises SpecError, not TypeError; every yaml example carried version: 2 at format version 3, and a test now holds each version: in the docs to FORMAT_VERSION.
  • The roadmap gains a Deferred on purpose section, starting with the ColSpec -> Domain restructure that two release plans had set aside without saying so on the page, and the condition under which it would be revisited.

0.5.0 - 2026-09-11

The headline is format=: a String column that says what its values look like, and is generated to satisfy its own validator. Around it, the release closes five silent edges, settles the validation option surface, spells out the FrameSpec signatures, and puts a type checker in CI. One narrow breaking change, under Changed.

Added

  • ColSpec(format=...): a String column that says what its values look like, and is generated to satisfy its own validator. Eight formats -- uuid4, email, ipv4, ipv6, mac, hostname, iso_country, iso_currency -- each with a sampler in the engine and a check in validation declared side by side in polspec.formats, and pinned by a round trip per format:
class Users(FrameSpec):
    user_id = ColSpec(pl.String, format="uuid4", unique=True)
    email   = ColSpec(pl.String, format="email")

Users.validate(Users.generate(1_000_000, seed=42))   # passes

Validation reports a new format finding. A format is one format: key in a spec file and takes part in the domain check a foreign key runs at declaration. It cannot be combined with choices or string_length, and only a String column can carry one; each is refused with a message saying which to drop. What a format promises is syntax: see Known limitations.

  • ValidationOptions is exported from polspec, and validate() and inspect() take it as options=. Every switch as one value, for when the same settings go through several calls:
lenient = ValidationOptions(extra_cols="drop", checks=False)
Orders.validate(df, options=lenient)
Customers.validate(other_df, options=lenient)

options= and the individual keywords are alternatives rather than a base and an override, so passing both raises instead of quietly picking one.

Changed

  • FrameSpec.validate, inspect and the four sink_* classmethods spell their options out instead of forwarding **kwargs. Editors complete them, a type checker sees a typo, and a mistyped option is a TypeError from the classmethod's own signature rather than from a function several frames away. The validation keywords default to None, meaning "the ValidationOptions default", so the defaults are still defined once. The sinks keep their trailing **kwargs: that is the documented passthrough to the underlying writer, not a gap.

  • A misspelled spec name in references={...} now warns instead of passing silently. A key nothing points at was skipped without a word, so the column was generated freely while the caller believed the parent had been used -- and validate() then reported the key as unresolved, which is the round trip failing with only its second half audible. The warning names the key that went unfilled and suggests the one supplied:

Orders: references={...} supplied ['Custmers'] that no foreign key points
at, while 'Customers' (supplied 'Custmers'?) went unfilled.

It takes both halves to warn -- something supplied that went unused and something unused that went unfilled -- so supplying no parent at all stays silent, as documented, and a Registry handing every spec the whole set of frames says nothing either. generate_batches warns once per call rather than once per batch. - ColSpec(pl.Int64, null_probability=0.9) now warns. nullable=False still wins and the rate is still ignored -- turning nullability off should not also require deleting the rate beside it -- but the same silence covered asking for nulls and forgetting nullable=True, where the column generates none and nothing says why. Only a rate that cannot be a leftover warns: the default and an explicit 0.0 already agree with nullable=False. - references= given something that is not a mapping raises SpecError naming the three key forms it accepts, rather than an AttributeError from inside resolve_references. Every collection argument elsewhere in the API is a sequence, so passing one here was an easy mistake with an unhelpful answer, and it was the one complaint polspec made that was not a PolspecError. - An unknown validation option names the option you meant. It used to surface as ValidationOptions.__init__() got an unexpected keyword argument 'validate_uniqe', naming a private class that is not exported and not the option intended; it is now Unknown validation option(s): 'validate_uniqe' (did you mean 'validate_unique'?), with the accepted list.

  • Breaking, narrowly: the check switches have one spelling. inspect(spec, df, unique=False) used to work alongside validate_unique=False, while validate(spec, df, unique=False) raised -- a second public spelling reachable through half the API, from a rename map that was meant to be internal. Only the validate_* form is accepted now, by both verbs. The bare names live on as the fields of ValidationOptions, which is where they were always meant to be.

Documentation

  • Decimal joins List, Struct and Array on the list of dtypes that declare and validate but cannot be generated. It had been missing from both Dtype coverage and Known limitations, and it is the one people miss, being the only one of the four that is not a nested type.
  • A Datetime carrying a time_zone generates, which the docs had never said either way and readers assumed meant no. Both claims are now pinned by tests in tests/test_roundtrip.py, so neither page can go stale.

Internal

  • The four copies of "declares no ColSpec columns" are one tablespec.require_columns.
  • mypy runs in CI. The package ships py.typed and a stub for the Rust extension, so every consumer's type checker trusts these signatures, and nothing was checking them. The first run found 247 errors, of which 87 trace to two declarations: ColSpec.dtype and ForeignKey.references are both annotated with what the constructor accepts rather than what the instance ends up holding. Narrowing either is a design decision on a public field, so the modules carrying that backlog are listed in pyproject.toml and everything else is enforced -- the list can only shrink.
  • The predicate nodes in expr.py no longer each carry their own root_names, literals and rename. A node reports its operands through children() and the three traversals are derived from that on Pred, which turns thirty-six implementations into twelve. rename is the one that mattered: the base implementation returned self, so a node that forgot to override it left a renamed spec pointing at a column that no longer existed, with nothing raised. Forgetting children() now raises.
  • References and Method were declared identically in four and two modules; _collect/_to_lazy in three. They are one polspec.frames.
  • The four sink_* functions built the same six-argument batch-stream call each. _prepare now returns the checked call as one value. The public signatures stay spelled out, which is what makes a typo in one of them fail at the call site.
  • registry.py imported serialization lazily in five methods and report in a sixth, while importing generation and validation at module level. There was no cycle; all six are hoisted.

0.4.1 - 2026-09-09

The hierarchy release. A self-referencing ForeignKey says that every parent value exists somewhere in the frame, and nothing more. This adds the declaration that says the rest of it -- one parent per reference, a bounded depth, no cycles -- and makes generate() satisfy it rather than leaving it to the draw.

Added

  • Hierarchy, declaring that two columns of a spec are a parent/child edge list drawn on one pool of references: a child pointing at its parent, and that parent pointing at its own parent, are the same row shape.
class Links(FrameSpec):
    PARENT_REF = ColSpec(pl.String)
    CHILD_REF = ColSpec(pl.String)
    __hierarchy__ = Hierarchy(
        child="CHILD_REF", parent="PARENT_REF", max_depth=5
    )

generate() produces a real forest: one parent per reference, so every row resolves to a single ultimate parent, and no chain longer than max_depth with at least one reaching it exactly -- so a test of a graph walk exercises the boundary rather than whatever the draw happened to give. branching shapes the tree, or roots pins the number of ultimate parents.

generate(n, cycles=10, self_references=5) then breaks it on purpose, which is the other half of testing a graph walk: a resolver written without a visited set does not fail on a loop, it runs forever. The spec still says the data should be acyclic, so validate() reports what was injected -- hierarchy_cycle, hierarchy_depth and hierarchy_multi_parent are new finding codes -- and a test can assert that its own resolver and polspec agree about what is broken.

Both checks are bounded, so validating deliberately cyclic data terminates: depth costs max_depth steps and cycle detection walks by pointer doubling, covering a million-row chain in about twenty. A validator that walked until it reached a root would hang on the fixtures this feature exists to make.

generate_batches and the sink_* functions refuse a spec carrying one: a forest spans the whole frame, and batches are generated independently. See Hierarchies and link tables.

Documentation

  • The spec file format is version 3, which adds the hierarchy: key. A version 2 file loads unchanged.
  • Known limitations now points a self-referencing ForeignKey at Hierarchy for the case that wants a real tree, rather than carrying a recipe of its own. Two tests pin the two apart.

0.4.0 - 2026-09-08

The internals release. 0.2.0 and 0.3.0 settled the vocabulary; this one goes underneath it, to the Rust generator and the places where the same table was being maintained in two or three languages.

Nothing about how a spec is written changes. One thing does break, and it is the same thing the roadmap has always reserved: the values a given seed produces are different. Any test asserting on specific generated values needs re-baselining; a test asserting on their properties -- bounds, distinctness, null share, distribution shape -- does not. polspec's own suite needed no changes, which is the shape of test this library is built to support.

Generation got faster, by between a tenth and a third depending on the column. Measured A/B against v0.3.0 on one machine, same build profile, twenty million rows: the four-column frame in benchmarks/bench.py 1.30x, a unique=True Int64 column 1.21x, a bounded nullable Int64 column 1.17x, an Enum column 1.25x, a String column unchanged. Treat the ratios rather than the absolute numbers as the claim.

Added

  • generate, generate_batches, inspect, validate, sink_parquet, sink_ipc, sink_csv and sink_ndjson are exported from polspec itself. Each takes a TableSpec as its first argument and each is what the matching FrameSpec classmethod already called -- but they lived in polspec.generation / polspec.validation, which the API reference calls internal and free to change in a patch release. So the TableSpec-first half of the library had no stable import path; now it does, and both halves appear in the API reference.

Changed

  • Breaking: the values a given seed produces have changed. polspec now builds on rand 0.10 (from 0.8), whose samplers draw differently. Same seed, same version, same frame -- as before; across this version boundary, not.
  • A generated numeric, boolean, temporal or categorical column arrives as one chunk rather than one per 65,536 rows. The values buffer and the validity bitmap are each allocated once at full length and filled in parallel through disjoint slices, instead of being built per chunk and appended together. Nothing downstream now pays for a column split into hundreds of pieces -- the gather behind a choices domain, the cast behind a temporal dtype, every sink_* write.

String columns are the exception and stay chunked, because Polars backs them with view arrays: merging those copies no string bytes, but it does copy sixteen bytes of view per row, which costs more than the split it removes. They still gain the other half of the change -- the chunks are collected in one go rather than appended one at a time, and an append rescanned both sides for their first and last non-null value to maintain a sorted flag that random strings will not have set anyway. - Drawing a unique=True column no longer materialises its domain. A domain only a little wider than the row count used to be built in full and partially shuffled, which allocates in proportion to the domain rather than to the output: ten million distinct values from a range of eighty million reserved 1.4 GB before writing anything. That branch is now Floyd's algorithm, which holds only the values it has chosen -- the same case now peaks at 491 MB. Roomier domains keep drawing and rejecting, which is faster there and was never the memory problem. - Every character of the generated-string alphabet is now exactly equally likely. Six random bits give 64 values for a 62-character alphabet, and the two spare ones fell back on % 62 over a fresh 32-bit draw, which is biased by about one part in 70 million -- far too little to see, but free to remove: the fallback now rejects properly instead. - rand 0.8 was compiled alongside the rand 0.10 that Polars already links, so the extension carried two copies of rand, rand_core and their chacha backends. There is now one of each. - The release profile builds the crate as a single codegen unit. Measured on one machine against otherwise identical v0.3.0 code, that alone is worth 2.1x on the unique=True path, for about twenty seconds of build time. Fat LTO on top of it was tried and dropped: a further 5% for eight more minutes per build.

Documentation

  • A self-referencing ForeignKey guarantees that every parent value exists, and nothing more -- in particular not that the result is a tree. Parents are sampled from the whole frame, so some rows end up in a cycle or pointing at themselves, which is what makes a recursive query fail to terminate, and validate() does not report it because no part of a spec can say "acyclic". Known limitations now says so, with a recipe for a genuine hierarchy, and the roadmap carries what closing the gap would need. Two tests pin the behaviour.

Fixed

  • Registry.validate_all applies each report's structural transformations using the same bound spec the report was produced against, rather than the unbound copy.

Internal

  • ColumnPlan::build takes a PlanArgs struct instead of thirteen positional arguments, so a call site names what it sets and leaves the rest to Default. Four #[allow(clippy::too_many_arguments)] and a great many Nones went with it.
  • Kind's three parallel lists -- the names, the parse, the reverse lookup -- are generated from one declaration, so a new column kind cannot be added to two of them and forgotten in the third.
  • The fixed-width integer ranges were written three times: once as polspec's default generation range, once as the limits a declared bound may not exceed, and once as the Rust samplers' defaults. The first is now read from the second.
  • The four sink_* functions share a typed batch-stream helper rather than forwarding **kwargs.
  • benchmarks/bench_generate.py is replaced by benchmarks/bench.py, which measures the same comparison and adds a regression mode. The old harness timed one run per case, in a process shared with the implementations it was comparing against, and recorded nothing about the machine -- so its numbers varied by around 25% between runs and could not be compared across days. It also measured exactly one column shape, which is how a change that made the unique path half as fast came within an afternoon of being released as a speed-up. The new one takes the fastest of several runs, gives every measurement its own process, repeats a short case until its floor settles, records the CPU, thread count, Polars version and cargo profile, and covers each column kind, both branches of the unique draw, the cartesian, rule, foreign-key and composite-key passes, and a sink. record writes a local baseline and check exits non-zero when a case regresses past a tolerance; repeated measurements now agree to within about 2%.
  • The benchmark table in the comparison guide is re-measured. It had been recorded on 2026-09-03, which is before both 0.2.0 and 0.3.0, so it had been describing an engine two releases old: the four-column frame at twenty million rows was published as 0.0827s, measures 0.1409s on v0.3.0, and 0.1127s here. Some of that gap is still unaccounted for and is worth chasing. The NumPy and pure-Python columns re-measure to within 1% of what was published, which is what says the difference is polspec's and not the machine's.

0.3.0 - 2026-09-06

The refactor 0.2.0 started, finished. CatSpec was the one declarative surface left doing everything in one class; it is now a value with a metaclass facade, like TableSpec and FrameSpec. Behind it came the fixes that were waiting for a release allowed to break something.

One thing breaks, and it is worth reading before you upgrade: naming a CatSpec entry now always gives back the dtype, whichever form declared the registry. pl.Enum(cats.STATUS) becomes cats.STATUS, and cats.CURRENCY.physical() becomes cats.get_categorical("CURRENCY").physical(). Nothing else in the public API changed shape.

The documentation gained a test: every Python example in docs/ is executed by the suite, which found five broken examples that had been shipping.

Changed

  • Breaking. CatSpec is a value, and the class body that declares one is read by a metaclass rather than left in the namespace -- the same split TableSpec and FrameSpec already had. What follows:

  • Naming an entry always gives back the dtype. It used to depend on how the registry was built: a class-body entry was a real class attribute and returned the dtype, while a dict-built registry's .STATUS returned the raw category list, and only one of the two could be handed to a ColSpec. Both now return the dtype, so ColSpec(cats.STATUS) and ColSpec(Categories.STATUS) mean the same thing. cats["STATUS"] and cats.get("STATUS") follow the same rule. Replace pl.Enum(cats.STATUS) with cats.STATUS, and cats.CURRENCY.physical() with cats.get_categorical("CURRENCY").physical().

  • An entry may share a name with a method. Entries are removed from the class body before the class exists, so declaring one called get no longer warns and no longer costs you CatSpec.get. The entry is reached through the registry (Categories.spec.get("get")).
  • CatSpec.infer_from_dataframe and CatSpec.infer_from_framespec are removed; CatSpec.infer(target, ...) dispatches on what it is given, as it already did. from_dataframe and from_framespec are unchanged -- those read what is declared rather than inferring what could be.
  • Categories.spec is the CatSpec a class body declares. Anywhere a registry is expected -- with_catspec, Registry(categories=...), FrameSpec.from_yaml(categories=...) -- the class and the value are now interchangeable.
  • CatSpec has value semantics. Two registries that say the same thing compare equal and hash equal, so one loaded from a file can be checked against one a class body declares.
  • CatSpec.dtype_of(name) is the one lookup everything else is built on: the dtype an entry names, or None. resolve_key still says which kind of entry a name binds to.
  • enums, categoricals and choices are read-only mappings rather than fresh dicts. dict(cats.enums) if you need a mutable copy.

Added

  • polspec.MultiValidationError, raised by Registry.validate_all when several frames fail at once. It is a ValidationError, so an existing except clause still catches it, and it carries every failing spec's ValidationReport as reports, keyed by spec name -- previously validate_all raised with a joined string and the reports were lost, so failing_rows(), by_code() and to_json() were unreachable from the registry path.
  • polspec.CliError is exported, so except polspec.CliError works. It was the one exception in the hierarchy reachable only from polspec.errors.
  • Every Python example in the documentation is executed by tests/test_doc_examples.py. A block that cannot run standalone says so in an HTML comment (<!-- docs: skip -->), and one that demonstrates an error is checked to still raise (<!-- docs: raises -->). This found four broken examples, fixed here: a drop() of a column the page never declared, a TableSpec example rebinding the name a later block used, and two blocks naming frames (broken_df, existing_df) that were never built.

Fixed

  • ColSpec(tags={...}) is reproducible. A set was kept in its own iteration order, which Python salts per process, so to_yaml wrote a different tags: line on every run and two identically-written specs compared unequal across processes. A set is now sorted; a list or tuple keeps the order it was written in.
  • Registry.generate_all, generate_related and inspect_all bind their cross-spec foreign keys before doing anything, so a key whose dtypes do not match is a RegistryError naming both columns rather than a Polars cast error from inside generation. Only resolve() used to run that check, and nothing said it had to be called first. A key whose target is supplied through references= rather than held by the registry is still accepted, as it was.
  • A ColSpec carrying the same validator twice keeps it once, so it produces one finding rather than two identical ones. TableSpec already collapsed identical checks and foreign keys.
  • generate_batches and the sink_* functions resolve references once per call rather than once per batch. A LazyFrame parent was collected inside every batch, so a scan-backed parent was re-read as many times as there were batches.
  • inspect() no longer raises a raw Polars error for a column whose dtype is wrong and whose spec declares choices or an Enum. The domain check was built before the dtype check could bail out, and comparing values against choices of another type is not something Polars will compile at all, so the frame most likely to arrive -- a column read back from CSV or JSON as the wrong type -- crashed instead of reporting a dtype finding.
  • A ColRule whose condition is null on a row no longer excuses every later rule on that row. Generation folds a null when to False before testing it and before accumulating it into the claimed mask; validation did neither, so the null propagated through Kleene logic and left rows that generation had rewritten unchecked.
  • TableSpec is hashable, so references={Orders.spec: df} works. It is one of the three forms generate() and validate() document, and the only one that could not be put in a dict: the dataclass's generated __hash__ cannot hash a mapping of columns, nor a ColSpec carrying distribution_params.

Documentation

  • The install sections of the README and the documentation home said polspec was not published to PyPI, directly below a pip install polspec block. Both now describe the published wheels, and point at CONTRIBUTING.md for building from a checkout.
  • The documentation workflow runs on changes to python/** and scripts/generate_llms_txt.py. The API reference is ::: directives filled in by mkdocstrings from the live docstrings, so a docstring that breaks --strict used to pass its own pull request and fail the next one to touch docs/.
  • The roadmap's "YAML format may change" section described the missing format version key that 0.2.0 shipped, and said an unsupported dtype raises TypeError rather than SpecError.
  • FINDING_COLUMN and ValidationOptions are documented in the validation guide; both are exported and appeared nowhere.
  • how-to/tablespec.md taught polspec.generation.generate(spec, ...) while the API reference says anything unlisted may change in a patch. The page now says which of the two it is.
  • CONTRIBUTING.md gives the runnable form of the Windows cargo test workaround, and names the STATUS_DLL_NOT_FOUND failure it fixes.

0.2.0 - 2026-09-05

The architecture release. Specs became data, constraints gained one definition each, and the generator learned to satisfy claims it used to only check.

This release breaks a lot. Every incompatible change below is marked Breaking and says what to do instead. Three are worth knowing before you upgrade: Spec._columns and friends are now Spec.spec.columns; the values a given seed produces have changed, so any test asserting on generated values needs re-baselining; and several declarations that used to be accepted and quietly misbehave are now refused at declaration time.

Added

  • docs/llms.txt and docs/llms-full.txt, published at the documentation site root in the llms.txt format: an index of every page, and the full text of all of them in one file. Generated by scripts/generate_llms_txt.py from the nav, the pages, and -- for the API reference, whose source is ::: directives -- the live docstrings, so a language model reads signatures rather than an empty page. A test fails if either file is stale.
  • polspec.constraints: the definitions generation and validation both read, so they cannot drift. Domain is what a column may hold (its choices, an Enum's categories, its bounds); Pass and order decide which rewrite of a generated frame runs first, from the columns each one reads and writes.
  • unique=True is generated, not just validated. The engine draws the column without replacement (src/unique.rs): it shuffles a materialised domain when the domain is barely larger than the frame, and rejects against a set when it is roomy. Every dtype is covered, nulls are exempt (a nullable unique column may repeat nulls and nothing else), and a domain too small to cover the row count is refused by name instead of quietly producing duplicates. method="cartesian" holds unique columns out of the coverage product and draws them once over the finished frame.
  • __unique_together__ is generated. A pass resamples the rows repeating a combination an earlier row already used, so only the repeats move and the rest keep the values their own columns' weights and bounds gave them. Rows with a null member are exempt, matching validation. A group whose columns cannot take enough distinct combinations is refused, naming the group; a foreign-keyed member is never resampled, since that would break its key.

  • The Rust boundary is typed. Python builds one ColumnPlan per column (a #[pyclass] validated at construction, with errors naming the column) instead of a positional tuple. Bounds cross as an i64, u64 or f64, so Int64/UInt64 bounds beyond 2^53 are exact and the generation clamp for an unbounded distribution reaches the dtype's true limits. Columns with a finite domain (choices, Enum) receive indices back and the typed values are gathered on the Python side, so a datetime, bytes or True choice never passes through a string; choices need only be distinct in the column's dtype, not as strings. python/polspec/_polspec.pyi is a stub for the extension; src/ is split into plan.rs, dist.rs and sample.rs with unit tests under cargo test; a test compares the distribution parameter tables on both sides.

  • import polspec works without the Rust extension: validation, spec files, the registry and the report renderers need no build. Only generation imports it, and raises one actionable ImportError when it is missing.
  • Registry: a declared set of specs. Registry(Customers, Orders, ...) resolves foreign keys declared against names (resolve(), running the checks a class-bound key gets at declaration), orders parents before children (order()), generates the whole set with every key satisfied (generate_all, with a per-spec seed so adding a table changes no other; generate_related for one spec and its ancestors), validates it in one call (inspect_all, validate_all), merges or checks shared categories (catspec(), categories=), writes and reads one file for the set (to_yaml/from_yaml, a specs: mapping plus categories:), collects specs from modules and directories (from_module, discover), and draws one entity-relationship diagram (to_mermaid). See the new Multiple specs guide.
  • inspect(): validation results as data. FrameSpec.inspect(df) (and polspec.validation.inspect(spec, df)) returns a ValidationReport of Finding records -- each with a code, a stable key, the columns involved, a count, samples and code-specific details -- and never raises for a bad frame. report.rows(finding) and report.failing_rows() return the offending rows lazily; by_column(), by_code() and to_json() slice and export them. Checks and validators now carry samples too.
  • ValidationError.report carries the same ValidationReport; .errors is still the list of messages.
  • polspec validate SPEC DATA [--references NAME=PATH] [--json] on the command line, exiting 1 on findings, so a spec can gate a pipeline in CI.
  • A foreign key whose parent was not supplied is a foreign_key_unresolved finding rather than a ValueError, matching how generate() already treats it; a parent lacking the referenced columns is a foreign_key finding.
  • Spec files carry a version: (now 2). Files from version 1 are migrated on read; a file from a newer polspec is refused with a clear message. A key the reader does not know is an error naming the closest known key; from_yaml(..., strict=False) downgrades it to a warning.
  • Foreign keys to other specs are written to YAML and Python as the target's name and read back unresolved, instead of being dropped with a warning.
  • polspec.serialization is a package driven by one field registry (fields.py): YAML in both directions, generated Python, and the import datetime decision all derive from it, and a test asserts every dataclass field has an entry. to_dict/from_dict are public.
  • CatSpec files keep choices recorded for plain string columns.
  • polspec.col(), a small predicate language for rules, validators and checks: col("total") >= col("subtotal"), col("email").str.contains("@"), is_in, is_between, is_null, &/|/~, arithmetic, and string operations. A predicate evaluates like the Polars expression it stands for and, unlike one, is written to and read from YAML and generated Python. __checks__ and ColSpec.validators written with col() now round-trip through to_yaml/from_yaml and to_python. Raw pl.Expr is still accepted and still warns on export.
  • ColRule.when accepts a predicate, so a rule may depend on several columns. The one-column dict form is still accepted and converted.
  • TableSpec: the spec as an immutable value. A FrameSpec class body now builds one, reachable as Spec.spec, and every verb (generate, validate, to_yaml, to_markdown, ...) is a function over it in polspec.generation, polspec.validation, polspec.serialization and polspec.report. TableSpec offers with_columns, drop, select, rename, with_checks, with_foreign_keys, with_unique_together, with_name and with_catspec; FrameSpec.from_spec wraps one in a class. See the new Specs as values guide.
  • FrameSpec.col(name) reaches a column whatever it is called.
  • ForeignKey.references may be a spec's name, for keys whose target is not importable where the key is declared.
  • An exception hierarchy under PolspecError: SpecError for declarations that cannot mean anything, ValidationError for data that fails its spec, GenerationError when a spec cannot be turned into data (including every error raised inside the Rust engine), SerializationError for files that cannot be written or read, and RegistryError, reserved for the spec registry. All are exported from polspec; see the new Errors reference page.

Changed

  • Breaking. ColRule.when no longer accepts the one-column dict ({"column": "region", "equals": "UK"}). col() is the only spelling: write col("region") == "UK". A spec file written by an earlier version still loads -- its conditions are converted as the file migrates -- but a file declaring the current version must carry the predicate form. The error names the column and says what to write.
  • Breaking. The le and ge condition keys are gone; they were undocumented duplicates of lte and gte. (le/ge remain the canonical operator names in a predicate's data form, which is unrelated.)
  • Breaking. polspec.serialization no longer re-exports the names of the pre-package module layout: _YAML_DTYPES, _YAML_NAME_TO_DTYPE, _dtype_to_yaml, _dtype_from_yaml, _dtype_to_python, _colspec_to_yaml, _colspec_from_yaml and _colspec_to_python. Use the names in polspec.serialization.fields and polspec.serialization.dtypes.
  • Breaking. ColRule.when is evaluated against the frame as it stands when the rule runs, not against the freely generated values. Rules and foreign keys are applied in dependency order, so a rule keyed on a column that another rule or a foreign key rewrites now reads the rewritten values -- the ones validate() checks it against. Chained rules, chained foreign keys, and a rule keyed on a foreign-keyed column all round-trip; the values a given seed produces for such a spec change.
  • Breaking. Two columns whose rules each read what the other writes have no order that satisfies both, and are now refused at declaration with a SpecError naming them.
  • Breaking. A ForeignKey whose parent's declared domain does not fit inside its own column's is refused at declaration (or when a Registry resolves a key that names its target as a string). A key overwrites its column with the parent's values, so bounds=(1, 50) on a column referencing keys in 100..200 could only ever generate data that fails its own validation. A column declaring no bounds or choices still accepts anything.
  • Breaking. unique=True can no longer be combined with weights, a non-uniform distribution, or rules. The first two describe how often a value recurs, which a draw without replacement has no room for; a rule assigns from a fixed set, which is how duplicates would get back in. Each is refused at declaration rather than silently ignored.
  • Breaking. A column carrying rules may not also be part of a __unique_together__ group, for the same reason: the repair that separates repeated combinations would overwrite what the rule put there.
  • Breaking. A ForeignKey filling a unique=True column now refuses when the parent holds fewer distinct values than there are rows, instead of falling back to sampling with replacement and producing the duplicates the column forbids.
  • polspec test no longer emits validate_unique=False in generated tests. Uniqueness is generated now, so the generated test asserts it.
  • A foreign key spanning textual dtypes -- a String column referencing an Enum key, which declaration has always allowed and generation has always handled -- now validates instead of raising SchemaError from the anti-join. The parent's keys are cast to the local dtype for the join, so ValidationReport.rows() still returns the frame's own rows unchanged.

  • Breaking. Each column's generation seed is derived from the frame seed and the column's name, not its position, so inserting a column no longer reshuffles the columns after it. The values a given seed produces change from previous versions.

  • ColRule application samples only as many values as there are matched rows and scatters them into place, instead of filling the whole column per rule.
  • polspec.validation is a package (report.py, constraints.py); foreign key anti-joins are collected together with pl.collect_all instead of one collect per key.
  • Breaking. ColSpec.distribution and distribution_params are stored in canonical form ("exp" becomes "exponential", mu/sigma become mean/std, and so on), so spec files are canonical. Every alias is still accepted when declaring.
  • Breaking. An unrecognised physical dtype in a CatSpec entry is now a SerializationError instead of silently becoming UInt32.
  • Breaking. ColRule.when is a predicate after construction rather than a dict (rule.when.root_names() lists the columns it reads); rules in YAML are written in the predicate data form, and the old dict form is still read.
  • Breaking. A column may now share a name with a FrameSpec method: the metaclass takes ColSpec attributes out of the class namespace, so schema, tag and friends no longer shadow anything and no longer warn. The private _columns, _checks, _unique_together and _foreign_keys class attributes are gone; read Spec.spec.columns and friends instead.
  • Breaking. ForeignKey.references is the target's name after construction (the bound spec is available as ForeignKey.target), and references={...} on generate/validate accepts the class, the TableSpec or the name as key.
  • Breaking. Removed: the FrameSchema alias; FrameSpec.generate_catspec, write_catspec, infer_catspec and with_inferred_catspec (use catspec(), catspec().to_yaml(), CatSpec.infer(...) and with_catspec(CatSpec.infer(...))); the max_unique and bounds alias keyword arguments of from_dataframe (use max_unique_enum and calculate_bounds).
  • to_yaml and to_python share one set of warnings about what a file cannot hold.
  • Breaking, mildly. Errors that were bare ValueError or TypeError are now the subclass above. Each keeps the built-in type it replaced, so except ValueError still catches it; only code matching on the exact type (type(exc) is ValueError) sees a difference. Plain argument misuse (n < 0, an unknown method=) is unchanged.
  • The command line prints any PolspecError as a one-line error: ... instead of a TypeName: message line.

Fixed

  • Foreign-key sampling during generation drew parent keys from an unordered unique(), so the same seed could give different child rows between runs. The parent's distinct keys now keep their order and generation is reproducible.

0.1.5 - 2026-09-03

Added

  • python/polspec/py.typed, so type checkers use the package's annotations.
  • CONTRIBUTING.md, this changelog, and a .python-version file.
  • examples/related_specs.py: a worked example of four related specs (foreign keys, shared categories, rules, checks, a YAML-declared spec). It runs in CI as a smoke test.
  • A release-workflow job that refuses a vX.Y.Z tag whose version does not match pyproject.toml.
  • CI now runs ruff check with a wider rule set, ruff format --check, cargo fmt --check, cargo clippy -D warnings and cargo test, tests on macOS as well as Linux and Windows, and tests against the newest Polars release inside the declared bound. The docs build runs strictly on pull requests.
  • Release builds now produce wheels for Linux aarch64 and macOS (x86_64 and arm64) alongside Linux and Windows x86_64, plus an sdist, and only publish when the test workflow is green.
  • An optional .pre-commit-config.yaml with ruff and cargo fmt hooks.
  • tests/test_colspec.py (2,000 lines, unsectioned) is split into test_generation.py, test_rules.py, test_serialization.py, test_profiler.py, test_framespec.py, test_report.py and test_foreign_key.py, each with a docstring saying what it covers.

Changed

  • The crate version in Cargo.toml is a placeholder; pyproject.toml is the only place the version is set, so uv version --bump works.
  • The parquet, ipc and all extras (all identical) are replaced by a single arrow extra. Install with polspec[arrow] for the Parquet and Arrow IPC sinks.
  • polars is bounded to <2; the Rust extension is coupled to a Polars release line.
  • The abi3 floor is now Python 3.12, matching requires-python.
  • cargo test links again (extension-module is no longer an unconditional crate feature; maturin enables it).

Fixed

  • Repository URL in package metadata pointed at the repository's old name.
  • README claimed the license was unspecified; it is MIT.
  • Documentation: ColSpec(col_name=...) is now described in Declaring columns, FrameSpec.to_python() in YAML specs, the getting-started example imports date, and the architecture page lists the cli module and its tests.

0.1.4 - 2026-09-02

Added

  • polspec schema infer --output spec.py and FrameSpec.to_python(), which write a spec as an editable Python module rather than YAML.

0.1.3 - 2026-09-01

Added

  • ColSpec(col_name=...), so a column's name in data may differ from the attribute name used to declare it.

Changed

  • Roadmap expanded with detailed plans for a spec registry, structured validation results, and generation guardrails.

0.1.2 - 2026-08-31

Version bump only; no user-facing change.

0.1.1 - 2026-08-31

Added

  • Test workflow on GitHub Actions (Linux and Windows, Python 3.12 to 3.14).

Fixed

  • ColSpec.dtype accepts a dtype class as well as an instance.

0.1.0 - 2026-08-31

First tagged release.

  • ColSpec and FrameSpec: declare a Polars schema with nullability, bounds, string lengths, choices and weights, distributions, tags, and conditional ColRules.
  • generate() backed by a parallel Rust extension, method="cartesian" for coverage sets, batched generation and Parquet/CSV/IPC/NDJSON sinks.
  • validate() collecting every violation in one Polars aggregation, with column validators, multi-column Checks, composite uniqueness and ForeignKeys.
  • CatSpec registries for shared Enum/Categorical domains.
  • YAML round-trip, from_dataframe() profiling, Markdown and Mermaid output.
  • CLI: polspec schema infer, polspec schema new, polspec test.
  • Documentation site, comparison guide, and release automation.