Errors¶
Everything polspec raises on its own behalf derives from one base class,
PolspecError, so a caller can separate "polspec objected" from "something
else went wrong" with a single except. The example on this page uses this
spec:
from datetime import date
import polars as pl
from polspec import ColSpec, ForeignKey, FrameSpec
class Orders(FrameSpec):
order_id = ColSpec(pl.Int64, bounds=(1, 1_000_000), unique=True)
customer_id = ColSpec(pl.Int64, bounds=(1, 10_000))
status = ColSpec(pl.Enum(["NEW", "PAID", "SHIPPED"]))
total = ColSpec(pl.Float64, bounds=(0.0, 5_000.0))
internal_note = ColSpec(pl.String, string_length=(0, 40), nullable=True)
df = Orders.generate(500, seed=2)
One clause catches anything polspec raises:
from polspec import PolspecError
try:
Orders.validate(df)
except PolspecError as exc:
log.warning("rejected: %s", exc)
| Exception | Raised when | Also a |
|---|---|---|
PolspecError |
Base class; never raised directly | Exception |
SpecError |
A declaration cannot mean anything: bounds a dtype cannot hold, a rule naming a column that does not exist, two attributes resolving to one column name | ValueError, TypeError |
ValidationError |
Data does not meet its spec. err.report is the ValidationReport; err.errors lists its messages |
ValueError |
GenerationError |
A spec that declares fine cannot be turned into data as asked: no column for method="cartesian" to cover, a coverage set past the size cap, a foreign key with an empty parent. Errors from the Rust engine surface as this |
ValueError |
SerializationError |
A spec file cannot be written or read: a dtype with no file representation, an unrecognised dtype name, a category reference the registry does not hold | ValueError |
RegistryError |
A Registry is inconsistent: an unknown or duplicated spec name, a key whose target is not in it, a cycle, two specs disagreeing about a shared category |
LookupError |
Each subclass keeps the built-in type it replaced, so except ValueError
written against an earlier version still catches it.
Ordinary argument misuse is not a PolspecError. A negative row count, an
unknown method=, a batch_size of zero, or the wrong object passed where a
DataFrame was expected raise the plain ValueError or TypeError any Python
API would.
The command line prints a PolspecError as a one-line error: ... and exits
with status 1. Any other exception is printed the same way, with its class
name (error: KeyError: ...), and also exits 1; it is a bug worth
reporting. A command line that is itself wrong exits 2.
Finding codes¶
Every violation inspect() reports, and validate() raises, is a Finding
with one of these codes — polspec.FindingCode, for a type checker. Row-level findings can return the offending rows
through report.rows(finding); structural ones describe the frame's shape.
| Code | Kind | Raised when |
|---|---|---|
extra_columns |
structural | the frame has columns the spec does not declare (extra_cols="raise") |
missing_columns |
structural | the frame lacks declared columns (missing_cols="raise") |
dtype |
structural | a column's dtype is not compatible with its declaration |
dtype |
row-level | a compatible, wider dtype holds a value the declared one cannot — Int8 declared, an Int64 of 1000; keyed <column>__dtype_range |
foreign_key_unresolved |
structural | a key references another spec and references= had no entry for it |
nullability |
row-level | a non-nullable column holds nulls |
nan |
row-level | a float value holds NaN while nan_probability is 0 |
choices |
row-level | a value is outside choices or the Enum categories |
bounds |
row-level | a value is outside bounds; details carry the extremes found |
string_length |
row-level | a string or binary value's length is outside string_length |
list_length |
row-level | a list holds a number of elements outside list_length |
format |
row-level | a string value does not have the declared format; details name the format |
pattern |
row-level | a string value does not match the declared pattern; details carry it |
rule |
row-level | a row matched a ColRule but holds a value outside its choices |
derived |
row-level | a derived column's value is not its derive expression's (a float, within a relative 1e-9); added in 1.4 |
validator |
row-level | a ColSpec.validators predicate is false |
unique |
row-level | a unique=True column holds duplicates |
unique_together |
row-level | a composite key holds duplicate combinations |
check |
row-level | a __checks__ predicate is false |
order |
row-level | a row belongs before the row above it, by the spec's order_by; details name the order |
row_count |
structural | the frame holds a number of rows outside the spec's rows; details carry the count and both ends |
foreign_key |
row-level | a key value has no matching parent row (also structural when the parent lacks the referenced columns) |
hierarchy_multi_parent |
row-level | a reference appears as a child in more than one row, so it has no single ultimate parent; see Hierarchies |
hierarchy_cycle |
row-level | a row's chain of parents never reaches an ultimate parent |
hierarchy_depth |
row-level | a chain is longer than the declared max_depth, without being a loop |
Drift codes¶
A DriftReport (from diff() or drift()) carries DriftFindings with one
of these codes — polspec.DriftCode, for a type checker. Each also carries a severity: breaking when a frame
that satisfied the old side could fail the new one, compatible
otherwise. See Schema and data drift.
| Code | From | Severity | Raised when |
|---|---|---|---|
column_added |
both | breaking; compatible for an optional=True column |
the new spec declares a column the old lacks, or the data has an undeclared column |
column_removed |
both | breaking | the old spec declares a column the new lacks, or the data lacks a declared column |
column_renamed |
diff |
compatible | a rename given through renames= |
dtype_changed |
both | as validation decides | the dtypes differ; compatible when the old values would still validate. Against data, only a dtype that fails validation is reported — or any difference, under strict_dtypes |
nullability_changed |
both | off → on compatible; on → off breaking; nulls in a non-nullable column breaking | nullable, or a list's element_null_probability between 0 and not |
domain_widened |
diff |
compatible | bounds, choices, format, string length or a table's rows accept more than before — or a float value now allows NaN |
domain_narrowed |
diff |
breaking | they accept less |
domain_changed |
diff |
breaking | neither is inside the other |
bounds_exceeded |
drift |
breaking | values escape bounds, string_length or list_length; details say which and by how much |
new_values |
drift |
breaking | values outside choices, an Enum or a finite format — or a NaN in a float value that declares none |
format_violated |
drift |
breaking | values that do not match the declared format |
null_rate_moved |
drift |
compatible | the null rate has moved from null_probability — or the element null rate from element_null_probability — by at least the tolerance, and significantly over the rows seen |
nan_rate_moved |
drift |
compatible | the share of present values that are NaN has moved from nan_probability by at least the tolerance, and significantly |
frequencies_moved |
drift |
compatible | a weighted column's frequencies have moved from its weights by at least frequency_tolerance, and significantly |
distribution_moved |
drift |
compatible | the values have moved from a sample drawn from the declared distribution by at least distribution_tolerance, and significantly |
cardinality_moved |
drift |
compatible | declared choices, Enum categories or extra values the data never holds, and at their declared share would not miss by chance — never a finite format's own codes |
constraint_added |
diff |
breaking | unique, a validator, rule, check, composite key, foreign key, hierarchy or rows present only in the new spec, or a column no longer optional |
constraint_removed |
diff |
compatible | the reverse, including a column made optional |
field_changed |
diff |
compatible | tags, description, default, weights, distribution, distribution_params, null_probability, pattern or seed_name differ, or a table's description |
A struct's fields are compared by every row above, as columns are, and a
finding about one is keyed by its path — point.lat__domain — with
columns naming the struct column.