Drift¶
polspec.diff() compares two declarations; polspec.drift()
compares a declaration to a frame. Both return a DriftReport, and both are
reachable from a spec as Orders.diff(...) and Orders.drift(df) — see
Schema and data drift.
diff¶
diff
¶
diff(old: TableSpec | type, new: TableSpec | type, *, renames: Mapping[str, str] | None = None, options: DriftOptions | None = None, **keywords: Unpack[DriftKeywords]) -> DriftReport
What changed between two declarations, as a DriftReport.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
old
|
TableSpec | FrameSpec class
|
The two declarations. A spec's name is not compared: the report is about what the two say, not what they are called. |
required |
new
|
TableSpec | FrameSpec class
|
The two declarations. A spec's name is not compared: the report is about what the two say, not what they are called. |
required |
renames
|
mapping
|
Columns renamed between the two, |
None
|
options
|
DriftOptions
|
Only |
None
|
**keywords
|
Unpack[DriftKeywords]
|
The fields of |
{}
|
Notes
A finding is breaking when a frame that satisfied old could fail
new: a narrowed domain, a dropped nullability, a column added (a frame
that lacks it fails missing_cols="raise"), a constraint added. A
widened domain or a removed constraint is compatible.
Examples:
drift¶
drift
¶
drift(spec: TableSpec | type, df: Frame, *, options: DriftOptions | None = None, **keywords: Unpack[DriftKeywords]) -> DriftReport
How df has moved relative to what spec declares, as a DriftReport.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
spec
|
TableSpec | FrameSpec class
|
The declaration. |
required |
df
|
DataFrame | LazyFrame
|
The frame. A LazyFrame is collected: every measurement here is a summary of the whole column. |
required |
options
|
DriftOptions
|
Every option at once. Cannot be combined with the keywords. |
None
|
**keywords
|
Unpack[DriftKeywords]
|
The fields of |
{}
|
Notes
A finding is breaking when this frame fails this spec on that column: values outside the domain, a bound exceeded, nulls where none are allowed, a format not matched. A null rate that moved within a nullable column, or declared values the data never holds, is compatible -- the data still validates; the declaration has stopped describing it well.
Uniqueness, composite keys, foreign keys and checks are not measured
here; they are pass/fail claims that validate() already reports.
Examples:
>>> spec = TableSpec("Orders", {"total": ColSpec(pl.Float64, bounds=(0, 100))})
>>> report = drift(spec, pl.DataFrame({"total": [10.0, 250.0]}))
>>> [(finding.code, finding.severity) for finding in report.findings]
[('bounds_exceeded', 'breaking')]
>>> print(report)
Drift: 1 breaking, 0 compatible, DataFrame against 'Orders'
- [breaking] Column 'total': values escape bounds [0, 100]: max found 250.0 by 150.0 above. Widen the bounds, or fix the source
DriftOptions¶
DriftOptions
dataclass
¶
DriftOptions(*, significance: float = 0.001, null_rate_tolerance: float = 0.05, frequency_tolerance: float = 0.05, distribution_tolerance: float = 0.05, unseen_values: bool = True, strict_dtypes: bool = False, max_samples: int = 10)
What counts as drift, said once.
A rate, a set of frequencies or a distribution is reported as moved only
when the move is both significant -- unlikely to be sampling noise at
the number of rows measured -- and at least as large as its tolerance.
Fifty rows cannot trip a finding on noise, and ten million cannot trip
one on a difference too small to matter. Each column's tests are run at
significance, uncorrected for how many columns there are: a frame of a
hundred columns, each tested once at 0.001, expects a false alarm about
one run in ten.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
significance
|
float
|
The false-alarm rate of each statistical test: how unlikely a move must be, were the declaration exactly right, before it is reported. |
0.001
|
null_rate_tolerance
|
float
|
The smallest move of a null rate -- or an element null rate, or a
NaN share -- worth reporting ( |
0.05
|
frequency_tolerance
|
float
|
The smallest move of a weighted column's frequencies worth
reporting ( |
0.05
|
distribution_tolerance
|
float
|
The smallest move of a declared distribution worth reporting
( |
0.05
|
unseen_values
|
bool
|
Whether to report declared |
True
|
strict_dtypes
|
bool
|
The same switch as |
False
|
max_samples
|
int
|
How many offending values a finding's |
10
|
Examples:
>>> spec = TableSpec("Orders", {"note": ColSpec(pl.String, nullable=True, null_probability=0.1)})
>>> frame = pl.DataFrame({"note": ["a", None, None, None] * 250}) # 75% null, not 10%
>>> [f.code for f in drift(spec, frame).findings]
['null_rate_moved']
>>> lenient = DriftOptions(null_rate_tolerance=0.7) # a move of up to 70 points is fine
>>> [f.code for f in drift(spec, frame, options=lenient).findings]
[]
DriftReport¶
DriftReport
dataclass
¶
DriftReport(kind: Kind, old: str, new: str, findings: tuple[DriftFinding, ...], options: DriftOptions | None = None)
Every difference between two specs, or between a spec and a frame.
kind says which: "diff" compares old to new, both declarations;
"drift" compares the declaration old to data, and new is what the
data was called. Like any collection it is falsy when empty, so
if report: means "something changed"; report.unchanged says the
opposite in words.
Examples:
>>> old = TableSpec("Orders", {"total": ColSpec(pl.Float64, bounds=(0, 100))})
>>> new = TableSpec("Orders", {"total": ColSpec(pl.Float64, bounds=(0, 200))})
>>> report = diff(old, new)
>>> report.breaking, [f.code for f in report.compatible]
((), ['domain_widened'])
by_column
¶
by_column() -> dict[str, tuple[DriftFinding, ...]]
Findings grouped by column; table-level findings under "".
by_code
¶
by_code(code: DriftCode) -> tuple[DriftFinding, ...]
Every finding of one kind, such as "domain_narrowed".
to_json
¶
This report as a JSON string. indent=None for one line.
to_markdown
¶
This report as Markdown, written to path if given.
DriftFinding¶
DriftFinding
dataclass
¶
DriftFinding(*, code: DriftCode, severity: Severity, key: str, message: str, columns: tuple[str, ...] = (), details: Mapping[str, Any] = dict())
One difference between two declarations, or a declaration and data.
Attributes:
| Name | Type | Description |
|---|---|---|
code |
DriftCode
|
Which kind of difference. |
severity |
'breaking' | 'compatible'
|
Breaking when data that satisfied the old side could fail the new one; compatible otherwise. |
key |
str
|
A stable identifier within the report, such as |
message |
str
|
What changed, naming the column, and what to do about it. |
columns |
tuple[str, ...]
|
The columns involved; empty for table-level differences. |
details |
Mapping
|
Code-specific facts: the old and new value, how far a bound was exceeded, which values were new. |
Examples:
>>> old = TableSpec("Orders", {"total": ColSpec(pl.Float64, nullable=True)})
>>> new = TableSpec("Orders", {"total": ColSpec(pl.Float64)})
>>> finding = diff(old, new).findings[0]
>>> finding.code, finding.severity, finding.columns
('nullability_changed', 'breaking', ('total',))
DriftCode¶
DriftCode
module-attribute
¶
DriftCode = Literal['column_added', 'column_removed', 'column_renamed', 'dtype_changed', 'nullability_changed', 'domain_widened', 'domain_narrowed', 'domain_changed', 'bounds_exceeded', 'new_values', 'format_violated', 'null_rate_moved', 'nan_rate_moved', 'frequencies_moved', 'distribution_moved', 'cardinality_moved', 'constraint_added', 'constraint_removed', 'field_changed']
The code of a DriftFinding: what moved. The full list is on the Errors
and findings page.
Severity¶
Severity
module-attribute
¶
A DriftFinding's severity: breaking when data that satisfied the old
side could fail the new one, compatible otherwise.