Skip to content

Validation

inspect() returns a report; validate() raises one. Both carry the same findings as data -- see Validating data.

inspect

inspect 

inspect(spec: TableSpec, df: DataFrame | LazyFrame, *, options: ValidationOptions | None = None, references: References = None, **option_kwargs: Any) -> ValidationReport

Everything spec has to say about df, as a ValidationReport.

Never raises for a frame that fails: every violation is a Finding on the report, with report.rows(finding) and report.failing_rows() giving the offending rows back lazily. See validate for the options.

validate

validate 

validate(spec: TableSpec, df: DataFrame, **options: Any) -> DataFrame

validate(spec: TableSpec, df: LazyFrame, **options: Any) -> LazyFrame

validate(spec: TableSpec, df: DataFrame | LazyFrame, *, options: ValidationOptions | None = None, references: References = None, **option_kwargs: Any) -> DataFrame | LazyFrame

Validates a DataFrame or LazyFrame against spec.

Parameters:

Name Type Description Default
df DataFrame | LazyFrame

The frame to validate. A LazyFrame comes back as a LazyFrame.

 required
options ValidationOptions

Every option at once, as a value -- useful for passing one setting through several calls. Cannot be combined with the keywords below.

 None
references mapping

Parent frames for foreign keys that reference another spec, keyed by that spec, its FrameSpec class, or its name. A key with no entry is reported as a foreign_key_unresolved finding.

 None
**option_kwargs Any

The fields of ValidationOptions, one at a time, with the check switches spelled validate_rules, validate_validators, validate_unique, validate_checks, validate_foreign_keys, validate_hierarchy, validate_pattern and validate_bounds. See ValidationOptions for what each means and what it defaults to; an unknown name raises TypeError naming the closest match.

 {}

Returns:

Type Description
The validated, optionally transformed frame -- a LazyFrame if `df` was one.

Raises:

Type Description
ValidationError

Carrying a ValidationReport of every violation.
TypeError

For an option name this does not accept.
ValueError

For an accepted option given a value outside its choices.

ValidationOptions

ValidationOptions dataclass 

ValidationOptions(extra_cols: Literal['drop', 'allow', 'raise'] = 'raise', missing_cols: Literal['add', 'allow', 'raise'] = 'raise', strict_dtypes: bool = False, rules: bool = True, validators: bool = True, unique: bool = True, checks: bool = True, foreign_keys: bool = True, hierarchy: bool = True, pattern: bool = True, bounds: bool = True, cast: bool = False, streaming: bool = False)

Which checks to run and what to do about structural mismatches.

Grouped rather than passed as ten separate arguments, so adding a check does not widen every signature between here and FrameSpec.validate.

ValidationReport

ValidationReport dataclass 

ValidationReport(spec_name: str, findings: tuple[Finding, ...], frame: LazyFrame, options: ValidationOptions = _FILLED_IN)

Every finding for one frame against one spec.

raise_if_failed 

raise_if_failed() -> None

Raises ValidationError carrying this report, if anything was found.

by_column 

by_column() -> dict[str, tuple[Finding, ...]]

Findings grouped by column; structural findings under "".

by_code 

by_code(code: FindingCode) -> tuple[Finding, ...]

Every finding of one kind, such as "bounds" or "foreign_key".

rows 

rows(finding: Finding) -> LazyFrame

The rows violating one finding, lazily.

failing_rows 

failing_rows() -> LazyFrame

Every row that violates a row-level finding, lazily.

Adds a __polspec_finding column naming the finding's key, so a row violating several claims appears once per claim.

to_dict 

to_dict() -> dict[str, Any]

This report as JSON-ready data: the spec, the verdict, the findings.

to_json 

to_json(*, indent: int | None = 2) -> str

This report as a JSON string. indent=None for one line.

Finding

Finding dataclass 

Finding(code: FindingCode, key: str, message: str, columns: tuple[str, ...] = (), count: int | None = None, samples: tuple[Any, ...] = (), details: Mapping[str, Any] = dict(), _locate: Callable[[LazyFrame], LazyFrame] | None = None)

One violation of one claim the spec makes.

Attributes:

Name Type Description
code FindingCode

Which kind of claim was violated.
key str

A stable identifier for the claim within its spec, such as "total__bounds" or "check:total_covers_subtotal".
message str

The human-readable description of what was violated.
columns tuple[str, ...]

The columns involved; empty for structural findings.
count int | None

How many rows violate the claim; None for structural findings.
samples tuple

Up to five offending values (or structs of values, for multi-column claims).
details Mapping

Code-specific facts: the expected and actual dtype, the observed extremes, the foreign key's target.

row_level property 

row_level: bool

Whether this finding can name the rows that violate it.

rows 

rows(frame: LazyFrame) -> LazyFrame

The rows of frame that violate this claim, lazily.

to_dict 

to_dict() -> dict[str, Any]

This finding as JSON-ready data.