Skip to content

Generated documentation

A spec already holds everything a data dictionary needs, so polspec renders one rather than asking you to keep a second copy in step.

Markdown data dictionary

The examples on this page document these specs:

from datetime import date

import polars as pl
from polspec import ColSpec, ForeignKey, FrameSpec, Registry


class Customers(FrameSpec):
    id        = ColSpec(pl.Int64, bounds=(1, 10_000), unique=True)
    name      = ColSpec(pl.String, string_length=(3, 20), tags="pii")
    country   = ColSpec(pl.Enum(["UK", "US", "DE"]))
    signed_up = ColSpec(pl.Date, bounds=(date(2020, 1, 1), date(2026, 1, 1)))


class Orders(FrameSpec):
    order_id      = ColSpec(pl.Int64, bounds=(1, 1_000_000), unique=True)
    customer_id   = ColSpec(pl.Int64, bounds=(1, 10_000))
    status        = ColSpec(pl.Enum(["NEW", "PAID", "SHIPPED"]))
    total         = ColSpec(pl.Float64, bounds=(0.0, 5_000.0))
    internal_note = ColSpec(pl.String, string_length=(0, 40), nullable=True)
    __foreign_keys__ = [ForeignKey("customer_id", references=Customers, ref_columns="id")]


class OrderLines(FrameSpec):
    order_id = ColSpec(pl.Int64, bounds=(1, 1_000_000))
    line_no  = ColSpec(pl.Int32, bounds=(1, 50))
    quantity = ColSpec(pl.UInt16, bounds=(1, 500))
    __unique_together__ = [["order_id", "line_no"]]
    __foreign_keys__ = [ForeignKey("order_id", references=Orders)]
Orders.to_markdown("docs/orders.md")   # writes and returns
markdown = Orders.to_markdown()        # just returns

The document has three parts: an overview, a table of every column, and — when the spec declares any — a constraints section covering composite keys, checks, foreign keys, conditional rules and column validators:

# Orders

## Overview
- **Schema:** `Orders`
- **Total Columns:** 5
- **Foreign Keys:** 1 key(s)

## Columns

| Column | Type | Nullable | Bounds | Domain / Choices | String Length | Tags | Rules | Unique |
|:---|:---|:---|:---|:---|:---|:---|:---|:---|
| `order_id` | `Int64` | No | [1, 1000000] | - | - | - | - | Yes |
| `customer_id` | `Int64` | No | [1, 10000] | - | - | - | - | No |
| `status` | `Enum` | No | - | `NEW`, `PAID`, `SHIPPED` | - | - | - | No |
| `total` | `Float64` | No | [0.0, 5000.0] | - | - | - | - | No |
| `internal_note` | `String` | Yes (10%) | - | - | [0, 40] | - | - | No |

## Constraints & Invariants

### Foreign Keys
- **`fk_customer_id__Customers`**: `['customer_id']` -> `Customers.['id']`

A column's choices, or an Enum's categories, appear in its Domain / Choices cell, each beside its share when the column is weighted. An open-ended bound reads as >= 0.0 rather than [0.0, None]. The Nullable cell shows how often a nullable column is null, and also notes a column's default and whether it is optional. Pass title= to override the heading, which otherwise uses the class name.

Long lists of choices

A cell lists four choices whole; a longer list shows its first three and its count, so a wide table stays readable. choices= decides what happens to the rest, added in 1.6.1:

class Bonds(FrameSpec):
    rating = ColSpec(pl.Enum, choices={"AAA": 1, "AA": 2, "A": 4, "BBB": 5, "BB": 2, "B": 1})


Bonds.to_markdown()                     # "summary": the first three and the count
Bonds.to_markdown(choices="appendix")   # the same, then every value in a Choices section
Bonds.to_markdown(choices="full")       # every choice in its cell

The table keeps its summary cell, and the appendix lists every value and share of each column it cut short, in declared order, in one table at the end of the document:

| `rating` | `Enum` | No | - | `AAA` 6.67%, `AA` 13.3%, `A` 26.7%, ... (6 total) | - | - | - | No |

## Choices

| Column | Value | Share |
|:---|:---|---:|
| `rating` | `AAA` | 6.67% |
|  | `AA` | 13.3% |
|  | `A` | 26.7% |
|  | `BBB` | 33.3% |
|  | `BB` | 13.3% |
|  | `B` | 6.67% |

A column whose choices fit its cell is already complete, so it is left out; an unweighted column's share is -, since its choices are what it allows, not how often. For a spec of many such columns, "appendix" keeps the table narrow; "full" suits a small one.

Descriptions

To say what a table or a column means, give it a description. The table's appears under the title, and the columns' in a Description column, which the table has only when something fills it. Nothing else reads one: it changes no generated value and no validation.

class Described(FrameSpec):
    """One row per order, as placed."""

    order_id = ColSpec(pl.Int64, unique=True, description="The order's number.")

A class's docstring is its description, added in 1.2; __description__ says it instead, and wins when a class has both — for a docstring written for whoever maintains the class rather than whoever reads the data. A subclass inherits neither, since it describes another table; its inherited columns keep their own. A TableSpec takes description=, and with_description() makes a copy.

Entity-relationship diagram

Orders.to_mermaid("docs/orders.mmd")
erDiagram
    Orders {
        Int64 order_id PK "bounds: [1, 1000000]"
        Int64 customer_id FK "bounds: [1, 10000]"
        Enum status
        Float64 total "bounds: [0.0, 5000.0]"
        String internal_note "nullable, len: [0, 40]"
    }
    Customers ||--o{ Orders : "fk_customer_id__Customers"

Columns are annotated with what the spec declares — nullability, bounds or choices, tags, string lengths — and keyed as PK (a unique column), UK (a member of a composite key) or FK.

Mermaid renders in GitHub, GitLab and most documentation sites, including this one, so the diagram stays live rather than becoming a stale screenshot.

A lone unique=True column is the entity's PK; when several columns are unique each is marked UK, as is every member of a __unique_together__ group, and a foreign-keyed column FK.

Mermaid reads a name as one word of ASCII letters, digits, _ and -, so a name it cannot read is drawn as the nearest one it can, with its real name first in the note: a column unit price is drawn unit_price "name: 'unit price'", and an entity 2024 Orders as _2024_Orders. Two columns that would draw alike are told apart by a suffix. The Markdown table escapes a | or a newline in a name or value, so an awkward name never breaks a row.

Several specs in one diagram

A single spec's diagram can only name the entity a key points at. A Registry draws every spec and every key between them:

Registry(Customers, Orders, OrderLines).to_mermaid("docs/schema.mmd")

Documenting a category registry

CatSpec renders the same two ways:

from polspec import CatSpec

categories = CatSpec(
    enums={"STATUS": ["NEW", "PAID", "SHIPPED"]},
    categoricals={"CURRENCY": pl.Categories("CURRENCY", physical=pl.UInt8)},
)

categories.to_markdown("docs/categories.md")
categories.to_mermaid("docs/categories.mmd")

The Markdown lists enums with their variants and categoricals with their physical dtype, namespace and domain pool. The Mermaid output is a class diagram, with each enum as an <<enumeration>>.

Keeping generated docs current

Both renderers are pure functions of the spec, so wiring them into a build or a pre-commit hook keeps the documentation honest:

from pathlib import Path

for spec in (Customers, Orders, Shipments):
    spec.to_markdown(Path("docs/schemas") / f"{spec.__name__.lower()}.md")