Generated documentation¶
A spec already holds everything a data dictionary needs, so polspec renders one rather than asking you to keep a second copy in step.
Markdown data dictionary¶
The examples on this page document these specs:
from datetime import date
import polars as pl
from polspec import ColSpec, ForeignKey, FrameSpec, Registry
class Customers(FrameSpec):
id = ColSpec(pl.Int64, bounds=(1, 10_000), unique=True)
name = ColSpec(pl.String, string_length=(3, 20), tags="pii")
country = ColSpec(pl.Enum(["UK", "US", "DE"]))
signed_up = ColSpec(pl.Date, bounds=(date(2020, 1, 1), date(2026, 1, 1)))
class Orders(FrameSpec):
order_id = ColSpec(pl.Int64, bounds=(1, 1_000_000), unique=True)
customer_id = ColSpec(pl.Int64, bounds=(1, 10_000))
status = ColSpec(pl.Enum(["NEW", "PAID", "SHIPPED"]))
total = ColSpec(pl.Float64, bounds=(0.0, 5_000.0))
internal_note = ColSpec(pl.String, string_length=(0, 40), nullable=True)
__foreign_keys__ = [ForeignKey("customer_id", references=Customers, ref_columns="id")]
class OrderLines(FrameSpec):
order_id = ColSpec(pl.Int64, bounds=(1, 1_000_000))
line_no = ColSpec(pl.Int32, bounds=(1, 50))
quantity = ColSpec(pl.UInt16, bounds=(1, 500))
__unique_together__ = [["order_id", "line_no"]]
__foreign_keys__ = [ForeignKey("order_id", references=Orders)]
Orders.to_markdown("docs/orders.md") # writes and returns
markdown = Orders.to_markdown() # just returns
The document has three parts: an overview, a table of every column, and — when the spec declares any — a constraints section covering composite keys, checks, foreign keys, conditional rules and column validators:
# Orders
## Overview
- **Schema:** `Orders`
- **Total Columns:** 5
- **Foreign Keys:** 1 key(s)
## Columns
| Column | Type | Nullable | Bounds | Domain / Choices | String Length | Tags | Rules | Unique |
|:---|:---|:---|:---|:---|:---|:---|:---|:---|
| `order_id` | `Int64` | No | [1, 1000000] | - | - | - | - | Yes |
| `customer_id` | `Int64` | No | [1, 10000] | - | - | - | - | No |
| `status` | `Enum` | No | - | `NEW`, `PAID`, `SHIPPED` | - | - | - | No |
| `total` | `Float64` | No | [0.0, 5000.0] | - | - | - | - | No |
| `internal_note` | `String` | Yes (10%) | - | - | [0, 40] | - | - | No |
## Constraints & Invariants
### Foreign Keys
- **`fk_customer_id__Customers`**: `['customer_id']` -> `Customers.['id']`
A column's choices, or an Enum's categories, appear in its Domain /
Choices cell, each beside its share when the column is weighted. An
open-ended bound reads as >= 0.0 rather than [0.0, None]. The
Nullable cell shows how often a nullable column is null, and also notes a
column's default and whether it is optional. Pass title= to override
the heading, which otherwise uses the class name.
Long lists of choices¶
A cell lists four choices whole; a longer list shows its first three and
its count, so a wide table stays readable. choices= decides what happens
to the rest, added in 1.6.1:
class Bonds(FrameSpec):
rating = ColSpec(pl.Enum, choices={"AAA": 1, "AA": 2, "A": 4, "BBB": 5, "BB": 2, "B": 1})
Bonds.to_markdown() # "summary": the first three and the count
Bonds.to_markdown(choices="appendix") # the same, then every value in a Choices section
Bonds.to_markdown(choices="full") # every choice in its cell
The table keeps its summary cell, and the appendix lists every value and share of each column it cut short, in declared order, in one table at the end of the document:
| `rating` | `Enum` | No | - | `AAA` 6.67%, `AA` 13.3%, `A` 26.7%, ... (6 total) | - | - | - | No |
## Choices
| Column | Value | Share |
|:---|:---|---:|
| `rating` | `AAA` | 6.67% |
| | `AA` | 13.3% |
| | `A` | 26.7% |
| | `BBB` | 33.3% |
| | `BB` | 13.3% |
| | `B` | 6.67% |
A column whose choices fit its cell is already complete, so it is left out;
an unweighted column's share is -, since its choices are what it allows,
not how often. For a spec of many such columns, "appendix" keeps the table
narrow; "full" suits a small one.
Descriptions¶
To say what a table or a column means, give it a description. The table's appears under the title, and the columns' in a Description column, which the table has only when something fills it. Nothing else reads one: it changes no generated value and no validation.
class Described(FrameSpec):
"""One row per order, as placed."""
order_id = ColSpec(pl.Int64, unique=True, description="The order's number.")
A class's docstring is its description, added in 1.2;
__description__ says it instead, and wins when a class has both — for a
docstring written for whoever maintains the class rather than whoever reads
the data. A subclass inherits neither, since it describes another table; its
inherited columns keep their own. A TableSpec takes description=, and with_description() makes a
copy.
Entity-relationship diagram¶
erDiagram
Orders {
Int64 order_id PK "bounds: [1, 1000000]"
Int64 customer_id FK "bounds: [1, 10000]"
Enum status
Float64 total "bounds: [0.0, 5000.0]"
String internal_note "nullable, len: [0, 40]"
}
Customers ||--o{ Orders : "fk_customer_id__Customers"
Columns are annotated with what the spec declares — nullability, bounds or
choices, tags, string lengths — and keyed as PK (a unique column), UK (a
member of a composite key) or FK.
Mermaid renders in GitHub, GitLab and most documentation sites, including this one, so the diagram stays live rather than becoming a stale screenshot.
A lone unique=True column is the entity's PK; when several columns are
unique each is marked UK, as is every member of a __unique_together__
group, and a foreign-keyed column FK.
Mermaid reads a name as one word of ASCII letters, digits, _ and -, so
a name it cannot read is drawn as the nearest one it can, with its real name
first in the note: a column unit price is drawn
unit_price "name: 'unit price'", and an entity 2024 Orders as
_2024_Orders. Two columns that
would draw alike are told apart by a suffix. The Markdown table escapes a
| or a newline in a name or value, so an awkward name never breaks a row.
Several specs in one diagram¶
A single spec's diagram can only name the entity a key points at. A
Registry draws every spec and every key between them:
Documenting a category registry¶
CatSpec renders the same two ways:
from polspec import CatSpec
categories = CatSpec(
enums={"STATUS": ["NEW", "PAID", "SHIPPED"]},
categoricals={"CURRENCY": pl.Categories("CURRENCY", physical=pl.UInt8)},
)
categories.to_markdown("docs/categories.md")
categories.to_mermaid("docs/categories.mmd")
The Markdown lists enums with their variants and categoricals with their
physical dtype, namespace and domain pool. The Mermaid output is a class
diagram, with each enum as an <<enumeration>>.
Keeping generated docs current¶
Both renderers are pure functions of the spec, so wiring them into a build or a pre-commit hook keeps the documentation honest: