Heaps' Law¶

The empirical regularity that a corpus's distinct-word-type count grows as a sublinear power of the token count, V = K·N^β with β below 1, so vocabulary rises without bound ever more slowly and can be extrapolated across scale from two fitted parameters.

Core Idea¶

Heaps' law (Heaps 1978; Herdan 1960) is the empirical regularity that a corpus's vocabulary size V — the count of distinct word types — grows as a sublinear power of the token count N: V = K·N^β, with K around 10–100 and β around 0.4–0.6 for text. Growth is perpetual but decelerating: doubling the corpus multiplies vocabulary by about 2^β, not 2, with no asymptote. It is the cumulative-vocabulary face of the same Zipfian generator whose frequency-rank face is Zipf's law, coupled analytically by β = 1/α.

Scope of Application¶

As a statistic, not a mechanism, Heaps' law applies literally wherever its one precondition holds — discrete-type sampling from a heavy-tailed, never-saturating generator.

Corpus-vocabulary estimation — the home use; distinct word types growing sublinearly in tokens.
Inverted-index provisioning — predicting term-dictionary memory at web scale from a small crawl.
Lexical-diversity comparison — the N^(β−1) correction making richness figures comparable across sizes.
BPE tokenizer training — projecting sub-word vocabulary a larger corpus will require.
Species-area relationship — the ecological sibling, computed identically.
k-mer discovery / software vocabulary / hashtag counts — literal cumulative-type counters over heavy-tailed sampling.

Clarity¶

The law dissolves the intuition that a corpus has a fixed vocabulary size one converges on, replacing it with a sample-dependent estimate rising without bound. It sharpens the type-token ratio into a length-dependent statistic (V/N declines as N^(β−1)), so "is this author's vocabulary richer?" becomes "is the comparison at equal N?"

Manages Complexity¶

An open-ended vocabulary-growth curve, in principle re-measured at every scale, collapses to two scalars K and β: the vocabulary at any size is read off the formula, and at a new size follows by the ratio (N₂/N₁)^β. The exponent is coupled to Zipf (β = 1/α), so both faces read off one parameter.

Abstract Reasoning¶

The law's signature moves are extrapolation across scale from a partial sample (and its inverse, coverage planning); a diagnostic that treats the type-token ratio as length-confounded until checked at equal N; a cross-view move exploiting β = 1/α to infer either face from the other; and a regime-and-boundary move reading β as a fingerprint of a heavy-tailed generator that fixes when extrapolation is licensed.

Knowledge Transfer¶

Because Heaps' law is a statistic, not a causal mechanism, it transfers literally wherever its precondition holds — discrete-type sampling from a heavy-tailed generator — with only K and β re-estimated, not by analogy. Within linguistics and IR it applies directly; beyond language the species-area relationship, k-mer discovery, and hashtag counts are genuine co-instances. The cross-substrate reach belongs to the underlying power_law_distribution / allometry_and_scaling_law construct; the name travels only as a label. The boundary to police is precondition validity, not substrate: a bounded or converging process is outside its scope.

Relationships to Other Abstractions¶

Current abstraction Heaps' Law Domain-specific

Parents (2) — more general patterns this builds on

Heaps' Law is a kind of Allometry and Scaling Law Prime

Heaps' Law is the discrete-type-growth specialization of an allometric scaling law, with distinct types scaling as a power of accumulated observations.
Heaps' Law presupposes Heavy-Tailed Distributions Prime

Heaps' Law presupposes a heavy-tailed type-frequency generator whose many rare types keep entering the cumulative count as the sample grows.

Children (1) — more specific cases that build on this

Type-Token Ratio Domain-specific presupposes Heaps' Law

The guarded interpretation of Type-Token Ratio presupposes Heaps' Law because sublinear vocabulary growth is what makes raw V/N decline with sample length and determines the correction.

Hierarchy paths (2) — routes to 2 parentless roots

Heaps' Law → Allometry and Scaling Law → Scaling and Scale Dependence → Scale

Show alternative path (1)

Neighborhood in Abstraction Space¶

Heaps' Law sits in a moderately populated region (52^nd percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Minimal Units & Generative Rules (14 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-07-12