Heaps' Law¶

The empirical regularity that a corpus's distinct-word-type count grows as a sublinear power of the token count, V = K·N^β with β below 1, so vocabulary rises without bound ever more slowly and can be extrapolated across scale from two fitted parameters.

Version: v5 · 2026-09-28 · History · 2 corrections
Domain-specific #: 497
Domain group: Humanities
Origin domain: Linguistics & Semiotics
Subdomain: Quantitative Linguistics and Information Retrieval → Linguistics & Semiotics

Core Idea¶

Heaps' law (Heaps 1978; Herdan 1960) is the empirical regularity that a corpus's vocabulary size V — the count of distinct word types — grows as a sublinear power of the token count N: V = K·N^β, with β around 0.4–0.6 for text and K around 10–100 — a textbook rule of thumb rather than a property of the law, since K depends on the tokenization, on the units N is counted in, and on the range fitted. Growth is perpetual but decelerating: doubling the corpus multiplies vocabulary by about 2^β, not 2, with no asymptote. It is the cumulative-vocabulary face of the same Zipfian generator whose frequency-rank face is Zipf's law, coupled analytically by β = 1/α.

How would you explain it like I'm…

New Words Slow Down

When you read a book, at first almost every word is new to the list. The longer you read, the more words you've already seen, so new ones show up more slowly. But they never totally stop coming — there are always some rare new words waiting.

The Slowing Word Count

Heaps' Law is about counting the different words in a big pile of text. If you count every word you read, and also count how many different words you've met, the different-word count keeps growing, but slower and slower. Reading twice as much text doesn't give you twice as many new words — more like one and a half times as many or less. Even with a giant library, new rare words keep turning up. This helps people who build search engines guess how big their word list will get.

Sublinear Vocabulary Growth Law

Heaps' Law says the number of distinct words (types) V in a text collection grows as a power of the total number of words (tokens) N: V = K·N^β, with β usually around 0.4 to 0.6. Since β is below 1, the growth slows down: doubling the text multiplies the vocabulary by only about 1.3 to 1.5. There is no ceiling, though — new, increasingly rare words keep appearing. It is closely tied to Zipf's law, which says a few words are extremely common and most are rare; Heaps' Law is that same pattern seen as a running count of new words. One practical result is that the share of distinct words, V/N, falls as texts get longer, so you can't fairly compare word variety in a short text and a long one without correcting for length.

Heaps' Law (Heaps 1978; earlier Herdan 1960, hence also Herdan's law) is the empirical regularity V(N) = K · N^β relating type count V to token count N, with β typically 0.4–0.6 for natural language and K often quoted as roughly 10–100 as a rule of thumb. Growth is perpetual but decelerating: no asymptote, but doubling N scales V by about 2^β. The mechanism is the Zipfian rank-frequency distribution: if frequency ∝ 1/r^α with α > 1, expected types grow asymptotically as N^(1/α), so β = 1/α in the large-sample limit. This is an asymptotic coupling, not an identity; for α ≤ 1 it degenerates to near-linear growth (β = 1). Real rank-frequency curves have two regimes — α₁ ≈ 1 in the core lexicon and α₂ ≈ 2 in the rare tail — and since the tail supplies new types, β ≈ 1/α₂ ≈ 0.5 matches observation, while a single-exponent fit using α₁ wrongly predicts near-linear growth. On finite corpora, separately fitted β and 1/α need not agree exactly. Practically, it lets IR engineers predict inverted-index dictionary size via V₂ = V₁(N₂/N₁)^β, and it implies the type-token ratio falls as N^(β−1), so lexical-diversity measures need length correction.

Scope of Application¶

As a statistic, not a mechanism, Heaps' law applies literally wherever its one precondition holds — discrete-type sampling from a heavy-tailed, never-saturating generator.

Corpus-vocabulary estimation — the home use; distinct word types growing sublinearly in tokens.
Inverted-index provisioning — predicting term-dictionary memory at web scale from a small crawl.
Lexical-diversity comparison — the N^(β−1) correction making richness figures comparable across sizes.
BPE tokenizer training — projecting sub-word vocabulary a larger corpus will require.
Species accumulation — the exact ecological co-instance: Heaps' law for the species-accumulation (rarefaction) curve, which counts individuals sampled from a heavy-tailed abundance distribution; the species–area relationship counts area instead and is a sibling instance of the same scaling parent, matching only under an assumed constant density.
k-mer discovery / software vocabulary / hashtag counts — literal cumulative-type counters over heavy-tailed sampling.

Clarity¶

The law dissolves the intuition that a corpus has a fixed vocabulary size one converges on, replacing it with a sample-dependent estimate rising without bound. It sharpens the type-token ratio into a length-dependent statistic (V/N declines as N^(β−1)), so "is this author's vocabulary richer?" becomes "is the comparison at equal N?"

Manages Complexity¶

An open-ended vocabulary-growth curve, in principle re-measured at every scale, collapses to two scalars K and β: the vocabulary at any size is read off the formula, and at a new size follows by the ratio (N₂/N₁)^β. The exponent is coupled to Zipf (β = 1/α), so both faces read off one parameter.

Abstract Reasoning¶

The law's signature moves are extrapolation across scale from a partial sample (and its inverse, coverage planning); a diagnostic that treats the type-token ratio as length-confounded until checked at equal N; a cross-view move exploiting β = 1/α to infer either face from the other; and a regime-and-boundary move reading β as a fingerprint of a heavy-tailed generator that fixes when extrapolation is licensed.

Knowledge Transfer¶

Because Heaps' law is a statistic, not a causal mechanism, it transfers literally wherever its precondition holds — discrete-type sampling from a heavy-tailed generator — with only K and β re-estimated, not by analogy. Within linguistics and IR it applies directly; beyond language species accumulation, k-mer discovery, and hashtag counts are genuine co-instances. The cross-substrate reach belongs to the underlying power_law_distribution / Allometry and Scaling Law construct; the name travels only as a label. The boundary to police is precondition validity, not substrate: a bounded or converging process is outside its scope.

Relationships to Other Abstractions¶

Current abstraction Heaps' Law Domain-specific

Parents (2) — outgoing typed relations

Heaps' Law is a kind of Allometry and Scaling Law Prime

Heaps' Law is the discrete-type-growth specialization of an allometric scaling law, with distinct types scaling as a power of accumulated observations.
Heaps' Law presupposes Heavy-Tailed Distributions Prime

Heaps' Law presupposes a heavy-tailed type-frequency generator whose many rare types keep entering the cumulative count as the sample grows.

Children (1) — abstractions linked here by typed relations

Type-Token Ratio Domain-specific presupposes Heaps' Law

The guarded interpretation of Type-Token Ratio presupposes Heaps' Law because sublinear vocabulary growth is what makes raw V/N decline with sample length and determines the correction.

Hierarchy paths (2) — routes to 2 parentless roots

Heaps' Law → is a kind of Allometry and Scaling Law → is a kind of Scaling and Scale Dependence → is a decomposition of Scale

Show alternative path (1)

Neighborhood in Abstraction Space¶

Heaps' Law sits in a moderately populated region (57^th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Scaling Laws & Growth Patterns (13 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25