Law of Large Numbers¶
Core Idea¶
Take a sequence of random observations X₁, X₂, X₃, … produced by a fixed probabilistic model and form the running average X̄ₙ = (X₁ + ⋯ + Xₙ)/n. The law of large numbers says this average approaches the model's expectation μ = E[X] as n grows. The claim is about the normalized aggregate and nothing else: the division by n is not bookkeeping but the whole content of the theorem, since it is what converts an accumulating quantity into a settling one. [1]
Two versions differ only in what "approaches" means. The weak law asserts convergence in probability: for every tolerance ε > 0, P(|X̄ₙ − μ| > ε) → 0, so at any single sufficiently large sample size a wide miss is improbable. The strong law asserts almost-sure convergence: with probability one the whole trajectory settles, so along a given realization only finitely many n ever miss by more than ε. The strong statement implies the weak; the weak alone permits a path that keeps returning, at ever rarer indices, to large excursions. [2]
The conditions are few and each is load-bearing. Observations must be independent, or dependent in a controlled way whose covariances decay; they must share a distribution, or vary within limits that still leave a common target; and the averaged quantity must be integrable, E|X| < ∞, so that a finite number exists for the average to approach. Finite variance is required by neither law — its absence costs the rate, and only later the result. [1]
Nowhere does the theorem say anything about a single trial. It describes the limiting behaviour of a sequence of estimators and is silent about every term of that sequence taken alone, about the next observation, and about any finite stretch of the record. Everything the prime is good for, and every way it is abused, follows from that silence.
Structural Signature¶
A stable generating model + observations accumulated into a sum + division by the count + a finite expectation as target → the empirical mean approaches that expectation in a declared mode, while the unnormalized deviation is left free to grow.
The canonical independent-and-identically-distributed hypothesis is a sufficient condition chosen for convenience, not a definition. Averages converge under stationary sequences with summable autocovariances, under martingale-difference structure, and under independent but non-identically distributed terms with controlled variances. The prime is the conclusion plus a budget for dependence and heterogeneity, not any one way of spending it. [1]
Recurring features:
- An indexed sequence under a fixed generating model. The mechanism producing observation 900,000 must be the mechanism that produced observation 12. Regime change does not degrade the pattern; it removes it.
- Normalization by the count. What converges is a per-unit quantity — an average, a rate, a relative frequency, a per-policy loss. The same data assembled without dividing converges to nothing.
- An integrable target. The expectation must exist finitely. Where the defining integral diverges there is no limit object, and the averages are not approaching something slowly; they are approaching nothing.
- A dependence budget. Independence is the cheapest way to buy convergence, but the real requirement is that no fixed set of common factors move an unbounded number of terms together.
- A declared mode of convergence. In probability, almost surely, or in mean square; the choice decides whether the guarantee attaches to a sample size or to a trajectory.
- Asymptotic form with no built-in rate. The statement quantifies over n → ∞, so every error bar and sample-size calculation comes from a second assumption layered on top.
- No term-level constraint. Nothing is imposed on any observation, any run of observations, or their order of arrival; deviations already accumulated are never reversed, only divided by a growing number. [3]
What It Is Not¶
It is not a force. Nothing in the model reaches into the sequence and nudges it. A model with an equalizing mechanism — tickets drawn without replacement, a thermostat, a quota — is a different model, and its averages converge for a different reason, one that does constrain individual draws.
It is not a claim about sums, totals, or counts. A head count, a cumulative profit, a running claim total, and the position of a random walk all typically drift further from their expected values in absolute terms as observations accumulate, even while the corresponding per-trial rates settle. Applying the prime to a quantity that has not been divided by n applies it to the wrong object.
It is not a promise about the next trial. Independence means the conditional distribution of the next observation given any history whatsoever is its unconditional distribution, so the record of past deviations is not evidence about what comes next — not weak evidence, not evidence at the margin, none. [3]
It is not a finite-sample statement. No sample size is large on the theorem's own authority; largeness becomes meaningful only once a tolerance and a scale of variation are named, and the theorem names neither.
It is not a claim that a sample resembles its distribution in every respect. Averages of integrable functions converge; that is the scope. Maxima, ranges, and extreme quantiles obey a different limit theory, and a sample can satisfy the law while its most extreme observation behaves nothing like the population's tail.
It is not a guarantee about whatever you actually care about. Solvency, ruin, a tail quantile, a compounded return, or a threshold that must not be crossed are functionals other than the mean, and convergence of the mean says nothing directly about any of them.
Broad Use¶
Survey sampling and estimation. Every estimate of a population proportion or mean from a random sample appeals to this prime, and the sampling design exists to make its independence and identical-distribution clauses approximately true. Non-response and frame coverage failures are attacks on those clauses, not statistical nuisances.
Insurance and actuarial pricing. An insurer cannot predict which house burns and does not try; it prices the per-policy expected loss and relies on the realized average across a large book converging to it. Reinsurance and geographic spread protect the independence assumption the pricing rests on.
Manufacturing and process control. Control charts monitor sample means because a mean has a known and shrinking spread while individual measurements do not; never reacting to a single measurement is this prime encoded as an operating rule.
Monte Carlo and randomized algorithms. Estimating an integral, a rare-event probability, or a posterior expectation by averaging simulated draws works because the law applies to the average of the simulated integrand, and the standard error falls as σ/√n regardless of how many dimensions the domain has. [4]
Machine learning. Empirical risk minimization justifies choosing a model by its average loss on a training sample — but the required law must hold uniformly over the whole hypothesis class rather than pointwise for a fixed predictor, which is why capacity control and held-out evaluation are necessary. [5]
Statistical physics. Macroscopic observables — pressure, temperature, magnetization — are per-particle averages over vast numbers of degrees of freedom, and they look sharp because the relative fluctuation of such an average shrinks with the particle count.
Markets and gambling houses. A casino's edge, a market maker's spread, and a lender's rate card are per-transaction expectations converted into a business by volume; broad index investing is the same move made by holding the average instead of picking terms.
Clarity¶
The confusion this prime dissolves is between an average settling down and a ledger being squared. Both get described in English as "it evens out in the long run," and only one of them is true. The average settles because the denominator grows without bound; nothing whatever squares the numerator.
Made concrete, the difference is arithmetic. A relative frequency converges to its rate while the absolute discrepancy between observed count and expectation does not converge at all — under independence the typical size of that discrepancy grows in proportion to the square root of the trial count, even as dividing it by that count sends the rate to its target. Deviations are diluted, never corrected. [3]
A second clarification separates the estimator sequence from the estimate in hand. Consistency is a property of a procedure as sample size grows, not a credential attached to the number currently on the screen.
A third fixes the phrase "in the long run": it refers to rates, and to a limit no experiment reaches. Casinos are profitable not because the long run arrives but because a per-bet edge times an enormous transaction count leaves a fluctuation small relative to reserves — a finite-sample argument the theorem cannot supply.
Manages Complexity¶
The prime is a licence to stop tracking almost everything about a system and keep two numbers. Once the conditions hold, an analyst may discard the identity of individual units, the order in which observations arrive, the outcome of any particular trial, the history of accumulated surpluses and deficits, and — for sizing purposes — the shape of the distribution, because under independence the spread of the mean is fixed entirely by the variance and the count, as σ²/n. Mean and variance size the pool; nothing else is needed. [3]
That is why an insurer employs actuaries rather than fortune-tellers and a manufacturer specifies a process capability rather than inspecting every unit: an intractable question about particulars becomes a tractable question about a rate.
The exchange has a price worth naming. Two things survive the simplification: the dependence structure, since a common factor invalidates the whole reduction, and the tail, since a distribution's rare region contributes to expectation and variance out of all proportion to how often it is seen. Those two are precisely what a per-unit average conceals, so managing complexity here means moving attention from the many things that stopped mattering to the two that now carry all of it.
Abstract Reasoning¶
The prime licenses a fixed diagnostic, run in order. Name the functional: is the quantity of interest genuinely an average of an integrable variable, or a total, a maximum, a ratio of two random quantities, or a compounded product in an average's clothing? Test integrability by asking what an observation looks like at the one-in-ten-thousand level; if that answer is unbounded or unknown, the expectation may not exist and everything downstream is void. Price the dependence by naming the common shocks — one weather system, one rate move, one shared upstream service, one interviewer — and asking how many terms each could move at once. Choose the mode by asking whether the decision concerns a snapshot or a whole path. Then obtain a rate, which requires a second assumption: the elementary variance bound gives P(|X̄ₙ − μ| ≥ ε) ≤ σ²/(nε²), so the required sample size scales as σ²/ε² — quadratic in precision, which is why halving an error bar costs four times the data. [6]
The procedure then inverts into its most useful form. When an average visibly refuses to settle — a claims ratio that will not stabilize, a benchmark whose mean keeps moving, a chart that never centres — the contrapositive says one of exactly three clauses has failed: integrability, the dependence budget, or stability of the generating model. Each has a signature: non-integrability shows as a running mean that plateaus and then steps on single observations, dependence as a variance that stops falling like 1/n while individual units still look ordinary, instability as segments of the record each internally well-behaved and mutually inconsistent. Non-convergence is not an inconvenience but a measurement of which assumption was wrong.
Knowledge Transfer¶
What crosses substrates cleanly is the algebra of the variance of a mean, and it crosses because it is a statement about covariances rather than about anything physical. In the equicorrelated case the variance of the average is σ²[ρ + (1 − ρ)/n], which tends to ρσ² rather than to zero: any positive common correlation installs a floor no sample size removes. [7] That expression governs an insurer's catastrophe exposure, a portfolio's undiversifiable risk, a poll's design effect, and a redundant system whose components share a power supply. Recognizing it in one field is recognizing it in all, and the instruction is identical everywhere: scale buys precision against idiosyncratic variation and nothing at all against common shocks.
The 1/√n rate travels with the same fidelity, as the universal exchange rate between data and precision, and so does the diagnostic order of the clauses.
What does not travel is the assumption that the average is the decision-relevant quantity, and the failure is severe wherever outcomes compound rather than add. For a multiplicative growth process the expectation across a hypothetical ensemble can grow without bound while almost every individual trajectory tends to zero, so the average over parallel worlds and the average over one world's history disagree permanently. [8] Nor does the availability of repetition travel: an agent deciding once, or facing ruin before the sequence lengthens, has no access to the guarantee however favourable the per-trial expectation. The stationarity clause travels worst of all into social and economic settings, where participants observe the average, respond to it, and thereby change the process generating it.
Examples¶
Formal/abstract¶
A fair coin is tossed ten times and lands heads eight times, a surplus of three over the balanced count. Continue tossing and ask what becomes of that surplus. The coin has no memory, so every subsequent toss contributes an expected surplus of zero and the expected surplus after any number of further tosses remains exactly three — not reduced at any horizon by any amount. Only the denominator changes. After 10,000 total tosses the expected proportion of heads is 0.5 + 3/10,000 = 0.5003; after a million it is 0.500003. The opening run is still fully present in the count and has become invisible in the rate.
The same arithmetic shows the surplus is not merely uncorrected but usually swamped. At 10,000 tosses the typical gap between count and expectation is about √10,000 / 2 = 50, so an excess of three now sits inside ordinary fluctuation seventeen times its size. Convergence of the proportion is compatible with the count wandering ever further from balance.
Now break the integrability clause. Let the observations be independent draws from a Cauchy distribution, whose tails are heavy enough that the integral defining the mean diverges. The sample mean of n such draws has exactly the same distribution as a single draw, for every n, so averaging a million observations yields an estimate no more concentrated than one observation. [9] The machinery runs, produces numbers, and buys nothing.
Mapped back: each clause of the signature does visible work here. Normalization by the count makes the proportion converge while the count does not, which is why the prime attaches to rates and never to totals. The absence of any term-level constraint keeps the surplus of three permanently on the books — the whole content of the gambler's error. Integrability separates the coin from the Cauchy case, and separates them totally rather than gradually.
Applied/industry¶
Consider a homeowners' book in which each policy carries a 0.25% annual chance of a $160,000 total loss and no loss otherwise. Expected loss per policy is $400 and the standard deviation is roughly $8,000 — a distribution dominated by an event almost no policyholder experiences. Charge $600, and the $200 loading looks thin beside an $8,000 standard deviation, which is exactly the intuition the prime overturns. Under independence the standard deviation of the average loss across n policies is $8,000/√n, so at 100,000 policies it is about $25. The loading covers roughly eight standard deviations of the pooled per-policy outcome, and the book is not merely profitable in expectation but overwhelmingly likely to be profitable in fact. [10]
Now introduce a common shock — one wildfire corridor, one flood plain, one hail season — that leaves each policy's own risk untouched while inducing a modest pairwise correlation of 0.05. Every marginal quantity used in pricing is unchanged: the same 0.25% frequency, the same $160,000 severity, the same $400 expectation. But the variance of the average no longer falls toward zero; it falls toward ρσ², a standard-deviation floor near $1,789 per policy. The loading that covered eight standard deviations now covers a ninth of one, and writing a further 900,000 policies barely moves the figure, because the surviving term contains no n.
Mapped back: the insurance business is this prime's independence clause converted into an organization. Pooling supplies the sequence, the premium is the expectation, and the commercial claim is that a per-policy average converges. Geographic diversification, per-peril limits, catastrophe reinsurance, and capital sized to tail events exist because the dependence budget is finite and its exhaustion is invisible in the marginal data an underwriter sees.
Structural Tensions¶
T1 — The asymptotic promise against the sample in hand. The theorem quantifies over n going to infinity and, unaided, delivers nothing at any particular n. Every practical use imports a second ingredient — a variance, a concentration inequality — to convert a limit statement into an interval. The tension is that the prime is invoked most eagerly where that ingredient is missing: a fund with forty months of returns, a trial whose rate looks promising. The limit is real; the entitlement at forty observations is not.
T2 — Independence carries the weight and fails invisibly. Convergence at the usual rate requires that observations not move together, yet dependence leaves no trace in the marginal distributions actually estimated. Each policy, loan, sensor, or respondent looks exactly as it did; only the joint behaviour has changed. Because a correlation floor cannot be detected in any single unit's history, the failure surfaces only when the common shock arrives, by which time the pool has been sized, priced, and sold as though the shock did not exist.
T3 — Heavy tails degrade the rate long before they break the law. With a finite mean and infinite variance the average still converges, but the usual error bar does not exist and finite-sample behaviour is dominated by rare enormous terms, so a running mean looks settled for a long stretch and then steps. Push further, to an infinite mean, and the theorem does not apply. The tension is that the regimes are indistinguishable from data, since a variance computed from any finite sample always comes back finite.
T4 — "Large" has no meaning without a scale. A sample size is never large in itself; it is large only relative to σ²/ε², so a thousand observations are ample for a proportion near one half and useless for a claims distribution whose expectation is carried by a rare six-figure loss. The tension is that practice requires a threshold and the theorem supplies none, so rules of thumb — thirty, a hundred, a thousand — migrate between problems whose variances differ by orders of magnitude.
T5 — The two convergence modes answer different questions, and the cheaper one is usually quoted. Convergence in probability speaks about a chosen sample size; almost-sure convergence speaks about a whole trajectory. For a terminal estimate the distinction is invisible and the weaker mode is easier to establish. For anything path-dependent — a bankroll that can reach zero, a controller acting on a running estimate, an adaptive stopping rule — only the stronger statement addresses the object of interest, and citing the weaker describes a guarantee nobody has.
T6 — The mean converges; the objective need not. Convergence of a per-unit average is compatible with ruin, with unbounded total exposure, and with almost every individual trajectory decaying while the ensemble expectation grows. Whenever the goal is non-linear in the average — solvency, a tail quantile, compounded wealth, a threshold that must never be crossed — the theorem's guarantee attaches to the wrong functional. The tension is structural rather than accidental: the mean is the quantity easiest to prove things about, and rarely the one being optimized.
Structural–Framed Character¶
The Law of Large Numbers sits at the structural end of the structural–framed spectrum, graded structural with an aggregate of 0.00 and every one of the five criteria reading zero. What travels is a relation among six roles: a probability model, a sequence of observations generated under it, an integrable quantity, an expectation serving as the limit target, dependence or stationarity conditions, and a declared mode of convergence. As sample size grows the empirical average approaches the target in that mode — an asymptotic claim about an aggregate, not a corrective force acting on short runs.
The criterion carrying the most weight is vocabulary, and it reads zero because the prime's terms are the relation itself rather than a lexicon borrowed from a home field. Sampling, insurance, manufacturing, simulation, finance, physics, and randomized algorithms use the same theorem, not a translated version of it.
Evaluative weight reads zero: the law describes where an average goes, and its misuse as a short-run corrective obligation is catalogued separately as the gambler's fallacy. Institutional origin reads zero — no court, market, or professional body need be present for a relative frequency to approach its rate. Human-practice-bound reads zero for the same reason: convergence is a property of the sequence and the model. Import-vs-recognize is recognition throughout.
A clean structural grade means the prime carries into any substrate — but its stated conditions travel with it. Where dependence, stationarity, integrability, or the declared mode of convergence fail, the label does not.
Substrate Independence¶
Law of Large Numbers is about as substrate-independent as a prime can be — composite 5 / 5 on the substrate-independence scale. It is a theorem, so its roles map exactly rather than approximately: a stable generating model, observations accumulated into a sum, division by the count, and a finite expectation the running average settles toward while the unnormalized deviation stays free to grow. Nothing in that statement names a field, and the same object does the work in survey sampling, insurance pricing, manufacturing tolerance, Monte Carlo simulation, portfolio construction, statistical mechanics, and randomized algorithms. The inferential discipline travels too, chiefly the silence about any single trial. Maximal abstraction, maximal spread, and instances that are the theorem itself rather than a resemblance leave nothing for a limiter to attach to.
- Composite substrate independence — 5 / 5
- Domain breadth — 5 / 5
- Structural abstraction — 5 / 5
- Transfer evidence — 4 / 5
Relationships to Other Abstractions¶
Current abstraction Law of Large Numbers Prime
Parents (3) — more general patterns this builds on
-
Law of Large Numbers is a kind of Convergence Prime
The law of large numbers is convergence specialized to stochastic empirical averages or frequencies approaching their expectation under stated conditions.An indexed family approaches a declared limit so that sufficiently late members lie arbitrarily close in the relevant convergence mode. The indexed family is a normalized aggregate of random observations, the limit is their expectation or long-run rate, and convergence is in probability or almost surely under probabilistic regularity conditions.
-
Law of Large Numbers is part of Aggregation Prime
The law contains aggregation because its empirical mean or relative frequency is constructed by combining observations into a normalized summary.Remove the sum, count, or empirical-measure aggregation across observations and there is no sample average or frequency whose limit can be asserted.
-
Law of Large Numbers presupposes Probability Prime
The law of large numbers presupposes probability because its random observations, expectation target, and convergence modes are probabilistic objects.Remove probability measures, random variables, expectation, and probabilistic convergence and the theorem loses both its sequence model and limit claim. The law is a theorem inside probability theory, not a probability measure or the general apparatus for quantifying uncertainty.
Children (1) — more specific cases that build on this
-
Gambler's Fallacy Domain-specific is part of Law of Large Numbers
Gambler's fallacy contains a misapplied law-of-large-numbers intuition, turning asymptotic aggregate convergence into a finite next-trial obligation.Remove the expectation that long-run frequencies must be enforced by short-run correction and a streak supplies no reason for the opposite outcome to feel due under independence.
Hierarchy paths (4) — routes to 3 parentless roots
- Law of Large Numbers → Convergence
- Law of Large Numbers → Aggregation → Micro Macro Linkage
- Law of Large Numbers → Probability → Measure → Set and Membership
- Law of Large Numbers → Probability → Measure → Aggregation → Micro Macro Linkage
Neighborhood in Abstraction Space¶
Law of Large Numbers sits in a moderately populated region (49th percentile for distinctiveness): it has near-neighbors but no dense thicket of synonyms.
Family — Aggregation & Distributional Effects (13 primes)
Nearest neighbors
- Asymptotic Behavior — 0.73
- Multiplicative Random Growth — 0.73
- Monte Carlo Simulation — 0.72
- Convergence — 0.72
- Central Limit Theorem — 0.70
Computed from structural-signature embeddings · 2026-09-10
Not to Be Confused With¶
The law of large numbers must first be distinguished from the Central Limit Theorem, which shares its setup and asks the opposite question. The law removes the fluctuation: the gap between sample mean and expectation goes to zero. The central limit theorem magnifies whatever is left, rescaling that same gap by the square root of the sample size to obtain a non-degenerate limiting distribution, approximately normal, whose spread is what an error bar reports. One result says the estimator lands on the right value; the other says how far off it currently is and in what shape. The division of labour has a practical edge: a confidence interval cannot be derived from the law at all, and the law holds for integrable variables with infinite variance where the standard normal limit is unavailable.
It is not Regression to the Mean, a finite-sample effect conditional on selection. That pattern concerns the next measurement of a unit chosen because its previous measurement was extreme, and it arises from imperfect correlation between an observed score and the underlying quantity. The prime here concerns the average of many units and arises from division by a growing count. The two make incompatible kinds of prediction: regression to the mean does license a directional expectation about a specific follow-up observation, which is precisely the short-run inference the law of large numbers refuses to make. A player with an exceptional opening month is expected to do worse next month; a coin that has produced a surplus of heads is owed nothing.
It is a special case of Convergence, its parent, and the specialization adds three commitments that general convergence lacks. The indexed family must be random rather than a sequence of fixed numbers; that family must be a normalized aggregate constructed from the observations rather than any indexed collection; and the limit must be an expectation fixed by the model rather than an arbitrary target. The convergence modes exist only because the terms are random variables, and they collapse into ordinary convergence once the randomness is removed.
It is not Risk Pooling or Diversification, both of which are engineered arrangements that exploit the theorem rather than restatements of it. Pooling is an institutional design — contracts, a balance sheet, reserves, a regulator — assembled so that the independence and stationarity clauses hold well enough to price against. Diversification is a construction problem in which the weights are decisions and the correlation matrix is the object being managed, so the equal-weight independent average the law describes is one corner of a much larger design space. Both can be executed well or badly; the theorem admits of no such thing.
It is not Wisdom of the Crowds, which requires additional and quite different conditions. Crowd aggregation involves heterogeneous, non-identically-distributed judges whose individual errors must be uncorrelated in order to cancel, and its accuracy decomposes into average individual error minus the diversity of judgments — a decomposition with no counterpart here, where observations carry no bias to cancel and diversity is not a design variable. The failure mode is diagnostic: a crowd sharing a systematic bias converges reliably on the wrong answer, and additional members make it more confident rather than more correct.
Finally, it is not the Clustering Illusion, though the two are usually met together. The clustering illusion is the perceptual tendency to read intent or structure into runs that an independent process generates at exactly the expected rate. The gambler's fallacy is its mirror image, expecting the process to erase such runs. Both misread this prime's silence about individual trials — one denies that streaks should occur, the other insists they must be repaid — and the correct response to a streak under the law is neither to infer a mechanism nor to bet against it.
Solution Archetypes¶
No catalogued solution archetypes reference this prime yet.
Notes¶
Two of the prime's failure modes look identical from inside a spreadsheet and demand opposite responses. An average that will not settle because the underlying distribution has a heavy tail needs a different estimator or a truncated exposure; one that will not settle because a common factor links the observations needs a redesigned pool, and no estimator can rescue it. Telling them apart requires evidence the average itself does not contain, which is why the diagnostic asks about mechanisms rather than about the series.
Most of the reasoning errors filed under the prime's name — the gambler's fallacy, the interval quoted without a variance, the "we have thousands of data points" defence against a correlated shock — claim a guarantee the theorem declines to give.
References¶
[1] Billingsley, Patrick. Probability and Measure. 3rd ed. Wiley, 1995. States the law of large numbers for the normalized partial sum under a finite expectation with no finite-variance requirement, and extends convergence of averages to ergodic stationary sequences, martingale differences, and independent non-identically distributed terms. registry ↩a ↩b ↩c
[2] Kolmogorov, A. N. Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer, 1933. Chapter VI treats both the ordinary and the strong law of large numbers, fixing the distinction between convergence in probability and almost-sure convergence of the empirical mean. registry ↩
[3] Feller, William. An Introduction to Probability Theory and Its Applications, Volume I. 3rd ed. Wiley, 1968. Shows that the law constrains no individual trial and compensates no past deviation, that the absolute deviation of the count from expectation grows on the order of the square root of the trial count, and that the variance of the mean is the variance over n. registry ↩a ↩b ↩c ↩d
[4] Robert, Christian P., and George Casella. Monte Carlo Statistical Methods. 2nd ed. Springer, 2004. Grounds Monte Carlo integration in the law of large numbers, with the error of the simulated average falling as sigma over root n irrespective of the dimension of the domain. registry ↩
[5] Vapnik, V. N., and A. Ya. Chervonenkis. "On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities". Theory of Probability & Its Applications 16(2), 1971, 264-280. Establishes that empirical frequencies must converge uniformly over the whole class of events, not pointwise for a fixed one, which is the condition empirical risk minimization actually needs. registry ↩
[6] Chebyshev, P. L. "Des valeurs moyennes". Journal de Mathematiques Pures et Appliquees, 2nd series, 12, 1867, 177-184. Gives the variance-based tail bound that yields the sigma-squared over n-epsilon-squared rate and hence the quadratic dependence of required sample size on precision. registry ↩
[7] Markowitz, Harry. "Portfolio Selection". The Journal of Finance 7(1), 1952, 77-91. Argues that the law of large numbers does not deliver its usual benefit across a portfolio because the component yields are intercorrelated, so diversification cannot eliminate all variance and a floor set by the common covariance survives any number of holdings. registry ↩
[8] Peters, Ole. "The ergodicity problem in economics." Nature Physics 15, 2019, 1216-1221. Demonstrates that for a multiplicative growth process the ensemble average over parallel realizations and the time average along a single trajectory diverge permanently. registry ↩
[9] Feller, William. An Introduction to Probability Theory and Its Applications, Volume II. 2nd ed. Wiley, 1971. Establishes the stable-law property under which the mean of n Cauchy variates has the distribution of a single variate, so averaging does not concentrate. registry ↩
[10] Bowers, Newton L., Hans U. Gerber, James C. Hickman, Donald A. Jones, and Cecil J. Nesbitt. Actuarial Mathematics. 2nd ed. Society of Actuaries, 1997. Develops insurance risk pooling, in which the standard deviation of the average claim cost falls as the reciprocal square root of the number of independent policies, and sets the premium loading against it. registry ↩