Alpha-Spending Plan¶
Sequential testing control — instantiates Multiple-Testing Discipline
Treats the total false-positive budget as a currency spent in pre-planned fractions across repeated interim looks, so peeking at accumulating data never inflates the error rate.
An Alpha-Spending Plan handles a multiplicity that lives in time rather than in breadth: one hypothesis, examined again and again as evidence trickles in. Each time you look at partial data and ask "is it significant yet?", you get another chance for chance to say yes — so a study that peeks five times at the naïve 0.05 bar is nowhere near a 5% false-positive rate. The plan fixes this by declaring the total tolerable error (alpha) a finite budget and pre-committing a schedule for spending small slices of it at each look, calibrated to how much information has accumulated. The single idea that makes it this mechanism and not a sibling: the many "tests" are repeated glances at the same growing dataset, and the correction is a spending curve over the sequence of looks — not a division across a fixed set of distinct comparisons.
Example¶
A cardiology trial testing whether a new drug lowers stroke risk enrolls patients over three years. Waiting the full three years to learn the answer is costly if the drug is clearly working — or clearly harming — so an independent Data Safety Monitoring Board schedules four interim looks plus a final analysis. Left uncorrected, five looks at p < 0.05 would let the true false-positive rate drift toward roughly one in six. Instead the statisticians set a total budget of α = 0.05 and adopt a conservative spending function that releases almost nothing early and most of the budget at the end: the first look might require p < 0.001, the second p < 0.004, and so on, with the boundary loosening as the data mature. At the third look the treatment curve crosses its (very strict) efficacy boundary; because the plan reserved error for exactly this possibility, the board can stop the trial and call the result significant without having spent more error than a single end-of-study test would have. The plan is what lets them look early and keep the answer trustworthy.
How it works¶
- Fix the total budget. Choose the overall α to protect across the whole sequence of looks.
- Choose a spending function. A monotone function maps the fraction of information collected so far to the cumulative α allowed to have been spent by then. Front-loaded shapes spend early and stop aggressively; back-loaded shapes hoard error for the end.
- Compute a boundary at each look. Given the information fraction and how much α remains, derive the significance threshold for that interim analysis.
- Act on crossing. If the statistic crosses the efficacy (or futility) boundary, stop and declare; otherwise continue and carry the unspent budget forward. Flexible spending functions[n1] allow the number and timing of looks to shift without re-inflating error, as long as the function was fixed in advance.
Tuning parameters¶
- Total alpha — the size of the error budget; smaller protects harder but demands more evidence to stop early.
- Spending-function shape — conservative-early (little spent until late, so early stops require overwhelming evidence) versus even-spend (equal chances throughout, easier early stops, weaker final look). This is the defining dial.
- Number and timing of looks — more looks buy earlier stopping opportunities but each nibbles the budget, stiffening every boundary.
- Futility boundary — whether to also pre-plan a lower boundary for stopping when the effect looks hopeless, trading some power to end dead studies sooner.
- Information metric — whether "fraction complete" is measured in enrolled patients, accrued events, or elapsed time; the choice shifts where the budget lands.
When it helps, and when it misleads¶
Its strength is that it makes ethical and efficient early stopping compatible with honest error control: you can watch a trial in flight and still report a clean false-positive rate, because every look was paid for in advance. It is the disciplined answer to the temptation that quietly ruins repeated testing — optional stopping, the practice of continuing to collect and re-test data until the result crosses significance, which can drive the true error rate arbitrarily high.[n2] The plan neutralizes that by pre-committing when and how hard each look bites.
Its failure mode is that all the protection is contingent on the schedule being fixed before looking. Adding an unplanned extra look, quietly moving the timing after seeing an encouraging trend, or picking the spending function post hoc to justify a stop already desired reintroduces exactly the inflation it exists to prevent — correction theater with a group-sequential veneer. And a plan tuned to stop early for efficacy will, by design, tend to stop at the moment the estimate is most flattered by noise, so early-stopped effects run large. The guarding discipline is to lock the spending function and look schedule in the protocol and let an independent monitoring board, not the sponsor, apply it.
How it implements the components¶
multiplicity_adjustment_rule— the spending function is the adjustment: it converts "five looks" into a sequence of stiffened boundaries that preserve the overall error rate.error_risk_profile— declaring the total α and the shape by which it is released is a precise statement of how much false-positive risk is tolerable and when it may be spent.multiplicity_inventory— the pre-registered schedule of interim looks is the counted inventory of attempts the correction is sized against.
It does not divide a single threshold across a fixed family of distinct simultaneous tests — that reliance on the breadth of a claim_family is Bonferroni-Like Correction's move; an alpha-spending plan allocates its budget across looks in time instead.
Related¶
- Instantiates: Multiple-Testing Discipline — supplies the sequential-monitoring form of multiplicity control.
- Sibling mechanisms: Bonferroni-Like Correction · False Discovery Rate Control · Claim Registry · Confirmatory Follow-Up · Holdout Validation · Metric Hierarchy · Multiverse Analysis Report · Preregistration · Replication Study
Editorial Notes¶
Form Classification¶
Form family: Rule, Policy & Commitment
Rationale: Treats the total false-positive budget as a currency spent in pre-planned fractions across repeated interim looks, so peeking at accumulating data never inflates the error rate, making its operative form a standing constraint, permission, threshold, obligation, or conditional rule.
Independent corroboration: The frozen evidence defines Alpha-Spending Plan as 'Treats the total false-positive budget as a currency spent in pre-planned fractions across repeated interim looks, so peeking at accumulating data never inflates the error rate', so its operative form is Rule, Policy & Commitment.
Nearest alternative: Representation, Specification & Plan — Its precommitted error-budget constraint governs every interim look, while the spending schedule records that commitment.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Sequential analysis and group-sequential experimental design developed alpha-spending functions, information-time boundaries, and control of type-I error across repeated interim looks.
Related originating lineages:
- Mathematics — Probability theory underwrites stopping boundaries and cumulative error guarantees.
- Medicine & Healthcare — Clinical trials and independent data-monitoring boards made ethical early stopping under preserved error control a canonical application.
Review outcome: Independent reviewer agreement; high confidence.
Notes¶
[n1] The Lan–DeMets alpha-spending function generalizes classic group-sequential boundaries (such as O'Brien–Fleming and Pocock) by letting the number and timing of interim analyses be chosen flexibly, as long as the spending function itself is fixed in advance — which is what preserves the overall error rate under a variable look schedule. ↩
[n2] Optional stopping is the practice of repeatedly testing accumulating data and halting as soon as a threshold is crossed; without a pre-committed spending plan it inflates the false-positive rate, in the limit toward certainty of a spurious "significant" result. ↩