Family-Wise Error Rate¶

The probability that a declared family of simultaneous hypothesis tests contains at least one false rejection, with weak or strong control determined by which configurations of true nulls are covered.

Version: v4 · 2026-09-19 · History
Domain-specific #: 1824
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Multiple Hypothesis Testing → Experimental Design & Statistics
Aliases: FWER, Familywise error rate, Family-wise Type I error rate

Core Idea¶

For a declared family of hypothesis tests, the family-wise error rate is the probability of making at least one Type I error: Π\((V\ge1)\), where (V) is the number of false rejections among true null hypotheses. A procedure controls FWER at level α when that probability does not exceed α under the specified guarantee.^[1]

Weak control applies only under the global null. Strong control applies under every configuration of true and false nulls. The family boundary is load-bearing because moving tests into or out of it changes the event whose probability is controlled.

How would you explain it like I'm…

Fooled-At-Least-Once Chance

Imagine you check lots of boxes to see if a toy is hidden inside. Sometimes you might shout "found it!" when a box is really empty. The Family-Wise Error Rate is the chance that you make at least one of those wrong shouts across all the boxes you check. The more boxes you check, the easier it is to slip up at least once, so careful checkers try to keep that chance small.

The Group False-Alarm Chance

Scientists often run many tests at once, and each test can give a 'false alarm.' That means it says something is happening when it really isn't. The Family-Wise Error Rate is the chance of getting at least one false alarm in the whole group, or 'family,' of tests. Running more tests makes that chance grow, so scientists use special methods to keep it small. Which tests count as part of the family matters, because adding or removing tests changes the chance.

At-Least-One False Positive Rate

When you run several hypothesis tests together, call them a "family." A Type I error is rejecting a null hypothesis that is actually true, which is a false positive. The Family-Wise Error Rate is the probability of making at least one Type I error anywhere in the family. A procedure "controls" it at a level such as 5% if that probability never goes above 5% under its stated guarantee. This differs from controlling each test's error separately, since many tests at 5% each can together have a much higher chance of at least one false positive. Which tests count as the family matters, because adding or removing tests changes what is being controlled.

For a declared family of hypothesis tests, let V be the number of true null hypotheses that are falsely rejected. The Family-Wise Error Rate is P(V ≥ 1), the probability of at least one Type I error. A multiple-testing procedure controls FWER at level α if this probability is at most α under its stated conditions. Weak control guarantees this only when all nulls are true (the global null). Strong control guarantees it under every combination of true and false nulls, which is the stronger and usually more useful property. The family's boundary is part of the definition: moving tests into or out of it changes the event V ≥ 1 and therefore what is being controlled.

Structural Signature¶

A prespecified family of (m) null hypotheses.
A truth configuration with (m_0) true nulls.
Rejection decisions produced jointly or separately.
False-rejection count (V).
Error event \(V\ge1\).
Probability evaluated under the relevant data-generating model.
A target level α.
Weak or strong control explicitly named.
Dependence assumptions for the control procedure.
A distinction from expected counts and false-discovery proportions.

What It Is Not¶

It is not the per-test significance level, the expected number of false positives, or false discovery rate. FWER is an error criterion, not the Bonferroni method used to control it. An experiment-wide collection is not automatically the most meaningful family when it mixes unrelated inferential selections.

Scope of Application¶

FWER is used when even one false positive in a coherent claim family is costly or simultaneous coverage is required. Holm's sequentially rejective procedure controls strong FWER without dependence restrictions and uniformly improves on single-step Bonferroni.^[2] Closed testing supplies a general framework for strong control through intersection hypotheses.^[3]

Clarity¶

Define the family before inspecting results; state confirmatory versus exploratory status, weak or strong control, dependence conditions, adjusted p-values or critical values, and whether selection occurred across analyses. Report the procedure separately from the criterion.

Manages Complexity¶

FWER turns many local false-positive risks into one global event: “any false claim in this family.” This supplies an interpretable simultaneous guarantee but spends error budget conservatively as families grow.

Abstract Reasoning¶

Define the inferential decision and smallest coherent selection family.
Enumerate its hypotheses and planned selection rule.
Choose the desired error criterion and level.
Select a procedure valid under the dependence structure.
Compute adjusted thresholds or p-values.
Reject according to the joint rule.
Interpret the guarantee over truth configurations.
Avoid silently redefining the family after seeing outcomes.

Westfall and Young show how resampling can exploit dependence to improve power while retaining joint error control under stated conditions.^[4]

Knowledge Transfer¶

The portable pattern is treat any one failure across a declared decision family as the adverse event and bound its probability jointly. It transfers to simultaneous confidence systems. The proposed immediate parent is Risk.

Examples¶

Testing 20 independent true nulls at .05 gives a chance of at least one false rejection far above .05. A Bonferroni rule tests each at .0025 to keep FWER at or below .05 without assuming independence.

Holm's ordered step-down rule can reject more hypotheses while maintaining the same strong family-level guarantee.

Structural Tensions¶

Simultaneous protection versus power.
Family coherence versus opportunistic boundary choice.
Strong control versus global-null-only control.
Distribution-free conservatism versus dependence-aware efficiency.
Any-error loss versus proportion-of-errors loss.

Structural–Framed Character¶

Joint adverse-event probability is structural. Null hypotheses, Type I errors, p-values, multiple testing, and simultaneous inference are constitutive. The identity is domain-specific.

Structural Core vs. Domain Accent¶

The structural core is many decisions -> any-failure event -> bounded joint risk. The domain accent is false rejection across hypothesis tests.

This entry is a kind of Risk.

Risk is the broader abstraction this entry instantiates. Type I & Type II Errors, Conditional Probability, Family Boundary, and Selection are related primes. Bonferroni Correction and Multiple Comparisons Correction are procedure neighbors.

Relationships to Other Abstractions¶

Current abstraction Family-Wise Error Rate Domain-specific

Parents (1) — outgoing typed relations

Family-Wise Error Rate is a kind of Risk Prime

Risk is the broader abstraction this entry instantiates.

Hierarchy paths (2) — routes to 2 parentless roots

Family-Wise Error Rate → is a kind of Risk → is a kind of Uncertainty

Show alternative path (1)

Neighborhood in Abstraction Space¶

Family-Wise Error Rate sits in a sparse region of the domain-specific corpus (93^rd percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2769 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25

Not to Be Confused With¶

Per-comparison error rate.
False discovery rate.
Expected number of false rejections.
Bonferroni correction.
Experiment-wise error without a justified family.
Probability that every rejected hypothesis is false.

References¶

[1] Yosef Hochberg and Ajit C. Tamhane, Multiple Comparison Procedures (Wiley, 1987), doi:10.1002/9780470316672. registry ↩

[2] Sture Holm, “A Simple Sequentially Rejective Multiple Test Procedure,” Scandinavian Journal of Statistics 6 (1979): 65–70. registry ↩

[3] Ruth Marcus, Eric Peritz, and K. R. Gabriel, “On Closed Testing Procedures with Special Reference to Ordered Analysis of Variance,” Biometrika 63, no. 3 (1976): 655–660, doi:10.1093/biomet/63.3.655. registry ↩ Show verification details

[4] Peter H. Westfall and S. Stanley Young, Resampling-Based Multiple Testing (Wiley, 1993), doi:10.1002/9780470316672. registry ↩