Family-Wise Error Rate¶

The probability that a declared family of simultaneous hypothesis tests contains at least one false rejection, with weak or strong control determined by which configurations of true nulls are covered.

Version: v4 · 2026-09-19 · History
Domain-specific #: 1824
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Multiple Hypothesis Testing → Experimental Design & Statistics
Aliases: FWER, Familywise error rate, Family-wise Type I error rate

Core Idea¶

For a declared family of hypothesis tests, the family-wise error rate is the probability of making at least one Type I error: Π\((V\ge1)\), where (V) is the number of false rejections among true null hypotheses. A procedure controls FWER at level α when that probability does not exceed α under the specified guarantee.

Weak control applies only under the global null. Strong control applies under every configuration of true and false nulls. The family boundary is load-bearing because moving tests into or out of it changes the event whose probability is controlled.

How would you explain it like I'm…

Fooled-At-Least-Once Chance

Imagine you check lots of boxes to see if a toy is hidden inside. Sometimes you might shout "found it!" when a box is really empty. The Family-Wise Error Rate is the chance that you make at least one of those wrong shouts across all the boxes you check. The more boxes you check, the easier it is to slip up at least once, so careful checkers try to keep that chance small.

The Group False-Alarm Chance

Scientists often run many tests at once, and each test can give a 'false alarm.' That means it says something is happening when it really isn't. The Family-Wise Error Rate is the chance of getting at least one false alarm in the whole group, or 'family,' of tests. Running more tests makes that chance grow, so scientists use special methods to keep it small. Which tests count as part of the family matters, because adding or removing tests changes the chance.

At-Least-One False Positive Rate

When you run several hypothesis tests together, call them a "family." A Type I error is rejecting a null hypothesis that is actually true, which is a false positive. The Family-Wise Error Rate is the probability of making at least one Type I error anywhere in the family. A procedure "controls" it at a level such as 5% if that probability never goes above 5% under its stated guarantee. This differs from controlling each test's error separately, since many tests at 5% each can together have a much higher chance of at least one false positive. Which tests count as the family matters, because adding or removing tests changes what is being controlled.

For a declared family of hypothesis tests, let V be the number of true null hypotheses that are falsely rejected. The Family-Wise Error Rate is P(V ≥ 1), the probability of at least one Type I error. A multiple-testing procedure controls FWER at level α if this probability is at most α under its stated conditions. Weak control guarantees this only when all nulls are true (the global null). Strong control guarantees it under every combination of true and false nulls, which is the stronger and usually more useful property. The family's boundary is part of the definition: moving tests into or out of it changes the event V ≥ 1 and therefore what is being controlled.

Scope of Application¶

FWER is used when even one false positive in a coherent claim family is costly or simultaneous coverage is required. Holm's sequentially rejective procedure controls strong FWER without dependence restrictions and uniformly improves on single-step Bonferroni. Closed testing supplies a general framework for strong control through intersection hypotheses.

Clarity¶

Define the family before inspecting results; state confirmatory versus exploratory status, weak or strong control, dependence conditions, adjusted p-values or critical values, and whether selection occurred across analyses. Report the procedure separately from the criterion.

Manages Complexity¶

FWER turns many local false-positive risks into one global event: “any false claim in this family.” This supplies an interpretable simultaneous guarantee but spends error budget conservatively as families grow.

Abstract Reasoning¶

Define the inferential decision and smallest coherent selection family.
Enumerate its hypotheses and planned selection rule.
Choose the desired error criterion and level.
Select a procedure valid under the dependence structure.
Compute adjusted thresholds or p-values.
Reject according to the joint rule.
Interpret the guarantee over truth configurations.
Avoid silently redefining the family after seeing outcomes.

Knowledge Transfer¶

The portable pattern is treat any one failure across a declared decision family as the adverse event and bound its probability jointly. It transfers to simultaneous confidence systems. The proposed immediate parent is Risk.

Relationships to Other Abstractions¶

Current abstraction Family-Wise Error Rate Domain-specific

Parents (1) — outgoing typed relations

Family-Wise Error Rate is a kind of Risk Prime

Risk is the broader abstraction this entry instantiates.

Hierarchy paths (2) — routes to 2 parentless roots

Family-Wise Error Rate → is a kind of Risk → is a kind of Uncertainty

Show alternative path (1)

Neighborhood in Abstraction Space¶

Family-Wise Error Rate sits in a sparse region of the domain-specific corpus (93^rd percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2769 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25