Skip to content

Family-Wise Error Rate

The probability that a declared family of simultaneous hypothesis tests contains at least one false rejection, with weak or strong control determined by which configurations of true nulls are covered.

Version
v3 · 2026-09-06 · History
Domain-specific #
1824
Origin domain
statistics
Subdomain
multiple hypothesis testing
Aliases
FWER, Familywise error rate, Family-wise Type I error rate

Core Idea

For a declared family of hypothesis tests, the family-wise error rate is the probability of making at least one Type I error: Π\((V\ge1)\), where (V) is the number of false rejections among true null hypotheses. A procedure controls FWER at level α when that probability does not exceed α under the specified guarantee.[1]

Weak control applies only under the global null. Strong control applies under every configuration of true and false nulls. The family boundary is load-bearing because moving tests into or out of it changes the event whose probability is controlled.

Structural Signature

  • A prespecified family of (m) null hypotheses.
  • A truth configuration with (m_0) true nulls.
  • Rejection decisions produced jointly or separately.
  • False-rejection count (V).
  • Error event \(V\ge1\).
  • Probability evaluated under the relevant data-generating model.
  • A target level α.
  • Weak or strong control explicitly named.
  • Dependence assumptions for the control procedure.
  • A distinction from expected counts and false-discovery proportions.

What It Is Not

It is not the per-test significance level, the expected number of false positives, or false discovery rate. FWER is an error criterion, not the Bonferroni method used to control it. An experiment-wide collection is not automatically the most meaningful family when it mixes unrelated inferential selections.

Scope of Application

FWER is used when even one false positive in a coherent claim family is costly or simultaneous coverage is required. Holm's sequentially rejective procedure controls strong FWER without dependence restrictions and uniformly improves on single-step Bonferroni.[2] Closed testing supplies a general framework for strong control through intersection hypotheses.[3]

Clarity

Define the family before inspecting results; state confirmatory versus exploratory status, weak or strong control, dependence conditions, adjusted p-values or critical values, and whether selection occurred across analyses. Report the procedure separately from the criterion.

Manages Complexity

FWER turns many local false-positive risks into one global event: “any false claim in this family.” This supplies an interpretable simultaneous guarantee but spends error budget conservatively as families grow.

Abstract Reasoning

  1. Define the inferential decision and smallest coherent selection family.
  2. Enumerate its hypotheses and planned selection rule.
  3. Choose the desired error criterion and level.
  4. Select a procedure valid under the dependence structure.
  5. Compute adjusted thresholds or p-values.
  6. Reject according to the joint rule.
  7. Interpret the guarantee over truth configurations.
  8. Avoid silently redefining the family after seeing outcomes.

Westfall and Young show how resampling can exploit dependence to improve power while retaining joint error control under stated conditions.[4]

Knowledge Transfer

The portable pattern is treat any one failure across a declared decision family as the adverse event and bound its probability jointly. It transfers to simultaneous confidence systems. The proposed immediate parent is Risk.

Examples

Testing 20 independent true nulls at .05 gives a chance of at least one false rejection far above .05. A Bonferroni rule tests each at .0025 to keep FWER at or below .05 without assuming independence.

Holm's ordered step-down rule can reject more hypotheses while maintaining the same strong family-level guarantee.

Structural Tensions

  • Simultaneous protection versus power.
  • Family coherence versus opportunistic boundary choice.
  • Strong control versus global-null-only control.
  • Distribution-free conservatism versus dependence-aware efficiency.
  • Any-error loss versus proportion-of-errors loss.

Structural–Framed Character

Joint adverse-event probability is structural. Null hypotheses, Type I errors, p-values, multiple testing, and simultaneous inference are constitutive. The identity is domain-specific.

Structural Core vs. Domain Accent

The structural core is many decisions -> any-failure event -> bounded joint risk. The domain accent is false rejection across hypothesis tests.

Risk is the proposed immediate parent. Type I & Type II Errors, Conditional Probability, Family Boundary, and Selection are related primes. Bonferroni Correction and Multiple Comparisons Correction are procedure neighbors.

The prospective queue contains one strict edge to prime:risk. No live DAG mutation is authorized.

Relationships to Other Abstractions

Local relationship map for Family-Wise Error RateParents appear above the current abstraction, mutual partners to the right, and children below. Node labels state whether each abstraction is prime or domain-specific; colors identify relation types.Family-WiseError RateDOMAINPrime abstraction: Risk — is a kind ofRiskPRIME

Current abstraction Family-Wise Error Rate Domain-specific

Parents (1) — more general patterns this builds on

  • Family-Wise Error Rate is a kind of Risk Prime

    Risk is the proposed immediate parent.

Hierarchy paths (3) — routes to 3 parentless roots

Neighborhood in Abstraction Space

Family-Wise Error Rate sits in a sparse region of the domain-specific corpus (92nd percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (1565 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-09-08

Not to Be Confused With

  • Per-comparison error rate.
  • False discovery rate.
  • Expected number of false rejections.
  • Bonferroni correction.
  • Experiment-wise error without a justified family.
  • Probability that every rejected hypothesis is false.

References

[1] Yosef Hochberg and Ajit C. Tamhane, Multiple Comparison Procedures (Wiley, 1987), doi:10.1002/9780470316672. registry ↩

[2] Sture Holm, “A Simple Sequentially Rejective Multiple Test Procedure,” Scandinavian Journal of Statistics 6 (1979): 65–70. registry ↩

[3] Ruth Marcus, Eric Peritz, and K. R. Gabriel, “On Closed Testing Procedures with Special Reference to Ordered Analysis of Variance,” Biometrika 63, no. 3 (1976): 655–660, doi:10.1093/biomet/63.3.655. registry ↩

[4] Peter H. Westfall and S. Stanley Young, Resampling-Based Multiple Testing (Wiley, 1993), doi:10.1002/9780470316672. registry ↩