Exact test¶

A hypothesis test whose null distribution and resulting type-I error control are derived without an asymptotic approximation under the stated sampling model.

Version: v2 · 2026-09-19 · History
Domain-specific #: 4455
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Statistical Inference → Experimental Design & Statistics

Core Idea¶

An exact test calibrates evidence against the actual finite-sample null distribution rather than a large-sample limit. The model enumerates or analytically derives statistic probabilities, and tail probabilities or randomized boundaries define a test whose rejection rate does not exceed the target level. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.

The load-bearing residual is not the broad topic of statistical inference. It is A hypothesis test whose null distribution and resulting type-I error control are derived without an asymptotic approximation under the stated sampling model.

How would you explain it like I'm…

Count Every Way Test

Say you flip a coin a few times and want to know if it's a trick coin. An Exact test counts up every way the flips could have come out with a fair coin, instead of using a rough guessing shortcut. That way, it doesn't fool you into blaming a fair coin more often than you agreed to.

The No-Shortcut Luck Test

Scientists use tests to decide whether a result is surprising enough to say 'something is going on.' Many tests use a handy shortcut formula that works well when you have lots of data. An Exact test skips the shortcut and works out the true chances for the actual small amount of data you have, often by counting every possible outcome. Because of that, you can trust its promise: it won't raise a false alarm more often than the rate you picked, as long as its starting assumptions are true.

Finite-Sample Exact Hypothesis Test

A hypothesis test checks whether data look too unusual to be explained by a 'null' assumption, like 'this treatment does nothing'. Most common tests rely on approximations, like the normal curve, that become accurate only as the sample size grows large. An Exact test instead derives the true probability distribution of the test statistic for your actual finite sample, by listing all outcomes or by exact math. The payoff is a guarantee: if the null is true, the chance of wrongly rejecting it (a type-I error) is no more than your chosen significance level, like 5%. This guarantee holds only under the sampling model you stated.

An Exact test calibrates evidence against the finite-sample null distribution of its test statistic rather than an asymptotic limit. Under a declared sampling model, the probabilities of each possible statistic value are enumerated or derived analytically, and tail probabilities define the rejection region. Because discrete distributions may not allow a region with exactly the target size, the test is either conservative or uses randomized boundaries to hit the level. The defining guarantee is that, for every distribution compatible with the null and the model's assumptions, the rejection probability at the actual sample size is at most the declared significance level α. This distinguishes it from large-sample tests, whose type I error control holds only approximately. The guarantee is conditional on the model: if the sampling assumptions fail, exactness does not transfer. Claiming a test is exact therefore requires stating the model, the statistic, the null set, and showing the size bound holds without approximation.

Scope of Application¶

Exact test belongs to statistical inference and is useful where the analyst can specify a null hypothesis, finite-sample sampling model, test statistic, exact null distribution, rejection region, discreteness and significance level, then evaluate under every distribution allowed by the null and assumptions, the finite-sample rejection probability is bounded by the declared significance level. The scope is broad within that domain but bounded by the need for under every distribution allowed by the null and assumptions, the finite-sample rejection probability is bounded by the declared significance level. The entry records a descriptive analytical identity; practical use requires the governing domain's evidence, standards, and safety obligations.

Clarity¶

The abstraction clarifies a crowded vocabulary by making under every distribution allowed by the null and assumptions, the finite-sample rejection probability is bounded by the declared significance level the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test. A bare label is insufficient because the name Exact test can be used for a formal identity, an implementation, or a neighboring result unless carrier and convention are stated.

Manages Complexity¶

Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Exact test. Exact test compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.

Abstract Reasoning¶

Identify the carrier. State what the elements, states, objects, or observations are: a null hypothesis, finite-sample sampling model, test statistic, exact null distribution, rejection region, discreteness and significance level. Reject examples whose alleged carrier belongs to a different problem.
Lock the constitutive rule. Express under every distribution allowed by the null and assumptions, the finite-sample rejection probability is bounded by the declared significance level independently of one notation or implementation.

Knowledge Transfer¶

Knowledge transfers strongly among subfields of statistical inference because they reuse a null hypothesis, finite-sample sampling model, test statistic, exact null distribution, rejection region, discreteness and significance level, The model enumerates or analytically derives statistic probabilities, and tail probabilities or randomized boundaries define a test whose rejection rate does not exceed the target level., and type the carrier, state every parameter and convention in the definition, test that under every distribution allowed by the null and assumptions, the finite-sample rejection probability is bounded by the declared significance level, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.

Relationships to Other Abstractions¶

Current abstraction Exact test Domain-specific

Parents (1) — outgoing typed relations

Exact test is a kind of Statistical Inference Prime

What makes it its own entry: A hypothesis test whose null distribution and resulting type-I error control are derived without an asymptotic approximation under the stated sampling model.

Hierarchy paths (3) — routes to 3 parentless roots

Exact test → is a kind of Statistical Inference → is a kind of Inductive Reasoning

Show alternative paths (2)

Neighborhood in Abstraction Space¶

Exact test sits in a crowded region of the domain-specific corpus (13^th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Statistical Inference & Hypothesis Testing (34 abstractions)

Nearest neighbors

Normality test — 0.92
Maximum likelihood estimation — 0.91
Testing hypotheses suggested by the data — 0.91
Generalized p-value — 0.91
Score test — 0.91

Computed from structural-signature embeddings · 2026-10-25