Counternull¶

A nonnull effect value or set that matches a designated null's p-value under a specified test of the observed data.

Version: v2 · 2026-10-03 · History
Domain-specific #: 13101
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomains: Hypothesis Testing, Effect Size Reporting → Experimental Design & Statistics
Aliases: Counternull Value, Counternull Set

Core Idea¶

A counternull is a nonnull effect or parameter value that receives the same p-value as a designated null value when each is tested against the observed data by a specified procedure. Rosenthal and Rubin introduced it as a reporting companion to the usual null result. It reminds readers that failing to reject zero does not show an effect is zero, while rejecting zero does not establish a scientifically important magnitude.[^ref-4ed491228e81] Equality here means equality of the test's tail scores, not equal probabilities that the competing values are true. In suitable symmetric one-dimensional tests with zero null, the counternull is twice the observed estimate. That is a conditional shortcut, not the definition; randomization-based extensions can yield nonunique counternull sets.[^ref-8767492fd68f]

Scope of Application¶

The construction began in psychological effect-size reporting and can be applied to other effect or parameter analyses when the estimand, null value, observed statistic, test and hypothesis-specific p-value rule are stated. A randomized experiment may produce a counternull Η set rather than one reflected number. In Bind and Rubin's body-worn-camera reanalysis, the use-of-force outcome yielded different approximate equal-p sets under an adjusted regression statistic and a Horvitz–Thompson statistic, each compared only with its own zero-null Fisher randomization p-value. The test used 10,000 simulated allocations, so the reported narrow sets are approximations; the estimand is the effect of assignment to cameras, not necessarily of actually wearing them.[^ref-8767492fd68f] A numerical counternull does not by itself establish practical importance or causality beyond the design assumptions.

Clarity¶

Ask what null is being tested, what data and statistic are fixed, how the test computes p under alternative candidate values, and which nonnull values attain the same p as the original null. A bare estimate and p-value do not always suffice to verify the counternull. In a simple symmetric normal test, an estimate of 0.25 with stipulated standard error 0.20 has the same two-sided tail area against values 0 and 0.50 because both are 1.25 standard errors away. This is an illustrative calculation, not a reported psychological study. Without the stipulated standard error and test, the numerical p cannot be inferred.[^ref-4ed491228e81]

Manages Complexity¶

The counternull puts a concrete nonzero comparator beside a frequently overinterpreted null p-value. It can interrupt the jump from “not significant” to “no effect,” while keeping attention on magnitude. Its compactness is also its hazard: a single number can conceal model assumptions, uncertainty across other values and dependence on the chosen statistic. It should accompany, not replace, an estimate, uncertainty analysis and substantive judgment.[^ref-8767492fd68f]

Abstract Reasoning¶

Hold the data and testing rule fixed, then look for nonnull parameter values on the same p-value level as the designated null. For a symmetric two-sided normal test with estimate \(\hat\theta\), null \(\theta_0\) and fixed standard error, the reflected value is \(2\hat\theta-\theta_0\) when distinct from \(\theta_0\). Under discrete or asymmetric tests there may be several values or no simple reflection. The resulting counternull belongs to the specified test, not to the data alone.[^{ref-4ed491228e81][}ref-8767492fd68f]

Knowledge Transfer¶

To carry the idea from a standardized psychological effect to a treatment contrast, preserve the null–data–test–equal-score roles and recompute on the target effect scale. Do not automatically double a risk difference, odds ratio or transformed coefficient, and do not translate equal tail areas into equal posterior belief or practical importance. The transferable question is “what nonzero hypothesis does this same inferential procedure score like the null?”[^ref-8767492fd68f]

[^ref-4ed491228e81]: Robert Rosenthal and Donald B. Rubin, “The Counternull Value of an Effect Size: A New Statistic,” Psychological Science 5, no. 6 (1994): 329–334, especially publisher abstract. Original publisher record. [^ref-8767492fd68f]: M.-A. C. Bind and Donald B. Rubin, “Counternull Sets in Randomized Experiments,” The American Statistician 79, no. 2 (2025): 275–285, especially Abstract and §§1–3. Original open-access paper.

Relationships to Other Abstractions¶

Current abstraction Counternull Domain-specific

Parents (1) — more general patterns this builds on

Counternull presupposes Statistical Significance (p-Value) Prime

A counternull is selected by matching the p-value calculated for a designated null under a specified test.

Hierarchy paths (11) — routes to 5 parentless roots

Counternull → Statistical Significance (p-Value) → Statistical Inference → Inductive Reasoning

Show alternative paths (10)

Neighborhood in Abstraction Space¶

Counternull sits in a sparse region of the domain-specific corpus (73^rd percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Statistical Hypothesis Tests & Diagnostics (9 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-08