Goodness-of-Fit Test¶

Compare a specified statistical model with observations through a discrepancy statistic calibrated under that model to test its adequacy.

Version: v1 · 2026-10-17 · History
Domain-specific #: 14169
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Statistical Model Checking → Experimental Design & Statistics
Aliases: Model goodness-of-fit test

Core Idea¶

A goodness-of-fit test asks whether a specified statistical model makes the observed data unusually hard to explain. It first turns the difference between a model's observable implication and the observations into a declared statistic. It then asks how extreme that statistic would be under the model and the actual sampling design.^[1] An extreme departure is evidence against the proposed fit. A non-extreme one only leaves the model unrefuted by this test; it does not establish that the model is true or that it will predict new data well. The Pu-sorption case below illustrates the prediction limit.

The operative relation persists when the objects and calculation change. In NIST's analysis of measured aluminum-alloy fatigue lives, a three-parameter Weibull distribution supplies expected counts and a chi-square statistic compares them with observed counts.^[1] In a Pu-sorption experiment, a response-surface regression supplies fitted values and a lack-of-fit F statistic compares systematic residual variation with variation among replicated center points.^[2] These are unlike distributions and data designs, but each runs model implication → observed discrepancy → model-conditional calibration → bounded adequacy inference.

This entry names the formal test relation, not every use of the phrase “goodness of fit.” NIST presents AIC and BIC as criteria for comparing alternative models.^[1] A visual residual plot, an in-sample fit score, and relative model rankings can inform assessment without themselves supplying this test's calibrated discrepancy comparison.

Structural Signature¶

Specified model. State the distribution, mean function, or other statistical claim whose observable implications are being checked. A model fitted from the same data can affect valid calibration, as NIST explains for its K-S test.^[1]
Observations and design. Identify the sampled values, categories, censoring or replication structure that determine what can be compared. NIST's chi-square example bins 101 failure lives.^[1] The Pu-sorption study uses replicate center points to estimate pure error.^[2]
Declared discrepancy. Specify what departure is measured before interpreting its size. NIST defines observed-minus-expected chi-square counts and the K-S maximum distribution-function gap.^[1] The Pu-sorption study uses regression lack-of-fit variation.^[2] These are distinct choices rather than interchangeable formulas.
Null calibration. Establish the statistic's reference behavior under the model with the relevant estimation and design constraints. NIST gives an approximate chi-square reference and a K-S critical region limited by full specification of the tested distribution.^[1] The Pu-sorption study uses a lack-of-fit F comparison.^[2] Each has different conditions.
Inferential readout. Compare the observed discrepancy with that reference and report what the result says about the tested implication. NIST's chi-square example reports p = 0.15.^[1] Without a calibrated comparison, a discrepancy remains a diagnostic number rather than a completed formal test.

Remove the specified model and there is no model-relative notion of fit. Remove the discrepancy and there is nothing to test. Remove valid calibration and the reported significance no longer has its claimed meaning. The interpretation after those steps can still be mistaken; good interpretation is a discipline for using the test, not an extra mathematical ingredient that makes it exist.

What It Is Not¶

Not a certificate of model truth. NIST reports p = 0.15 for its Weibull fatigue-life example without questioning adequacy for those data.^[1] That result does not prove that every future sample follows the same law.
Not a guarantee of prediction. The Pu-sorption study reports a nonsignificant lack-of-fit F test for its revised model, yet two of four separate test-point observations fall outside its reported 95% prediction intervals.^[2]
Not one universal statistic. NIST describes both chi-square counts and K-S distribution-function distance.^[1] The Pu-sorption study uses regression lack-of-fit variation.^[2] They answer different observable questions and have different reference conditions.
Not relative model ranking alone. AIC and BIC compare candidates in NIST's model-selection discussion.^[1] A preferred member of that set need not thereby pass a specified-model adequacy test.
Not merely a large R-squared or an uncalibrated residual display. The original Pu-sorption article reports R-squared separately from its lack-of-fit F result.^[2] Visual residual structure can motivate further checking, but the formal test requires its own discrepancy and calibration.

Scope of Application¶

The identity belongs to statistical model checking wherever a proposed probability law or fitted response model has observable consequences and a defensible reference comparison. Distributional tests may compare empirical counts with probabilities or an empirical distribution function with a specified cumulative law.^[1] Regression lack-of-fit tests can compare a mean model's residual structure with replicated pure error when the design permits that partition.^[2]

The label does not erase method-specific restrictions. NIST's chi-square approximation needs adequate counts and depends on binning; standard K-S critical values for a continuous, fully specified distribution do not carry over unchanged after its parameters are estimated from the tested sample.^[1] The Pu-sorption F comparison uses center-point replication and the reported model after an instrument-limited run was excluded.^[2] Those conditions are part of the respective applications, not optional decorations.

Clarity¶

Before reporting a p-value, identify the exact null model, the data used to estimate it, the observations being tested, the discrepancy formula, and the reference distribution or resampling scheme. State which model parameters were estimated and whether the calibration adjusts for that estimation. NIST specifies binning and approximate-reference conditions for chi-square tests.^[1] The Pu-sorption study displays lack-of-fit and pure-error components, degrees of freedom, and their F ratio.^[2] Those are the details that make the inference inspectable.

Make changes to the model or data visible. The Pu-sorption authors excluded Run 6 after identifying a measurement near the detection limit and recording a concern before their statistical analysis. Their reported Table 8 test pertains to the revised fit without that run.^[2] Reporting only the later p-value would hide which model and sample the test actually evaluated.

Manages Complexity¶

A test compresses many model-data comparisons into one calibrated readout. The compression is useful when a model predicts a distribution or response pattern too large to inspect cell by cell, but it hides where a departure occurs. NIST's chi-square count statistic reduces deviations across bins to one number; its chapter on grouping explains how a different grouping can change test power.^[1] An analyst who needs more than an omnibus rejection decision should also inspect influential bins, residual structure, or independent checks.

The compressed readout is conditional on design and model specification. It cannot repair a wrong sampling frame, create unobserved replication, or convert weak detection power into evidence that the model is correct.

Abstract Reasoning¶

The useful abstraction is a model-relative surprise test. First say what the model makes plausible under the chosen design. Then choose a discrepancy sensitive to a departure that matters for the question. Finally calibrate that statistic under the model before deciding whether the observed departure is surprising. NIST's chi-square and K-S definitions each make this model-relative comparison explicit.^[1] This reasoning separates a formal test from the simpler act of observing that data and model differ: every finite sample differs, while the reference law asks whether the difference is unusual for that model.

The result is asymmetric. A large calibrated discrepancy can count against a specified implication, subject to the test's assumptions. A non-extreme discrepancy may reflect a reasonable fit for the tested feature, limited sample size, an insensitive statistic, or a model misspecification the test was not designed to see. The Frontiers article reports test-point misses after its nonsignificant lack-of-fit result.^[2] That observation shows why one inferential readout should not be expanded into a claim of comprehensive predictive validation.

Knowledge Transfer¶

The structure transfers across statistical settings by changing the carrier of each role, not by moving a formula unchanged. In distributional reliability analysis, the model implies category probabilities, measured lifetimes fill bins, and chi-square comparison is calibrated under a fitted Weibull law.^[1] In response-surface analysis, the model implies mean sorption responses, repeated design points estimate pure error, and an F ratio tests whether systematic lack of fit is large relative to that error.^[2]

The transfer question is: What would count as a departure from this model here, and what reference behavior makes that departure interpretable? NIST's standard K-S presentation concerns a fully specified continuous distribution, and estimated parameters require recalibration.^[1] Copying that formula onto discrete counts or using unadjusted critical values after fitting parameters does not preserve those test conditions.

Examples¶

Aluminum-alloy fatigue life. NIST describes 101 measured failure lives, binned into ten groups with roughly ten expected observations each, to check a three-parameter Weibull model. The specified distribution gives expected bin counts; their weighted differences from observed counts form the chi-square statistic. After accounting for fitted parameters in the approximate reference, the reported p-value is 0.15.^[1] Mapped back: Weibull law = model; measured lives and bins = observations and design; observed-minus-expected count statistic = discrepancy; approximate chi-square law = calibration; non-rejection for this dataset = inferential readout. The original experiment report was not inspected, and this result is not a claim about all aluminum fatigue lives.

Pu sorption to cementitious mortar. In an original experiment, calcium and isosaccharinic-acid levels vary while a fitted response surface predicts log Pu distribution ratio. Repeated center points estimate pure experimental error. The authors excluded a previously flagged Run 6 whose measured Pu level was near detection; for the revised model they report lack-of-fit mean square relative to pure-error mean square, F = 0.1619 with p = 0.8558.^[2] Mapped back: response surface = model; sorption experiments and replicates = observations and design; lack-of-fit component = discrepancy; F comparison = calibration; non-rejection under the revised fit = readout. Two among three test points and one reserve point fell outside the reported prediction intervals.^[2] That limits a broader prediction claim.

A distribution-function variant. NIST's K-S test compares the empirical cumulative distribution with a fully specified continuous theoretical distribution by their maximum separation. If the parameters were estimated from these observations, its ordinary tabulated critical region no longer supplies the stated calibration without adjustment.^[1] It instantiates the same five roles with a different discrepancy.

Structural Tensions¶

The sources establish method-specific sensitivity and validity constraints, rather than a universal opposed-pressure law for all goodness-of-fit tests. For chi-square testing, the choice of bins changes power, while enough observations are needed for its approximate reference.^[1] For K-S testing, estimating distribution parameters from the same sample invalidates an unadjusted fully specified null critical region.^[1] These are concrete choices that an analyst must resolve, not reasons to claim a universally optimal test exists.

The Pu-sorption case also separates passing a local lack-of-fit check from performance at new design points. Its revised model passed the reported F comparison while two among three test points and one reserve point exceeded their prediction intervals.^[2] This is a scope limit on the test's conclusion, not a contradiction in the arithmetic.

Structural–Framed Character¶

The internal relation is formal: model implication, measured departure, null calibration, and test readout. Five character tests fix its grade. Evaluative weight: an adequacy result is evidence about a model implication, not a moral appraisal. Human-practice dependence: an analyst chooses the model, discrepancy, and error convention, but the resulting comparison is a mathematical operation on data, not a social rule. Institutional origin: no particular laboratory, regulator, or publication practice defines the test. Vocabulary travel: “null,” “sampling distribution,” and “p-value” retain technical meanings in statistics; an ordinary comparison of expectation with reality does not literally carry them. Import versus recognition: applying the test imports a specified null model and sampling design, then recognizes an observed discrepancy relative to that specification.

Its character: structural within statistics, domain-specific as a named procedure. A broad “compare expectation with reality” paraphrase would absorb residual inspection, performance monitoring, and ordinary falsification while losing the null-calibrated operation that makes this test distinctive.

Structural Core vs. Domain Accent¶

Core: a specified statistical model implies observable behavior; a declared discrepancy measures observed departure; a valid model-conditional reference makes the departure interpretable as an adequacy test. Remove any of those relations and the formal test dissolves.

Domain accent: a chi-square bin statistic, K-S distance, or regression F ratio; Weibull fatigue lives, Pu-sorption measurements, and particular significance thresholds. These determine how the core is instantiated but do not define every goodness-of-fit test. NIST's fatigue-life example yields p = 0.15; the Pu-sorption article reports p = 0.8558.^[1]^[2] Neither outcome is constitutive.

This procedure remains specific to statistics because null-model sampling behavior and calibrated inferential comparison are essential to it. Calling a business forecast “a model” and a surprise “a discrepancy” does not make a business review an instance of this test. A broader abstraction would require unlike nonstatistical cases preserving the full model–observation–discrepancy–calibration–inference relation without merely borrowing statistical words. Whether such cases exist is a separate identity question.

This entry is a kind of Statistical Test.

NIST compares observed and expected Weibull bin counts using a chi-square reference.^[1] The Pu-sorption study compares lack-of-fit and pure-error variation with an F reference.^[2] Both are calibrated comparisons of observed departure from a specified model, so the direct genus is Statistical Test. Other statistical tests can target a different question.

Hypothesis Testing (Null vs. Alternative) concerns the broader formal comparison of a null claim with an alternative through a calibrated statistic. Statistical Inference covers reasoning from data under sampling uncertainty more broadly. NIST gives count and empirical-distribution-function tests.^[1] Residual Analysis can reveal structure that motivates a fit test, but these tests need no residual series.

Lack-of-Fit Sum of Squares is a regression-specific component: it partitions residual variation under replicated predictor settings. The Frontiers article reports both the lack-of-fit and pure-error rows and the resulting F test.^[2] The formal F comparison can consume that component, but the sum of squares alone is not the general goodness-of-fit-test relation.

Relationships to Other Abstractions¶

Current abstraction Goodness-of-Fit Test Domain-specific

Parents (1) — more general patterns this builds on

Goodness-of-Fit Test is a kind of Statistical Test Domain-specific

A goodness-of-fit test is a statistical test whose null-calibrated comparison specifically measures departure from a specified model’s observable implications.

Children (1) — more specific cases that build on this

Pearson's Chi-Squared Test Domain-specific is a kind of Goodness-of-Fit Test

Pearson's categorical-count procedure is a goodness-of-fit test with a fixed weighted residual statistic.

Hierarchy path (1) — routes to 1 parentless root

Goodness-of-Fit Test → Statistical Test

Neighborhood in Abstraction Space¶

Goodness-of-Fit Test sits in a sparse region of the domain-specific corpus (94^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2721 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-17

Not to Be Confused With¶

General model assessment: plots, substantive plausibility, external validation, and sensitivity analysis may matter even when one formal test is nonsignificant.
Relative model selection: NIST uses AIC and BIC to rank alternative distributions.^[1] The preferred model does not thereby pass the specified-model discrepancy test described here.
Normality testing as the whole identity: NIST checks a Weibull model for measured fatigue lives.^[1] The Pu-sorption study tests a regression response surface.^[2] Normality is only one possible model implication.
Residual Analysis: looking for systematic leftovers can lead to a test, but studying leftovers without a calibrated inferential comparison is not yet this formal operation.
Proof by non-rejection: NIST explains that bin choice changes power.^[1] The Pu-sorption study reports test-point misses despite a nonsignificant lack-of-fit result.^[2] Neither result licenses proof by non-rejection.

References¶

[1] National Institute of Standards and Technology, “NIST/SEMATECH e-Handbook of Statistical Methods,” sections “Chi-Square Goodness-of-Fit Test” (§1.3.5.15), https://itl.nist.gov/div898/handbook/eda/section3/eda35f.htm; “Kolmogorov-Smirnov Goodness-of-Fit Test” (§1.3.5.16), https://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm; “Chi-square goodness-of-fit test” (§7.2.1.1), https://itl.nist.gov/div898/handbook/prc/section2/prc211.htm; and “Graphical Output and Interpretation” (§1.4.2.9.2), https://itl.nist.gov/div898/handbook/eda/section4/eda4292.htm (official technical handbook; first three full HTML sections inspected, fourth official indexed Model Selection passage inspected because direct page fetch failed). registry ↩a ↩b ↩c ↩d ↩e ↩f ↩g ↩h ↩i ↩j ↩k ↩l ↩m ↩n ↩o ↩p ↩q ↩r ↩s ↩t ↩u ↩v ↩w ↩x ↩y ↩z ↩27 ↩28

[2] Erik Coppens, Katinka Wouters, Benny de Blochouse, and Delphine Durce, “Investigating the effect of pore water composition and isosaccharinic acid (ISA) on sorption of Pu to a CEM III/C-based mortar,” Frontiers in Nuclear Engineering 2 (2023), DOI 10.3389/fnuen.2023.1151271, https://www.frontiersin.org/journals/nuclear-engineering/articles/10.3389/fnuen.2023.1151271/full (original experiment, publisher full HTML inspected; model revision around Run 6, Table 8 lack-of-fit/pure-error F comparison, Table 9 test-point prediction limits). registry ↩a ↩b ↩c ↩d ↩e ↩f ↩g ↩h ↩i ↩j ↩k ↩l ↩m ↩n ↩o ↩p ↩q ↩r ↩s ↩t ↩u