Goodness-of-Fit Test¶
Compare a specified statistical model with observations through a discrepancy statistic calibrated under that model to test its adequacy.
Core Idea¶
A goodness-of-fit test checks whether observed data depart unusually far from what a specified statistical model predicts. It chooses a way to measure that departure, then compares the observed value with what could occur under the model and the sampling design.[^ref-351fec640cf0] An unusually large departure counts against the tested fit. A result that is not unusual leaves the model unrefuted by this test; it does not prove the model true or guarantee good prediction.
The formal pattern is model implication → observed discrepancy → null calibration → bounded adequacy conclusion. In NIST's fatigue-life example the discrepancy comes from observed and expected category counts.[^ref-351fec640cf0] In a Pu-sorption experiment it comes from regression lack-of-fit variation compared with replicated pure error.[^ref-d8b4bf1e5c80] Both are tests despite their different data and formulas.
This is a kind of Statistical Test. A calibrated statistical test may instead ask whether a sequence is random or whether two variances are equal; the goodness-of-fit subtype asks whether a specified model fits its observations adequately.
What It Is Not¶
A visual fit check, a residual plot, or an in-sample score is not by itself this formal test. NIST treats AIC and BIC as tools for comparing models.[^ref-351fec640cf0] Ranking candidate models also does not establish that the preferred one passes an adequacy test.
A nonsignificant result is not proof that a model is correct. A test might have limited power or look for a departure different from the one that matters. In the Pu-sorption study, the revised model's lack-of-fit test was nonsignificant, yet two among three test points and one reserve point lay outside reported prediction intervals.[^ref-d8b4bf1e5c80]
Scope of Application¶
The method applies when a statistical model has observable implications and a defensible reference comparison is available. A distributional test can compare observed counts with model probabilities or an empirical distribution function with a theoretical one.[^ref-351fec640cf0] A regression lack-of-fit test can compare model residual variation with pure error where replicated design points permit that split.[^ref-d8b4bf1e5c80]
Each method has conditions. Chi-square results depend on grouping and adequate expected counts; ordinary K-S critical values assume a fully specified continuous distribution and need adjustment when parameters are estimated from the tested data.[^ref-351fec640cf0] The Pu-sorption F comparison uses center-point replication and pertains to the authors' revised model after a previously flagged, instrument-limited run was excluded.[^ref-d8b4bf1e5c80]
Clarity¶
Before interpreting a p-value, name the model, observations, design, discrepancy statistic, and reference rule. State whether parameters were fitted from the same data and how the calibration handles that fitting. If the data or model changed, say which version the test evaluates.
For the Pu-sorption study, the authors flagged Run 6 near the detection limit before statistical analysis and excluded it from the revised model. Table 8's F result describes that revised fit.[^ref-d8b4bf1e5c80] The reported p-value cannot be transferred to the earlier fit without rerunning the analysis.
Manages Complexity¶
One calibrated statistic condenses many data-model comparisons into a readout. This is useful, but it can hide where a departure occurred. NIST's chi-square statistic summarizes deviations across bins, and its grouping discussion explains that bin choice can change power.[^ref-351fec640cf0] Inspecting individual bins, residuals, and new-data performance may answer questions the omnibus test does not.
A formal test cannot make up for an unsuitable sampling design or missing replication. It also cannot turn a weak ability to detect a mismatch into evidence of model truth.
Abstract Reasoning¶
Ask: What would this model predict? What departure matters? How surprising would that departure be if the model and design assumptions held? The last question separates a formal test from merely noticing that data and predictions differ. Finite samples ordinarily differ from exact model predictions; the reference comparison judges whether the difference is unusually large for the specified model.
The inference is asymmetric. A large calibrated departure can challenge the tested implication. A small departure means only that this test did not detect a mismatch. The two examples below illustrate different forms of this reasoning.
Knowledge Transfer¶
Carry the five roles to a new statistical setting, then choose an appropriate discrepancy and calibration for that setting. Do not copy a test formula without its conditions. NIST explains that the standard K-S reference assumes a fully specified continuous distribution; fitting its parameters from the tested sample changes the calibration problem.[^ref-351fec640cf0]
A goodness-of-fit test is more specific than Statistical Test because its target is a model's adequacy. It is also related to Statistical Inference, which includes estimation and other ways of drawing uncertain conclusions from samples. NIST describes count and empirical-distribution-function tests.[^ref-351fec640cf0] Residual Analysis can motivate a test, but those two tests do not require a residual series.
Examples¶
Aluminum-alloy fatigue life. NIST describes 101 measured failure lives grouped for a chi-square check of a fitted three-parameter Weibull distribution. The model supplies expected counts, observed counts supply the departure, and an approximate chi-square reference yields p = 0.15.[^ref-351fec640cf0] Mapped back: Weibull model = proposed implication; measured lives and bins = data/design; count differences = discrepancy; chi-square law = calibration; non-rejection for these data = bounded conclusion. The original experiment report was not inspected.
Pu sorption to cementitious mortar. Coppens and colleagues varied calcium and isosaccharinic-acid levels, fitted a response surface, and used replicated center points to estimate pure error. For their revised model they reported F = 0.1619 and p = 0.8558 in a lack-of-fit comparison.[^ref-d8b4bf1e5c80] Mapped back: response surface = model; experiments and replicates = data/design; lack-of-fit variation = discrepancy; F reference = calibration; non-rejection = bounded conclusion. Two among three test points and one reserve point exceeded reported prediction intervals.[^ref-d8b4bf1e5c80] Thus this local fit result did not guarantee prediction.
Relationships to Other Abstractions¶
Current abstraction Goodness-of-Fit Test Domain-specific
Parents (1) — more general patterns this builds on
-
Goodness-of-Fit Test is a kind of Statistical Test Domain-specific
A goodness-of-fit test is a statistical test whose null-calibrated comparison specifically measures departure from a specified model’s observable implications.
Children (1) — more specific cases that build on this
-
Pearson's Chi-Squared Test Domain-specific is a kind of Goodness-of-Fit Test
Pearson's categorical-count procedure is a goodness-of-fit test with a fixed weighted residual statistic.
Hierarchy path (1) — routes to 1 parentless root
- Goodness-of-Fit Test → Statistical Test
Neighborhood in Abstraction Space¶
Goodness-of-Fit Test sits in a sparse region of the domain-specific corpus (94th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.
Family — Unclustered & Miscellaneous (2721 abstractions)
Nearest neighbors
- Universal Hypothesis Testing — 0.78
- F-test of equality of variances — 0.77
- Statistical Conclusion Validity — 0.77
- Normality test — 0.77
- Chauvenet's criterion — 0.77
Computed from structural-signature embeddings · 2026-10-17
References¶
[^ref-351fec640cf0]: National Institute of Standards and Technology, “NIST/SEMATECH e-Handbook of Statistical Methods,” sections “Chi-Square Goodness-of-Fit Test” (§1.3.5.15), https://itl.nist.gov/div898/handbook/eda/section3/eda35f.htm; “Kolmogorov-Smirnov Goodness-of-Fit Test” (§1.3.5.16), https://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm; “Chi-square goodness-of-fit test” (§7.2.1.1), https://itl.nist.gov/div898/handbook/prc/section2/prc211.htm; and “Graphical Output and Interpretation” (§1.4.2.9.2), https://itl.nist.gov/div898/handbook/eda/section4/eda4292.htm (official technical handbook; first three full HTML sections inspected, fourth official indexed Model Selection passage inspected because direct page fetch failed). [^ref-d8b4bf1e5c80]: Erik Coppens, Katinka Wouters, Benny de Blochouse, and Delphine Durce, “Investigating the effect of pore water composition and isosaccharinic acid (ISA) on sorption of Pu to a CEM III/C-based mortar,” Frontiers in Nuclear Engineering 2 (2023), DOI 10.3389/fnuen.2023.1151271, https://www.frontiersin.org/journals/nuclear-engineering/articles/10.3389/fnuen.2023.1151271/full (original experiment, publisher full HTML inspected; model revision around Run 6, Table 8 lack-of-fit/pure-error F comparison, Table 9 test-point prediction limits).