Lack-of-Fit Sum of Squares¶

Decompose regression residual variation at replicated predictor settings into irreducible within-setting pure error and systematic discrepancy between fitted values and setting means.

Version: v2 · 2026-09-06 · History
Domain-specific #: 2151
Origin domain: statistics
Subdomain: regression analysis
Aliases: Lack-of-fit SS, Sum of squares due to lack of fit, SSLOF

Core Idea¶

When response observations are replicated at identical predictor settings, the residual sum of squares from a regression can be decomposed orthogonally into pure-error sum of squares and lack-of-fit sum of squares. Pure error measures variation among responses at the same setting; lack of fit measures the weighted discrepancy between each setting mean and the fitted model value.^[1]

The decomposition distinguishes noise no deterministic mean curve could remove from discrepancy attributable to the proposed functional form. Dividing each component by its degrees of freedom yields a classical F test under independent, homoscedastic normal errors. Without replicated settings, pure error cannot be estimated internally and the ordinary residual sum of squares cannot be split this way.

Structural Signature¶

The replicated predictor settings. At least one design point has multiple independent responses.
The fitted mean model. A regression predicts one mean response at each setting.
The cell means. Replicate responses are averaged within identical settings.
The pure-error residuals. Individual responses deviate from their setting mean.
The lack-of-fit deviations. Setting means deviate from fitted model values.
The orthogonal partition. Residual SS equals pure-error SS plus lack-of-fit SS.
The degrees of freedom. Replication and number of fitted parameters determine component dimensions.
The F comparison. Mean-square lack of fit is compared with mean-square pure error under assumptions.

What It Is Not¶

Not the total residual sum of squares. It is one component after pure error is removed.
Not estimable from a completely unreplicated design without external error information. Identical predictor settings supply the separation.
Not proof of a particular alternative model. A large value detects discrepancy but does not identify its form.
Not the same as unexplained variance in general. Its definition depends on replicated design cells.
Not automatically a valid F test under heteroscedastic or dependent errors. Distributional assumptions matter.
Not evidence that a nonsignificant model is true. Power and replicate precision limit detection.

Scope of Application¶

The quantity is literal in regression experiments with replicated predictor combinations and a candidate mean function.

Calibration studies. Testing whether a linear or nonlinear response curve is adequate.
Response-surface experiments. Using replicated center or design points to estimate pure error.
Industrial experimentation. Separating process repeatability from model-form discrepancy.
Dose–response modeling. Checking a specified mean curve when replicated doses exist.
Analytical chemistry. Diagnosing calibration-function adequacy.
Regression pedagogy. Demonstrating the design-dependent ANOVA partition.

Clarity¶

List distinct predictor settings, replicate counts, fitted parameters, cell means, and all three sums of squares with degrees of freedom. Verify RSS=SS_PE+SS_LOF numerically. State error assumptions and the exact F ratio. If predictors are rounded or near-equal rather than truly replicated, justify the grouping before calling the within-cell variation pure error.

A complete calculation should make the indexing visible. Let the distinct predictor settings be indexed by i, with replicate count n_i, cell mean y-bar_i, individual response y_ij, and fitted value y-hat_i. Pure error sums (y_ij-y-bar_i)^2 inside cells; lack of fit sums n_i(y-bar_i-y-hat_i)^2 across cells. The replicate weight is essential because a setting supported by ten observations contributes differently from one supported by two. Degrees of freedom must reconcile as residual df = pure-error df + lack-of-fit df, with pure-error df equal to total observations minus distinct settings and lack-of-fit df equal to distinct settings minus fitted-parameter rank. Rank, rather than a memorized parameter count, matters when the design matrix is deficient. Report zero-degree components rather than forming an undefined mean square, and keep grouping rules fixed before looking at responses.^[1]

Manages Complexity¶

The partition converts a single undifferentiated residual total into two actionable sources: irreducible repeatability noise and removable model discrepancy. It tells analysts whether collecting more precise data or changing the mean function addresses the residual. The clarity depends on experimental replication; retrospective grouping can manufacture a pure-error benchmark and invalidate the diagnosis.

The diagnostic separates three questions that ordinary residual inspection can blur: whether repeated measurement is noisy, whether the declared mean model misses systematic structure, and whether the design has enough independent settings to test that distinction. A large residual total can arise almost entirely from pure error, leaving little evidence that a more elaborate mean curve will help. Conversely, small replicate scatter can make modest systematic departures detectable. The partition also exposes design dependence. Adding replicates changes the precision of the pure-error benchmark, while adding distinct settings changes the space in which model discrepancy can appear. Analysts should inspect leverage, cell imbalance, variance by setting, and dependence among replicates before treating the F reference distribution as exact. If repeated readings share a specimen, batch, or instrument drift, they may be technical repeats rather than independent experimental replicates; the algebraic identity still holds, but the inferential interpretation changes.

Abstract Reasoning¶

Identify exact replicated predictor settings.
Fit the proposed regression model.
Compute each setting's response mean.
Sum within-setting squared deviations for pure error.
Compute replicate-weighted squared deviations between cell means and model predictions.
Verify the orthogonal residual partition.
Allocate degrees of freedom and form mean squares.
Use the F comparison only under defensible error assumptions and power.

Knowledge Transfer¶

The strict parent is Decomposition: one residual total is split exactly into pure noise and model-form discrepancy, each with distinct interpretation. Residual Analysis is related, but the defining move is the design-enabled additive partition.

Decomposition is the strict parent because the operative move is an exact additive split of one squared-distance total into orthogonal components. The domain-specific residue is not merely that two numbers add: it is the projection geometry created by replicated regression settings, one subspace for within-cell departures and another for cell-mean departures from the fitted model. Transfer to another model is valid only when an analogous replication structure and nested projection can be declared. Cross-validation, penalized prediction loss, or residual plots may diagnose inadequacy, but they do not become lack-of-fit sum of squares without this cell structure. Likewise, a generic ANOVA decomposition can share formulas while targeting factor effects rather than adequacy of a fitted response surface. The parent relation therefore licenses the additive reasoning pattern, not substitution of any residual partition for the named statistic.

Examples¶

Canonical¶

Suppose a linear calibration is measured repeatedly at several concentrations. Variation among repeated readings at each concentration contributes pure-error SS. The squared distance from each concentration mean to the fitted line, multiplied by its replicate count, contributes lack-of-fit SS. Their sum exactly equals the line's residual SS.^[1]

Mapped back: replicated design → within-cell deviations + cell-mean/model deviations → orthogonal sums → residual decomposition.

Applied / In Practice¶

A response-surface experiment includes replicated center points. The lack-of-fit F test is significant while pure-error variance is small. Investigators inspect residual shape and fit curvature terms, then reserve new runs for validation. The test motivates model revision but does not by itself select the quadratic model.

Consider four calibration levels with replicate counts two, two, five, and five. A straight line is fitted to all observations. The within-level deviations establish pure error, while the weighted deviations of the four level means from the line establish lack of fit. If the two heavily replicated upper levels curve away from the line, their larger weights correctly make that systematic departure influential. The analyst verifies the three sums of squares and their degrees of freedom, then checks whether variance rises with concentration. If it does, a weighted or variance-model analysis may be required before using the classical F ratio. A significant comparison motivates a prespecified curved alternative or additional validation levels; it does not prove that every higher-order polynomial is warranted. If the levels had each been observed once, the same residual plot could suggest curvature, but this internal pure-error/lack-of-fit decomposition would not exist.

Mapped back: replicated center → pure-error benchmark → excess model discrepancy → revised candidate → external validation.

Structural Tensions¶

Noise estimation vs. design cost. Replication enables the partition but consumes runs. Diagnostic: Is the pure-error estimate precise enough for the planned test?
Exact replication vs. broader coverage. Repeated points improve error estimation while unique points map the response surface. Diagnostic: Does the design balance both goals?
Detection vs. diagnosis. A large component flags misspecification but does not name its shape. Diagnostic: What rival model is tested next?
Classical F simplicity vs. error realism. The test is clean under constant independent variance. Diagnostic: Are dependence and heteroscedasticity plausible?
Autonomous statistic vs. generic decomposition. Decomposition travels; regression replication gives these components their meaning. Diagnostic: Are cell means and a fitted mean function present?

Structural–Framed Character¶

The statistic is structural-leaning. The sum-of-squares identity is formal; predictor grouping, model family, replication design, and error assumptions are analyst-framed. It is evaluatively neutral but used to judge adequacy. It remains domain-specific because it requires regression residual geometry and replicated design cells.

A useful stopping rule is to ask which conclusion the component can support. It can establish that replicated setting means depart more than expected from within-setting variation under the fitted model and error assumptions. It cannot establish that the measurements are unbiased, that the next proposed model is adequate, or that prediction outside the observed settings will improve. Report component size as well as significance, since abundant replication can detect a departure too small to matter for the task. Conversely, a nonsignificant result with few distinct settings or imprecise pure error can be uninformative. This decision-oriented reading keeps the statistic tied to model adequacy rather than ritual hypothesis testing.

Structural Core vs. Domain Accent¶

The skeleton is aggregate discrepancy → orthogonal components with different causes → component comparison. The accent is least squares, replicated predictor cells, pure-error and lack-of-fit degrees of freedom, and the F distribution. Removing them yields generic decomposition.

Decomposition is the strict parent because residual variation is exactly split into within-setting pure error and between-mean model discrepancy. Residual Analysis is related as the diagnostic practice consuming the result.

The prospective workspace queue contains one strict upward edge to prime:decomposition. No live DAG mutation is authorized.

Relationships to Other Abstractions¶

Current abstraction Lack-of-Fit Sum of Squares Domain-specific

Parents (1) — more general patterns this builds on

Lack-of-Fit Sum of Squares is a kind of Decomposition Prime

Decomposition is the strict parent because residual variation is exactly split into within-setting pure error and between-mean model discrepancy.

Hierarchy path (1) — routes to 1 parentless root

Lack-of-Fit Sum of Squares → Decomposition

Neighborhood in Abstraction Space¶

Lack-of-Fit Sum of Squares sits in a sparse region of the domain-specific corpus (94^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Longitudinal Models & Time-Series Structure (8 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-09-08

Not to Be Confused With¶

Residual sum of squares. The total before partition.
Pure-error sum of squares. Within-setting replicate variation.
Fraction of variance unexplained. A normalized fit statistic without this replication partition.
General regression F test. Compares fitted model with a reduced model, not lack of fit against pure error.
Cross-validation error. Out-of-sample prediction loss rather than in-design ANOVA decomposition.

References¶

[1] Norman R. Draper and Harry Smith, Applied Regression Analysis, 3^rd ed. (Wiley, 1998), chapter 1. registry ↩a ↩b ↩c