Lack-of-Fit Sum of Squares¶

Decompose regression residual variation at replicated predictor settings into irreducible within-setting pure error and systematic discrepancy between fitted values and setting means.

Version: v3 · 2026-09-19 · History
Domain-specific #: 2151
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Regression Analysis → Experimental Design & Statistics
Aliases: Lack-of-fit SS, Sum of squares due to lack of fit, SSLOF

Core Idea¶

When response observations are replicated at identical predictor settings, the residual sum of squares from a regression can be decomposed orthogonally into pure-error sum of squares and lack-of-fit sum of squares. Pure error measures variation among responses at the same setting; lack of fit measures the weighted discrepancy between each setting mean and the fitted model value.

The decomposition distinguishes noise no deterministic mean curve could remove from discrepancy attributable to the proposed functional form. Dividing each component by its degrees of freedom yields a classical F test under independent, homoscedastic normal errors. Without replicated settings, pure error cannot be estimated internally and the ordinary residual sum of squares cannot be split this way.

How would you explain it like I'm…

Wrong Shape or Just Wiggles?

Imagine you measure how tall a plant grows with different amounts of water, and you try each amount a few times. Plants with the same water still grow a little differently, and that wiggle is just luck. If your drawn line keeps missing the middle of each group, that miss means your line has the wrong shape.

Noise Versus Wrong Shape

Scientists often draw a line or curve through their data to describe a pattern. If they repeat the experiment several times at the exact same setting, the answers still differ a bit; that spread is 'pure error,' random noise no curve can fix. They can also check how far the average at each setting is from the curve; that gap is the 'lack of fit.' Splitting the total miss into these two parts tells you whether your curve has the wrong shape or the data are just noisy. Without repeats at the same setting, you can't do this split.

Splitting Error: Pure vs. Misfit

After fitting a regression model, the leftover error (the residual sum of squares) mixes two things. If you have repeated measurements at identical predictor settings, you can split it into pure error, the scatter among responses at the same setting, and lack of fit, the gap between each setting's average and the model's prediction, weighted by how many repeats there are. Pure error is noise no curve could ever remove, since the same input gave different outputs. Lack of fit is the part blamed on choosing the wrong form of model, like a straight line for a curved trend. Comparing the two, each divided by its degrees of freedom, gives an F test of whether the model's shape is adequate. Without repeated settings, there's no way to estimate pure error from the data, so the split can't be made.

The lack-of-fit sum of squares arises when a regression dataset contains replicated responses at identical predictor settings. In that case the residual sum of squares decomposes orthogonally as SS_residual = SS_pure error + SS_lack of fit. Pure error sums the squared deviations of each response from its own setting mean, capturing variation that no deterministic mean function could explain. Lack of fit sums the squared differences between each setting mean and the fitted value, weighted by the number of replicates at that setting, capturing discrepancy due to the chosen functional form. Dividing each by its degrees of freedom gives mean squares whose ratio is an F statistic, valid under independent, homoscedastic, normal errors. A large ratio suggests the model form is inadequate rather than the data merely noisy. Without replication, there is no internal estimate of pure error, so the residual sum of squares cannot be partitioned this way.

Scope of Application¶

The quantity is literal in regression experiments with replicated predictor combinations and a candidate mean function.

Calibration studies. Testing whether a linear or nonlinear response curve is adequate.
Response-surface experiments. Using replicated center or design points to estimate pure error.
Industrial experimentation. Separating process repeatability from model-form discrepancy.
Dose–response modeling. Checking a specified mean curve when replicated doses exist.
Analytical chemistry. Diagnosing calibration-function adequacy.
Regression pedagogy. Demonstrating the design-dependent ANOVA partition.

Clarity¶

List distinct predictor settings, replicate counts, fitted parameters, cell means, and all three sums of squares with degrees of freedom. Verify RSS=SS_PE+SS_LOF numerically. State error assumptions and the exact F ratio. If predictors are rounded or near-equal rather than truly replicated, justify the grouping before calling the within-cell variation pure error.

Manages Complexity¶

The partition converts a single undifferentiated residual total into two actionable sources: irreducible repeatability noise and removable model discrepancy. It tells analysts whether collecting more precise data or changing the mean function addresses the residual. The clarity depends on experimental replication; retrospective grouping can manufacture a pure-error benchmark and invalidate the diagnosis.

Abstract Reasoning¶

Identify exact replicated predictor settings.
Fit the proposed regression model.
Compute each setting's response mean.
Sum within-setting squared deviations for pure error.
Compute replicate-weighted squared deviations between cell means and model predictions.
Verify the orthogonal residual partition.
Allocate degrees of freedom and form mean squares.
Use the F comparison only under defensible error assumptions and power.

Knowledge Transfer¶

The strict parent is Decomposition: one residual total is split exactly into pure noise and model-form discrepancy, each with distinct interpretation. Residual Analysis is related, but the defining move is the design-enabled additive partition.

Decomposition is the strict parent because the operative move is an exact additive split of one squared-distance total into orthogonal components. The domain-specific residue is not merely that two numbers add: it is the projection geometry created by replicated regression settings, one subspace for within-cell departures and another for cell-mean departures from the fitted model.

Relationships to Other Abstractions¶

Current abstraction Lack-of-Fit Sum of Squares Domain-specific

Parents (1) — outgoing typed relations

Lack-of-Fit Sum of Squares is a kind of Decomposition Prime

Decomposition is the strict parent because residual variation is exactly split into within-setting pure error and between-mean model discrepancy.

Hierarchy path (1) — routes to 1 parentless root

Lack-of-Fit Sum of Squares → is a kind of Decomposition

Neighborhood in Abstraction Space¶

Lack-of-Fit Sum of Squares sits in a sparse region of the domain-specific corpus (95^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2769 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25