PRESS Statistic¶
The sum of squared leave-one-out prediction errors from a fitted regression model, computed by refitting without each case or through leverage-adjusted ordinary residuals.
Core Idea¶
The prediction error sum of squares (PRESS) evaluates an ordinary least-squares regression by withholding each observation in turn, predicting it from the model fitted to the remaining observations, squaring the deleted residual, and summing across cases. It therefore measures within-sample leave-one-out predictive discrepancy rather than training fit.
For a fixed full-rank linear model fitted by OLS, the deleted residual can be computed without n refits as e_i/(1-h_ii), where e_i is the ordinary residual and h_ii is leverage. Thus PRESS=Σ[e_i/(1-h_ii)]². High-leverage cases are amplified because removing them changes the fit substantially. The identity depends on the fixed design/model; feature selection, preprocessing, tuning, dependence, weights, regularization, and clustered data require the entire modeling procedure to be replayed or a different validation design.
Scope of Application¶
PRESS is literal in linear-regression diagnostics and related small-sample predictive comparisons where leave-one-out validation is suitable.
- Regression comparison. Contrasting fixed candidate predictor sets.
- Influence diagnosis. Finding cases with large leverage-adjusted deleted residuals.
- Chemometrics. Evaluating calibration models under carefully nested procedures.
- Response-surface modeling. Checking predictive adequacy inside a design region.
- Teaching. Connecting hat-matrix geometry with cross-validation.
- Approximate validation. Computing exact OLS leave-one-out error efficiently.
- Sensitivity analysis. Comparing PRESS with training error and external validation.
Clarity¶
Report outcome scale, cases, model matrix, intercept, weights, preprocessing, missing-data rule, feature selection/tuning, leverage values, deleted residuals, aggregation, normalization if any, and dependence structure. State whether the analytic shortcut or explicit refits were used. Compare models only on identical outcomes and folds.
Declare the fitted model, response scale, observation unit, and whether deletion is truly one case at a time.
Manages Complexity¶
The leverage identity compresses n OLS refits into one fit and exposes influential observations. A single sum supports model comparison. Aggregation can hide concentrated failure, and the shortcut ceases to represent the actual pipeline when transformations or selections depend on all observations.
Training residuals are optimistically small because each observation helps fit the model that predicts it.
Abstract Reasoning¶
- Freeze the predictive procedure and dataset. 2. Exclude one observation. 3. Fit every data-dependent step on the remaining cases. 4. Predict the held-out outcome. 5. Record and square the deleted residual. 6. Repeat for all observations. 7. Sum or transparently normalize errors. 8. Inspect case contributions and validate deployment relevance. 9. The central derivation uses the geometry of a linear smoother. An ordinary fitted value contains a fraction of its own observed response measured by leverage.
Knowledge Transfer¶
PRESS is residual analysis turned outward: each discrepancy is computed against a prediction that did not train on that observation. Residual Analysis is the strict parent; leave-one-out refitting and the leverage identity supply the statistical accent.
Residual Analysis is the strict parent because PRESS transforms fitted errors into leave-one-out residuals and aggregates them to diagnose predictive behavior. The transferable pattern is remove each case's fitting influence → predict it → aggregate out-of-fit discrepancy. It relates to cross-validation generally, but the PRESS identity and leverage adjustment belong specifically to ordinary least-squares geometry.
Relationships to Other Abstractions¶
Current abstraction PRESS Statistic Domain-specific
Parents (1) — more general patterns this builds on
-
PRESS Statistic is a kind of Aggregation Prime
The accepted reference-grade review places PRESS Statistic under Aggregation because the child instantiates or depends on the parent's broader structure while retaining its own constitutive identity.
Hierarchy path (1) — routes to 1 parentless root
- PRESS Statistic → Aggregation → Micro Macro Linkage
Neighborhood in Abstraction Space¶
PRESS Statistic sits in a sparse region of the domain-specific corpus (88th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.
Family — Unclustered & Miscellaneous (1565 abstractions)
Nearest neighbors
- Machine-Learning Learning Curve — 0.80
- Regression — 0.80
- Robust Regression — 0.80
- Lack-of-Fit Sum of Squares — 0.80
- DFFITS — 0.79
Computed from structural-signature embeddings · 2026-09-08