Regression

The statistical method of modelling an outcome as a systematic function of explanatory variables plus specified noise, fit by minimising a loss — supporting three distinct uses (prediction, effect estimation, variance attribution) each gated by its own validity conditions.

Core Idea

Regression is the statistical method of modelling a dependent quantity as a function of explanatory variables, fitting that function to data by minimising a loss, and using the fit for prediction, effect estimation, or variance attribution. It decomposes the outcome into a systematic part (a committed functional form) and a specified stochastic part, estimates parameters under an explicit loss whose assumptions are load-bearing, and supports three structurally distinct uses whose conflation is the canonical source of misinterpretation.

Scope of Application

Because regression is a statistical method, not a causal mechanism, it applies literally wherever its precondition holds — data with an outcome decomposable into a modelled systematic part and a specified noise distribution. The venues below re-use the identical apparatus.

• Statistics and data science — the home: the whole GLM family (OLS, logistic, Poisson, ridge, lasso, GAM, mixed-effects).
• Econometrics — causal-effect estimation with identification machinery layered on the fit.
• Epidemiology — Cox, Poisson, and logistic regression for survival, incidence, and case-control.
• Social science, operations, and quality engineering — effect estimation, forecasting, response-surface modelling.
• Machine learning and genetics — supervised regression; GWAS as phenotype-on-SNP-dosage regression.

Clarity

The label forces into the open four choices "we modelled the relationship" leaves implicit: variables, functional form, loss, and intended use — each contestable, each shifting interpretation. Its sharpest service is prying apart three uses one fit fuses — prediction, effect estimation, variance attribution — and enforcing that effect estimation needs identification assumptions living outside the fitting procedure the regression cannot supply.

Manages Complexity

Applied statistics confronts endless association-and-prediction problems, each seemingly its own derivation. Regression compresses that variety into one named procedure fixed by a few explicit choices, read off a portable diagnostic kit (residual plots, leverage, collinearity, cross-validation) identical across substrates. The analyst tracks four declared choices plus which of three uses is in play, and reads the verdict off that small set.

Abstract Reasoning

Regression licenses diagnosis from patterned residuals (shape back to broken assumption and bias direction), interventionist omitted-variable-bias reasoning (predict a coefficient's error before measuring the confounder; the bias-variance tradeoff), boundary-drawing that gates each use by its warrant (prediction needs out-of-sample adequacy, causal reading needs external identification), and prediction that in-sample fit is an optimistic lower bound, so cross-validated error is the quantity to read.

Knowledge Transfer

Regression is an instrument, not a causal mechanism, so it transfers by literal instrument re-use, not metaphor: the whole GLM family shares the decomposition, form commitment, loss, use discipline, and diagnostic kit, and the same machinery is deliberately embedded across econometrics, epidemiology, GWAS, and budgeting. The boundary to police is over-reading (a coefficient as causal without external warrant), not metaphor. The substrate-neutral structural residue belongs to the parents function, statistical_inference, model_based_inference, signal_extraction, and causal_inference (with distributional_assumption for the noise model).

Relationships to Other Abstractions

Current abstraction Regression Domain-specific

Parents (4) — more general patterns this builds on

• Regression is a kind of Signal Extraction Prime

  Regression is signal extraction specialized to recovering a systematic input-output relationship from an outcome entangled with stochastic residual variation.

• Regression is a kind of Statistical Inference Prime

  Regression is statistical inference specialized to estimating a systematic outcome function under an explicit stochastic model and loss.

• Regression is part of Distributional Assumption Prime

  Regression contains a distributional assumption as the internal commitment that specifies its stochastic outcome or residual component.

• Regression is part of Function (Mapping) Prime

  Regression contains a function mapping as its systematic component from explanatory-variable inputs to an outcome value or distribution parameter.

Children (2) — more specific cases that build on this

• Collinearity Inflation Domain-specific presupposes Regression

  Collinearity inflation presupposes a regression design matrix whose near-linear dependence inflates coefficient variance.

• Endogeneity Domain-specific presupposes Regression

  Endogeneity presupposes a regression-style model whose regressor, error term, and exogeneity assumption make the violation definable.

Hierarchy paths (13) — routes to 7 parentless roots

• Regression → Signal Extraction

Show alternative paths (12)

Neighborhood in Abstraction Space

Regression sits in a moderately populated region (58^th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Unclustered & Miscellaneous (309 abstractions)

Nearest neighbors

• Underfitting — 0.85
• Label Shift — 0.85
• Distributional Blind Spot — 0.84
• Attenuation Bias — 0.84
• Prior-Probability Shift — 0.83

Computed from structural-signature embeddings · 2026-07-12