Regression

The statistical method of modelling an outcome as a systematic function of explanatory variables plus specified noise, fit by minimising a loss — supporting three distinct uses (prediction, effect estimation, variance attribution) each gated by its own validity conditions.

Core Idea

Regression is the statistical method of modelling a dependent quantity as a function of one or more explanatory variables, fitting that function to observed data by minimising a loss (ordinarily least squares, maximum likelihood, or a regularised variant), and using the fitted model for prediction, effect estimation, or variance attribution. The structural commitments are five. First, the outcome is decomposed into a systematic part — the function of the explanatory variables — and a stochastic part — residual noise whose distribution is explicitly specified; the model is fit by making the systematic part account for as much of the outcome variation as the functional form permits. Second, the analyst commits to a functional form (linear, logistic, Poisson, polynomial, kernel, neural) before fitting; this form constrains what relationships the model can express and which departures from it will show up as mis-specified residuals. Third, the fitted parameters are estimated from data under an explicit loss, and the assumptions encoded in that loss — independence of observations, noise homoscedasticity, correct link function — are load-bearing: violations shift inferences in predictable directions. Fourth, the fitted model supports three structurally distinct uses — prediction (anticipate new outcomes), effect estimation (interpret a coefficient as a causal effect size, which requires identification assumptions external to the fitting procedure), and variance attribution (decompose outcome variance among predictors) — and conflating these three uses is the canonical source of regression misinterpretation. Fifth, validation assesses out-of-sample predictive performance and assumption violations; in-sample fit is an optimistic lower bound on both. The regression family spans ordinary least squares, logistic, Poisson, quantile, ridge, lasso, generalised additive, mixed-effects, and hierarchical Bayesian models — all sharing the decomposition, functional-form commitment, parameter estimation, and use-case discipline while varying in link function, noise model, and penalty structure.

Structural Signature

Sig role-phrases:

• the outcome variable — the dependent quantity being modelled, decomposed into a systematic part and a stochastic part
• the explanatory variables — the inputs whose relationship to the outcome is being characterised by the systematic part
• the functional form — the committed shape (linear, logistic, Poisson, polynomial, kernel, neural) constraining what relationships the model can express and which departures surface as mis-specified residuals
• the noise model — the explicitly specified distribution of residual variation, whose assumptions (independence, homoscedasticity, correct link) are load-bearing
• the loss — what counts as a good fit (least squares, maximum likelihood, regularised, robust), minimised to estimate parameters from data
• the validation strategy — out-of-sample assessment of predictive performance and assumption violations, against which in-sample fit is an optimistic lower bound
• the use-case discipline — keeping the three structurally distinct uses tagged with their own validity conditions: prediction (out-of-sample adequacy), effect estimation (identification assumptions external to the fit), variance attribution (decomposition tracking the structure)
• the identification boundary — the characteristic limitation: a coefficient is a causal effect only by an external warrant (no omitted confounders, no reverse causation, correct form) the fitting procedure itself cannot supply, however good the fit

What It Is Not

• Not a machine for reading coefficients as causal effects. A fitted coefficient is a causal effect only under identification assumptions — no omitted confounders, no reverse causation, no selection on the outcome, correct form — that live outside the fitting procedure and that the regression itself cannot supply, however good the fit. Crossing from prediction to causation without that external warrant is the field's canonical error.
• Not correlation. Correlation is a scalar summary of bivariate association; regression is a model with a committed functional form and noise distribution that supports prediction, effect estimation, and variance attribution. A regression encodes a directional outcome-on-predictors structure and a fitting discipline that a correlation coefficient does not.
• Not regression to the mean. Despite the shared word, regression to the mean is an unrelated phenomenon — extreme measurements drifting toward the average on remeasurement. The statistical method of fitting an outcome to explanatory variables has nothing to do with that reversion effect; conflating them is a pure name collision.
• Not in-sample fit as performance. A high R-squared on the training data is an optimistic lower bound on both predictive performance and assumption adequacy, and the gap widens with model flexibility. The quantity that matters is cross-validated out-of-sample error, not how well the model reproduces the data it was fit on.
• Not assumption-free. The loss encodes load-bearing commitments — independence of observations, homoscedastic noise, the correct link — whose violations shift inferences in predictable directions (narrowed standard errors under positive autocorrelation, understated uncertainty under unmodelled clustering). A regression is only as trustworthy as the residual checks that test those commitments.
• Not a single use with one validity condition. The same fit is read three structurally distinct ways — prediction, effect estimation, variance attribution — each gated by its own conditions. Treating "the model" as one verdict, rather than tagging which use is in play and checking only that use's warrant, is what licenses reading a predictively adequate coefficient as a vetted causal estimate.
• Not a law of the world that "recurs" across domains. When regression appears in econometrics, epidemiology, GWAS, or budgeting, the same apparatus is being deliberately re-embedded into a new substrate — instrument transfer, not independent structural recurrence. Its generality is a fact about the method's reach, not evidence of a substrate-neutral pattern in those fields.

Scope of Application

Because regression is a statistical method, not a causal mechanism, it is not bounded by a single domain: it applies literally wherever its precondition holds — data with an outcome decomposable into a modelled systematic part and a specified noise distribution — and the venues below are genuine re-uses of the identical apparatus, not metaphors. The boundary to police is over-reading (reading a coefficient as a causal effect without the external identification warrant), and the substrate-neutral structural residue belongs to the parents function, statistical_inference, and causal_inference.

• Statistics and data science — the substantive home: the whole generalised-linear-model family (OLS, logistic, Poisson, quantile, ridge, lasso, GAM, mixed-effects, hierarchical Bayesian) as the workhorse of applied modelling.
• Econometrics — causal-effect estimation from observational data, with identification machinery (instrumental variables, panel and difference-in-differences methods) layered on the regression fit.
• Epidemiology — Cox proportional-hazards regression for survival, Poisson regression for incidence, and logistic regression for case-control data.
• Social science — regression on survey and administrative data to estimate effects of policy, education, and environment.
• Operations and budgeting — regression of cost or demand on drivers, the standard tool of forecasting and variance attribution.
• Quality and process engineering — response-surface modelling and designed experiments analysed by regression.
• Machine learning — regression as the supervised-learning task for continuous outcomes, the boundary with classification being task-defined rather than structural.
• Genetics — genome-wide association studies as regressions of phenotype on SNP dosage with population-structure correction.

Clarity

The label forces into the open four choices that the casual phrase "we modelled the relationship" leaves implicit and therefore unexaminable: the outcome and explanatory variables, the functional form assumed, the loss minimised to fit, and the use the fit will serve. Each is contestable, each silently shifts the interpretation, and naming the procedure obliges the analyst to declare all four — turning an unaccountable claim into one whose assumptions can be inspected and attacked. A residual that does not look like the posited noise, a coefficient sensitive to the chosen link, a fit that holds in-sample but not out — these become visible only once the form and loss are stated, not buried in "we modelled it."

Its sharpest service is prying apart three uses that prose fuses under one fitted model: prediction (can it anticipate new outcomes?), effect estimation (does the coefficient measure the causal effect of moving the predictor?), and variance attribution (does the decomposition track the underlying structure?). The crucial recognition the concept enforces is that effect estimation needs identification assumptions — no omitted confounders, no reverse causation, correct form — that live outside the fitting procedure, so a coefficient is a causal effect only by an external warrant the regression itself cannot supply. Keeping the three uses tagged with their distinct validity conditions lets a practitioner ask the right question of each fit, and is what blocks the field's canonical error: reading a predictively adequate coefficient as if it were a vetted causal estimate.

Manages Complexity

Applied statistics confronts an endless variety of association-and-prediction problems — continuous outcomes, binary outcomes, counts, durations, clustered observations, non-linear shapes — each of which, taken on its own terms, demands its own derivation of how the inputs relate to the outcome and how confident one may be. Regression compresses that variety by casting any such problem into a single named procedure fixed by a small set of explicit choices: the outcome and explanatory variables, the functional form, the loss minimised to fit, the validation strategy, and the intended use. Once a problem is in regression form, the analyst no longer reasons about it from scratch but tracks those few declared choices and reads the analysis off a shared, portable diagnostic kit — residual plots, leverage and influence, collinearity, cross-validation — that applies identically whether the data are test scores, disease incidence, or SNP dosages. The functional-form commitment further collapses the open question "what relationship holds?" into a bounded one: departures from the assumed form surface as patterned residuals, so a single set of checks suffices to interrogate any fit. The use-case discipline supplies the branch structure that keeps the compression honest — the same fitted model is read three structurally distinct ways (prediction, effect estimation, variance attribution), each gated by its own validity conditions, with effect estimation alone requiring identification assumptions external to the fit. The analyst thus tracks four declared choices plus which of three uses is in play, and reads the qualitative verdict — adequate prediction, warranted causal effect, defensible decomposition, or none of these — off that small set, rather than re-deriving the whole modelling problem for each new substrate. That reduction from a high-dimensional, problem-specific modelling task to a fixed procedure with a few parameters and a shared diagnostic kit is precisely why regression is the workhorse of the field.

Abstract Reasoning

Regression licenses a family of reasoning moves keyed to its decomposition (systematic + noise), its functional-form commitment, and its three-way use discipline.

Diagnostic, from patterned residuals. The central inference reads the residuals as evidence about what the fit got wrong. Because the model posits a specific noise distribution and a specific functional form, any structure left in the residuals is a signature of mis-specification: curvature in a residual-versus-fitted plot says the assumed form is too rigid (a linear fit on a log-linear relationship); a fan shape says the homoscedasticity assumption is violated; serial correlation in the residuals says the independence assumption fails. The analyst reasons from the shape of what the systematic part failed to absorb back to which assumption is broken, and predicts the direction the inference is therefore biased — narrowed standard errors under positive autocorrelation, understated uncertainty under unmodelled clustering. Leverage and influence diagnostics extend this: a single high-leverage point that swings a coefficient is inferred from the fit's sensitivity, not from the raw data alone.

Interventionist on bias, via omitted-variable reasoning. Regression's signature analytic move is reasoning about a coefficient that would change if the model were specified differently — even when the relevant variable is unmeasured. From the correlation between an included predictor and an omitted causal variable, the analyst predicts both the direction and rough magnitude of the bias on the included coefficient (positive confounder positively correlated with the predictor inflates it), and can therefore say which way the estimate is wrong before ever measuring the confounder. The companion move is the bias-variance tradeoff, most legible here: adding terms or interactions reduces bias at the cost of variance and, absent a penalty, overfitting — so the analyst predicts that a more flexible form will fit the sample better while generalizing worse, and reaches for ridge/lasso to trade a little bias for a large variance reduction. The licensed interventions — add a control, change the form, regularize, instrument — each come with a predicted effect on the estimate.

Boundary-drawing, on which warrant a use requires. The sharpest reasoning the concept enforces is gating a claim by its use. A coefficient may be read three ways, and the boundary between them is a boundary between validity conditions: as prediction it needs only out-of-sample adequacy; as a causal effect it needs identification assumptions — no omitted confounders, no reverse causation, no selection on the outcome, correct form — that live outside the fitting procedure and cannot be supplied by the fit, however good; as variance attribution it needs the decomposition to track the underlying structure. The move is: before interpreting any number, ask which use is in play, then check only that use's conditions, and refuse to read a predictively adequate coefficient as a vetted causal estimate. Crossing from prediction to causation without the external warrant is the canonical regression error, and the use discipline draws exactly that line.

Predictive, on in-sample optimism and order of validation. The concept specifies that in-sample fit is an optimistic lower bound on both predictive performance and assumption adequacy, so the analyst predicts that hold-out performance will fall short of training fit and that the gap widens with model flexibility — making cross-validated error, not R-squared, the quantity to read. The order of operations follows: commit to form and loss, fit, then interrogate residuals and out-of-sample behavior, because the diagnostics are only interpretable once the form is stated and the fit has been pushed against data it did not see.

Knowledge Transfer

Regression is a method, not a causal mechanism in the world, so the "mechanism within the home domain / metaphor beyond it" framing does not apply to it the way it applies to a structural pattern. It is an instrument — case (C) — and the right thing to characterize is the reach of the construct itself. Within statistics and data science the apparatus transfers across its whole family without translation: ordinary least squares, logistic, Poisson, quantile, ridge, lasso, generalised additive, mixed-effects, and hierarchical Bayesian models all share the systematic-plus-noise decomposition, the functional-form commitment, the loss-minimised fit, and the prediction/effect-estimation/variance-attribution use discipline, varying only in link function, noise model, and penalty. The portable diagnostic kit — residual and leverage plots, collinearity checks, cross-validation — and the omitted-variable-bias and bias-variance reasoning travel with the apparatus intact.

The crucial honesty is that beyond statistics the cross-domain movement of regression is literal instrument transfer, not metaphor and not independent structural recurrence. When regression appears in econometrics, epidemiology (Cox, Poisson, logistic), social science, operations and budgeting, quality engineering (response-surface analysis), machine learning, and genetics (GWAS as phenotype-on-SNP-dosage regression with population-structure correction), the same machinery is being deliberately embedded into a new substrate — the construct applies wherever its precondition holds, namely data with an outcome to be decomposed into a modelled systematic part and a specified noise distribution. These are venues that re-use one inferential apparatus, not domains that independently reinvented a shared skeleton, so the right boundary to mark is not analogy-versus-mechanism but instrument-reach versus over-reading. Two over-readings are the failure modes. The first is mistaking method transfer for a law of the world: that regression "recurs" across budgeting and disease and genotypes says something about the apparatus's generality, not about a substrate-neutral structural pattern in those substrates. The second — the field's canonical error — is reading a fitted coefficient beyond the warrant of the use in play: a predictively adequate coefficient is a causal effect only under identification assumptions (no omitted confounders, no reverse causation, no selection on the outcome, correct form) that live outside the fitting procedure and that the regression itself cannot supply, however good the fit. The genuinely substrate-independent structural content that regression embodies — systematic part plus noise, model the systematic part as a function, estimate it from data under a loss, use the fit for prediction or attribution — does recur across domains, but it does so at the level of the parent primes regression instantiates: function, statistical_inference, model_based_inference, signal_extraction (separating systematic from noise), and causal_inference (when coefficients are read as effects), with distributional_assumption carrying the noise-model commitment. So the cross-domain reach belongs to those parents; "regression," as named, is the methodological embodiment — an instrument that transfers literally wherever its precondition holds, whose boundary to police is over-reading, not whose boundary is metaphor (see Structural Core vs. Domain Accent).

Examples

Canonical

Ordinary least squares is the defining construction (the least-squares criterion goes back to Legendre in 1805 and Gauss). Fit a straight line ŷ = a + bx to four points: (1, 2), (2, 2), (3, 4), (4, 5). With means x̄ = 2.5 and ȳ = 3.25, the slope is b = Σ(x−x̄)(y−ȳ) / Σ(x−x̄)² = 5.5 / 5.0 = 1.1, and the intercept is a = ȳ − b·x̄ = 3.25 − 1.1(2.5) = 0.5, so ŷ = 0.5 + 1.1x. The fitted values are 1.6, 2.7, 3.8, 4.9, leaving residuals 0.4, −0.7, 0.2, 0.1 that sum to zero — the signature of a least-squares fit. The line is the systematic part; the residuals are the estimated noise, and their pattern is what one inspects to test whether the linear form and constant-variance assumptions hold.

Mapped back: y is the outcome variable split into a systematic line plus residual; x is the explanatory variable; the straight line is the functional form; and minimizing squared residuals is the loss. Treating the residuals as mean-zero scatter is the noise model, and inspecting them for pattern is the seed of the validation strategy. With only four points, b = 1.1 is a description, not a warranted causal effect — the identification boundary.

Applied / In Practice

The Framingham cardiovascular risk functions are regression doing clinical work. Using decades of follow-up from the Framingham Heart Study cohort, statisticians fit regression models (logistic and Cox proportional-hazards forms) predicting the probability of a cardiovascular event over a time horizon from a handful of predictors: age, sex, total and HDL cholesterol, systolic blood pressure, smoking, and diabetes. The fitted coefficients become weights in a published risk score that clinicians use to estimate an individual patient's 10-year heart-disease risk and decide on statins or blood-pressure treatment. The model is validated by out-of-sample discrimination (does it rank higher-risk patients above lower-risk ones?) and calibration (do predicted risks match observed rates?), and is periodically refit and recalibrated for new populations where its performance drifts.

Mapped back: A cardiovascular event is the outcome variable; age, cholesterol, blood pressure, and the rest are the explanatory variables; the logistic/Cox model is the functional form and noise model. Discrimination and calibration on new cohorts are the validation strategy. Critically, the score is used for prediction under the use-case discipline — its coefficients are predictive weights, not clean causal effects, respecting the identification boundary.

Structural Tensions

T1: Predictive adequacy versus causal warrant (the fit that reads like an effect). The same fitted coefficient supports three uses gated by different conditions, and the most consequential gap is between prediction (needs only out-of-sample adequacy) and effect estimation (needs identification assumptions — no omitted confounders, no reverse causation, correct form — that live outside the fit). The tension is that a coefficient produced by an excellent predictive model is quantified, signed, and sitting right there, looking exactly like an effect size, so the fit's very success at prediction is what tempts the canonical error of reading it causally. The quality of the fit is orthogonal to the causal warrant — no amount of predictive accuracy or residual health can supply the external identification the regression cannot generate — yet the better the fit, the more authoritative the illegitimate causal reading appears. Diagnostic: Is this coefficient being used as a predictive weight (its warrant) or as a causal effect (requiring external identification the fit cannot supply, however good it is)?

T2: The functional form as lens versus blinder (commitment both reveals and limits). Committing to a functional form before fitting is what makes the whole diagnostic apparatus work: because the form and noise model are posited, departures from them show up as patterned residuals, so mis-specification becomes visible. But the same commitment constrains what the model can express, and a wrong-but-adequate form can fit acceptably in-sample while systematically distorting the relationship — and the residual diagnostics that would catch this themselves presuppose the form is close enough for residuals to be interpretable. The tension is that the form is simultaneously the lens that renders mis-specification legible and the blinder that bounds what the model can see, so the very structure enabling the checks is a structure the checks partly take for granted. A form badly enough wrong can make its own residuals uninformative. Diagnostic: Are the residuals being read as a genuine test of the assumed form, or is the form wrong enough that the diagnostics built on it can no longer reveal the distortion?

T3: Fidelity versus generalization (the bias-variance trade is unavoidable). The analyst wants both to capture the true relationship (low bias, achieved by a flexible form with more terms and interactions) and to generalize to new data (low variance, achieved by a simpler form). These pull against each other structurally: added flexibility reduces bias but raises variance and, absent a penalty, overfits — fitting the sample better while generalizing worse. Regularization (ridge, lasso) resolves it only by deliberately introducing bias to buy a large variance reduction, i.e., accepting a knowingly-shrunk, knowingly-wrong estimate for the sake of stability. The tension is that there is no form that is both maximally faithful and maximally generalizable; every complexity choice spends one to buy the other, and the "best" model is a negotiated point on that trade, not an attainable ideal that is both. Diagnostic: Is the model's flexibility set to minimize out-of-sample error (the operative goal), or tuned to in-sample fidelity in a way that trades away the generalization that matters?

T4: In-sample fit versus out-of-sample truth (the reported number is the optimistic one). In-sample fit (R-squared) is the cheap, always-computed, reflexively-reported quantity — and it is an optimistic lower bound on both predictive performance and assumption adequacy, with the gap widening as the model grows more flexible. The honest quantity, cross-validated out-of-sample error, requires holding data out (less data to fit, more work) and is less flattering. The tension is that the metric most readily produced and quoted systematically overstates how good the model is, precisely in proportion to how much flexibility was used to inflate it, so the number that advertises the fit is the number least entitled to be trusted. Reading R-squared as performance rewards exactly the overfitting that widens the in-sample/out-of-sample gap. Diagnostic: Is the model being judged by in-sample fit (an optimistic bound that flexibility inflates) or by cross-validated out-of-sample error (the quantity that actually bears on performance)?

T5: Autonomy versus reduction (statistical instrument or the composition of function, inference, and causal warrant). Regression is an instrument, not a causal mechanism, so it transfers literally wherever its precondition holds — data with an outcome decomposable into a modelled systematic part and a specified noise distribution — and across its whole family (OLS, logistic, Poisson, quantile, ridge, lasso, GAM, mixed-effects, hierarchical Bayesian) and every venue (econometrics, epidemiology, GWAS, budgeting) the same apparatus is deliberately re-embedded, not independently reinvented. But the substrate-neutral structural content it embodies — systematic part plus noise; model the systematic part as a function; estimate from data under a loss; use the fit for prediction or attribution — lives at the level of the parents it instantiates: function, statistical_inference, model_based_inference, signal_extraction (systematic from noise), distributional_assumption (the noise commitment), and causal_inference (when coefficients are read as effects). The tension is that regression's cross-domain reach is instrument-reach, not evidence of a substrate-neutral law recurring in those fields, so the boundary to police is over-reading (method generality mistaken for a worldly pattern; a coefficient read past its use) rather than metaphor — and note the pure name-collision with regression to the mean. Diagnostic: Resolve toward the parents (function, statistical_inference, causal_inference, signal_extraction) for the substrate-neutral structural content; reserve named regression for the statistical instrument, policing over-reading (method reach ≠ worldly law; coefficient ≠ effect without external warrant) rather than metaphor.

Structural–Framed Character

Regression is an unusually structural domain-specific entry — best read as mixed-structural, near the structural end but held short of a prime because it is a methodological instrument rather than a worldly mechanism or a bare relational form. It scores structural on three of the five criteria. Its evaluative_weight is nil: "regression" convicts and praises nothing — it names a fitting procedure, not a defect or a virtue, and the analysis is thoroughly neutral. On vocab_travels it patterns structural in a way most DS entries do not: the operative vocabulary — outcome and explanatory variables, functional form, loss, noise model, residuals, cross-validation — does not stay pinned to a home substrate but applies with full content wherever the precondition holds (data with an outcome decomposable into a modelled systematic part and a specified noise distribution), reading identically over test scores, disease incidence, or SNP dosages. On import_vs_recognize the entry is emphatic and the profile is again structural-leaning: cross-domain movement is literal instrument transfer — the same apparatus deliberately re-embedded in a new substrate — "not metaphor and not independent structural recurrence," so the boundary to police is over-reading rather than analogy. These three marks put it well toward structure.

What keeps it mixed-structural rather than at the pole (or a prime) is the remaining two criteria. On human_practice_bound it patterns framed in the instrument sense: regression does not run in nature observer-free the way a buoyant column or a memory curve does — there is no "regression" happening in the world without an analyst who commits to a form, chooses a loss, and fits parameters; it is an analytical procedure, a thing done, not a process nature performs. On institutional_origin it is a mixed case tilting framed: the least-squares core is a discovered mathematical fact (Legendre, Gauss), but "regression" as delivered — the generalised-linear-model family, the identification apparatus, the diagnostic kit, the three-use discipline — is the worked instrument of statistical methodology, an artifact of that discipline's development rather than a bare form nature marks.

The portable structural skeleton is model an outcome as a systematic function of inputs plus specified noise, estimate it from data under a loss, and use the fit for prediction or attribution. That skeleton is genuinely substrate-neutral, and here it legitimately resolves into several parents because the entry decomposes it explicitly: function (the systematic part), statistical_inference and model_based_inference (estimation under a loss), signal_extraction (separating systematic from noise), distributional_assumption (the noise-model commitment), and causal_inference (when coefficients are read as effects). That parent set is exactly what the substrate-neutral content is — and it is what regression instantiates, not what makes "regression" itself a prime: the cross-domain reach belongs to those parents, while "regression," as named, is the methodological embodiment that carries statistical-craft cargo (the GLM family apparatus, the identification boundary, the use-case discipline) held together as an instrument. Its character: an evaluatively neutral, literally-transferable statistical instrument whose vocabulary travels intact across substrates, structural in the function-plus-noise-plus-loss skeleton it composes from its inference parents, but a discipline's method rather than a worldly mechanism — mixed-structural, not a free-floating prime.

Structural Core vs. Domain Accent

This section decides why regression is a domain-specific abstraction and not a prime, and it carries the case for its domain-specificity — no other section does. The case is atypical: regression is an instrument, not a worldly mechanism, so its cross-domain movement is literal method transfer rather than metaphor, and the not-a-prime verdict turns not on limited reach but on the statistical-craft cargo bundled into the named procedure.

What is skeletal (could lift toward a cross-domain prime). Strip the statistical craft away and a thin relational structure survives: model an outcome as a systematic function of inputs plus specified noise, estimate that function from data under a loss, and use the fit for prediction or attribution. The portable pieces are abstract — a systematic-plus-stochastic decomposition, a committed functional form, a data-driven estimate under an explicit criterion, and a use to which the fit is put. That skeleton is genuinely substrate-neutral, and unusually it does not resolve into a single parent but a composite the entry names explicitly: function (the systematic part), statistical_inference and model_based_inference (estimation under a loss), signal_extraction (separating systematic from noise), distributional_assumption (the noise-model commitment), and causal_inference (when coefficients are read as effects). This parent set is the core it shares — the genuinely substrate-neutral content — not what makes "regression" distinctive.

What is domain-bound. What is specific to regression is not the skeleton but the worked statistical-methodology apparatus that implements and disciplines it. The generalised-linear-model family (OLS, logistic, Poisson, quantile, ridge, lasso, GAM, mixed-effects, hierarchical Bayesian) with its link functions, noise models, and penalty structures; the portable diagnostic kit (residual and leverage plots, collinearity checks, cross-validation); the omitted-variable-bias and bias-variance reasoning; the identification boundary and the external-warrant machinery (instrumental variables, panel and difference-in-differences methods) layered on the fit; and the three-use discipline (prediction / effect estimation / variance attribution, each with its own validity conditions) — these are the craft furniture of statistics and data science. The decisive test is unusual here: because regression is a method, it does not "become a looser thing" when carried to a new field — it transfers literally, the same apparatus deliberately re-embedded wherever its precondition holds (an outcome decomposable into a modelled systematic part and a specified noise distribution). What stays home is not reach but the discipline's constructed instrument itself: the GLM apparatus, the identification jurisprudence, and the use-case discipline are things an analyst does, not a process the world runs observer-free.

Why this does not clear the prime bar. A prime is a relational structure whose vocabulary travels and whose cross-domain transfer is recognition of the same mechanism, not analogy. Regression's transfer is neither analogy nor independent structural recurrence but literal instrument transfer, and that is precisely why the named construct is not a prime: what genuinely recurs across econometrics, epidemiology, GWAS, budgeting, and the rest is the substrate-neutral content carried by the parents, while "regression" is the methodological embodiment that re-uses one inferential apparatus in each venue. Two over-readings mark the boundary the concept must police: mistaking the method's generality for a law of the world (that regression "recurs" everywhere is a fact about the instrument's reach, not a substrate-neutral pattern in those fields), and reading a coefficient past its use (a predictively adequate coefficient is a causal effect only under identification assumptions external to the fit). When the bare structural lesson — systematic part plus noise, modelled as a function, estimated under a loss, used for prediction or attribution — is genuinely wanted cross-domain, it is already carried, in more general form, by the parents regression instantiates: function, statistical_inference, model_based_inference, signal_extraction, distributional_assumption, and causal_inference. The cross-domain reach belongs to those parents; "regression," as named, keeps the GLM-family apparatus, the diagnostic kit, the identification boundary, and the three-use discipline — the statistical-craft cargo that should stay home. (Note too the pure name-collision with regression to the mean, an unrelated reversion effect that shares only the word.)

Relationships to Other Abstractions

Current abstraction Regression Domain-specific

Parents (4) — more general patterns this builds on

• Regression is a kind of Signal Extraction Prime

  Regression is signal extraction specialized to recovering a systematic input-output relationship from an outcome entangled with stochastic residual variation.

• Regression is a kind of Statistical Inference Prime

  Regression is statistical inference specialized to estimating a systematic outcome function under an explicit stochastic model and loss.

• Regression is part of Distributional Assumption Prime

  Regression contains a distributional assumption as the internal commitment that specifies its stochastic outcome or residual component.

• Regression is part of Function (Mapping) Prime

  Regression contains a function mapping as its systematic component from explanatory-variable inputs to an outcome value or distribution parameter.

Children (2) — more specific cases that build on this

• Collinearity Inflation Domain-specific presupposes Regression

  Collinearity inflation presupposes a regression design matrix whose near-linear dependence inflates coefficient variance.

• Endogeneity Domain-specific presupposes Regression

  Endogeneity presupposes a regression-style model whose regressor, error term, and exogeneity assumption make the violation definable.

Hierarchy paths (13) — routes to 7 parentless roots

• Regression → Signal Extraction

Show alternative paths (12)

Not to Be Confused With

• Regression to the mean. A different phenomenon that shares only the word: extreme measurements drift toward the average on remeasurement, a property of the world (imperfect correlation between paired measurements). Regression the method fits an outcome as a function of predictors and estimates parameters under a loss; nothing about the reversion effect is involved. Tell: is a fitted model with a functional form and noise distribution in play, or a bare tendency of outliers to be less extreme next time?
• Regression (the psychoanalytic defense mechanism). A third homonym from psychology: reverting to an earlier developmental stage under stress. It has no relation to the statistical construct beyond the borrowed Latin root. Tell: is the subject fitting data, or a person's behaviour retreating to a younger pattern?
• Classification. The sibling supervised-learning task for discrete outcomes (labels, categories). Regression here targets a continuous outcome; the entry is explicit that the boundary is task-defined, not structural — the same apparatus, a different response type. Tell: is the modelled outcome a number on a continuum, or membership in a category?
• Curve fitting / interpolation. Passing a function through points to describe or interpolate a relationship, with no explicitly specified noise model, no inferential loss commitment, and no prediction/effect/attribution use discipline. Regression adds the systematic-plus-stochastic decomposition and the validity conditions that let one infer rather than merely draw. Tell: is a noise distribution posited and residuals interrogated, or is a smooth line simply threaded through the data?
• ANOVA (analysis of variance). A neighbour that partitions outcome variance among grouped factors — overlapping regression's variance-attribution use and expressible as a regression on categorical predictors. But ANOVA is one specialization keyed to group comparisons, not the general outcome-on-predictors modelling family with its three-way use discipline. Tell: is the question a variance decomposition across discrete groups, or a general fit supporting prediction and effect estimation as well?
• The parent primes it instantiates (function, statistical_inference, model_based_inference, signal_extraction, distributional_assumption, causal_inference). The substrate-neutral content — model a systematic part as a function of inputs plus specified noise, estimate under a loss, use the fit for prediction or attribution — that genuinely recurs across domains. Those parents, not "regression," carry the cross-domain lesson; the named construct is the statistical instrument that re-embeds one apparatus in each venue. Tell: strip the GLM family, diagnostic kit, and use discipline, and the function-plus-noise-plus-loss skeleton is what remains — treated fully in a later section.

Neighborhood in Abstraction Space

Regression sits in a moderately populated region (58^th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Unclustered & Miscellaneous (309 abstractions)

Nearest neighbors

• Underfitting — 0.85
• Label Shift — 0.85
• Distributional Blind Spot — 0.84
• Attenuation Bias — 0.84
• Prior-Probability Shift — 0.83

Computed from structural-signature embeddings · 2026-07-12