Neural Networks and the Bias/Variance Dilemma.

Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural Networks and the Bias/Variance Dilemma. Neural Computation, 4(1), 1-58.

Type

Journal article

Intellectual base

Primary research

Year

1992

DOI

10.1162/neco.1992.4.1.1

Link

https://doi.org/10.1162/neco.1992.4.1.1

Cited from

economics_finance, ethnography_qualitative_methods, mathematics, physics, statistics_experimental_design

Cited by

6 citations across 6 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Bias
  • It separates two components of total error (the signed, reproducible part and the unsigned, averaging part) and names the part that carries a direction, a decomposition Geman, Bienenstock, and Doursat (1992) made canonical in their analysis of the bias/variance dilemma for learning systems.

    This sourceCanonical decomposition of a learning system's total error into bias and variance components; grounds the cross-domain inference that aggregation can rescue an unbiased noisy process but never a biased one, recurring in polling, ensembles, sensor fusion, and forecasting.

• Conjugate-Observable Complementarity
  • In machine learning, the fit–generalization (bias–variance) trade-off is a conjugate pair on one model: driving training error toward zero (sharp fit) inflates variance and degrades generalization, and the two cannot both be made arbitrarily small on unknown data — regularization chooses a frontier point rather than abolishing the floor.

    This sourceFormalizes the bias–variance decomposition as a conjugate trade-off in statistical learning, with a floored expected error.

• Grain of Analysis
  • Qualitative coding: the coding grain (sentence, paragraph, episode, theme) determines whether patterns survive synthesis; overcoding fragments meaning, undercoding loses warranted distinctions. Quantitative model fitting: model complexity sets the grain at which a model resolves the data; overfitting resolves below the signal level, underfitting above it, and the bias-variance trade-off is the standard framing.
• No Free Lunch Theorem
  • NFL is a conservation law over the space of problems constraining how any procedure performs averaged across all instances — a meta-claim about methods, not a method. Not the bias-variance trade-off alone. See bias: that trade-off is the special case of NFL for estimation under squared error.

    This sourceStandard reference for the bias-variance trade-off, the estimation-under-squared-error special case of the no-free-lunch conservation result.

• Overfitting
  • The essential commitment is relational: overfitting is not a property of the model alone or the data alone, but of the model-data-target relationship, diagnosed by the gap between in-sample and out-of-sample performance

    This sourceCanonical decomposition of a learning system's total error into bias and variance components; grounds the cross-domain inference that aggregation can rescue an unbiased noisy process but never a biased one, recurring in polling, ensembles, sensor fusion, and forecasting.

• Trade-offs
  • Computer science and information systems

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Registry ID ref:9ef11aedda7f · see in the full table