Underspecification¶
Core Idea¶
A criterion is treated as picking out a single answer when in fact an equivalence class of answers satisfies it equally well; a hidden closure (a seed, an optimisation trajectory, a default, a prior) silently picks one representative, so the load-bearing behavior may be governed entirely by the closure rather than the constraint.
How would you explain it like I'm…
Many Answers, One Clue
Imagine a clue says 'I'm thinking of an animal with four legs.' That clue fits a dog, a cat, a horse — lots of animals! The clue can't tell you which one, but you still have to guess one. Something secret you didn't notice ends up choosing for you, so two people can follow the very same clue and end up picking different animals.
The Clue That Doesn't Decide
Underspecification is when a rule or set of clues seems to point at one answer, but really lots of different answers fit it equally well. The clues don't pin down the choice — yet a choice gets made anyway, by hidden things like a random starting point, the default settings, or which tool you happened to use. Because the rule was satisfied by a whole group of answers, something outside the rule quietly picks one. The sharp result: two systems built to the exact same rules can act differently when you finally test the part the rules never nailed down. Nothing was holding that part in place.
Constraint Versus Hidden Closure
Underspecification is the pattern where a specification process treats observed evidence as if it picked out a single answer, when in fact many distinct answers fit that evidence equally well. The evidence underdetermines the choice — but the choice still gets made, by hidden factors (a random seed, an optimization path, default settings, the analyst's prior, the available software) that the explicit criterion doesn't control. So the criterion is satisfied by an entire equivalence class of conforming answers, and something outside it silently picks one. The downstream consequence is sharp: two systems built to the same spec, by the same rules, behave differently once you test the behavior that distinguishes them — because nothing in the build was holding that behavior in place. The trick is to separate three usually-fused things: the constraint the evidence imposes, the closure (the extra implicit choices that pick one answer), and the load-bearing behavior, which may be governed entirely by the closure. Holding these apart lets you predict which properties are robust (set by the constraint) and which are contingent (set by the closure, and liable to flip).
Underspecification is the structural pattern in which a specification process treats observed evidence as if it picked out a single answer, when in fact many distinct answers fit that evidence equally well. The evidence underdetermines the choice — but the choice gets made anyway, by hidden factors (a random seed, an optimization trajectory, default settings, the analyst's prior, the available software) that the explicit selection criterion does not control. The criterion is satisfied by an equivalence class of conforming answers, and something outside the criterion silently picks one representative from it. The downstream consequence is sharp: two systems built to the same specification, by the same rules, will behave differently when the behavior that distinguishes them is finally tested in the field. Because the selection criterion did not constrain that behavior, nothing in the build process was holding it in place. The essential commitment is to separate three things ordinarily fused. There is the constraint the evidence or specification imposes; there is the closure — the additional, often implicit, choices that pick a single answer from the constrained set; and there is the load-bearing behavior, which may be governed entirely by the closure rather than the constraint. Holding these apart makes it possible to predict which properties of a system are robust — controlled by the constraint and so invariant across admissible choices — and which are contingent — controlled by the closure and so liable to flip when the closure changes. The mistake the pattern names is treating a solution as if it were the solution when the criterion admits an equivalence class.
Broad Use¶
- Machine learning: many models with identical validation accuracy encode different decision surfaces that diverge sharply on a stress test.
- Inverse problems and physics: recorded data is consistent with infinitely many internal states (Hadamard ill-posedness), and a regulariser picks one.
- Causal inference: multiple causal graphs imply the same conditional independences, and the data alone cannot adjudicate.
- Compiler and language specs: a standard leaves a behavior unspecified, and two conforming implementations produce different programs.
- Legal interpretation: statutory text is consistent with several readings, and precedent or canon picks one.
- Intelligence analysis: the same indicators are consistent with several adversary-intent hypotheses, and the chosen one is the analyst's default.
Clarity¶
Forces a question success metrics never ask — "under my criterion, what other answers are also acceptable, and how do they differ?" — and exposes the closure as a load-bearing input that ordinarily hides.
Manages Complexity¶
Consolidates model brittleness, compiler-dependent bugs, irreproducible conclusions, and doctrine drift into one move: probe the equivalence class, not just the chosen representative.
Abstract Reasoning¶
Licenses a structural prediction available before any field test: properties controlled by the constraint are robust, those controlled only by the closure are free coordinates liable to flip — identifiable by inspecting where admissible builds disagree.
Knowledge Transfer¶
- ML → physics → law: generating the admissible set is one move — training many networks from different seeds, deriving the family of regularised inversions, or laying out admissible statutory readings.
- Across domains: disclosing the closure (naming the tiebreaker) is identical reasoning everywhere.
Example¶
A team trains a classifier selected on validation accuracy; many models differing only in seed achieve identical accuracy yet diverge under a subgroup stress test the criterion never constrained — so the shipped behavior was a free coordinate, not a determined one.
Relationships to Other Primes¶
Parents (1) — more general patterns this builds on
- Underspecification is a kind of, typical Inductive Reasoning — Underspecification is the specific case where an inductive inference is UNDERDETERMINED — many generalizations fit the particulars equally and a hidden closure picks one. The file frames it as generalizing theory-underdetermined-by-data into selection pipelines. is-a inductive_reasoning specialized to a criterion that admits an equivalence class.
Path to root: Underspecification → Inductive Reasoning
Not to Be Confused With¶
- Underspecification is not Overfitting because overfitting is a fit too tight to noise with one determined solution, whereas underspecification is a fit too loose to signal with many equally-good solutions and a hidden tiebreaker — the remedies are opposite.
- Underspecification is not Confirmation Bias because confirmation bias is a cognitive preference for belief-supporting evidence, whereas underspecification is a structural property of the criterion itself, true regardless of anyone's preferences.
- Underspecification is not Selection Bias because selection bias is a defect in how the sample was drawn, whereas underspecification can occur on a flawless sample whose criterion simply fails to pin a unique solution.