Evolutionary Trap

Version

v1 · 2026-08-24 · History

Prime #

846

Origin domain

Biology & Ecology

Subdomain

behavioral ecology → Biology & Ecology

Core Idea

An agent follows a once-reliable cue more eagerly the stronger it is, straight into harm, because the environment changed and the cue-value coupling broke while the fast cue-response did not — the environment-driven sibling of measure-as-target collapse, with the invariant cue-update rate < environment-change rate.

How would you explain it like I'm…

Moth And The Lamp

A moth uses the moon to fly straight at night, and that always worked. Then people made bright lamps, and now the moth flies right into the lamp because it follows the light the same way it followed the moon. The thing it learned to trust used to help it, but the world changed and now that same trick hurts it.

Old Trick, New World

An Evolutionary Trap happens when an animal, person, or system follows a signal that USED to lead to something good, but the world changed and the signal no longer means what it used to. The trouble is that they follow the signal even harder when it's stronger — straight into harm or wasted effort. It's not that they're dumb; their behavior was a great fit for the OLD world, and they have no way to notice the signal stopped being trustworthy. The trap needs a fast, deep habit of reacting to the cue, plus a slow or missing way to update what the cue actually means, plus a change in the world that broke the old connection.

Stale Cue, Broken Link

An Evolutionary Trap is when an agent — organism, person, institution, or learning system — keeps using a cue that was historically tied to a good outcome, but the environment changes so the cue persists while its link to value breaks or reverses. The agent then follows the cue more eagerly the stronger it is, straight into harm or wasted effort. The defining feature isn't stupidity: the cue-following is well-adapted to the old environment, and the agent has no mechanism to detect that the proxy has been decoupled from value. Structurally it needs a fast, deeply-installed cue-to-action coupling, a slow or absent update of the cue's meaning, an environmental change that breaks the cue-to-value correlation, and a cost that makes the now-mistaken response harmful. It's the environment-driven sibling of measure-as-target collapse, which is agent-driven — there the agent games a measure; here the world moves while the agent and proxy stay put.

 

An Evolutionary Trap arises when an agent — organism, person, institution, learning system — uses a cue that was historically correlated with a beneficial outcome to guide its behavior, then the environment changes: the cue persists, but its correlation with the underlying value breaks or reverses. The agent now follows the cue more eagerly the stronger it is, straight into harm or wasted effort. The trap is built from a stale proxy with a fast cue-response and a slow update of the cue's meaning. Its defining feature is not that the agent is stupid; it is that the cue-following behavior is well-adapted to the old environment and the agent has no mechanism to detect that the proxy has been decoupled from value. The load-bearing structure requires four pieces: a fast, deeply-installed cue-to-action coupling (instinct, habit, hard-coded policy, calibrated controller); a slow or absent update mechanism for what the cue means; an environmental change that breaks or reverses the cue-to-value correlation; and a cost asymmetry that makes the now-mistaken response harmful. The precise mismatch is cue update rate < environment change rate, with a tight cue-to-action loop in between. This is one realization of a broader proxy-drift pattern, but the trap version is specifically environment-driven: the agent and proxy don't change, the world does. It is the sibling of measure-as-target collapse, which is agent-driven (the agent games a measure made a target); the two differ in mechanism and intervention even though both produce proxy failure. The evolutionary-ecology phrasing travels with the prime, but the underlying pattern — stale proxy plus fast cue-response plus slow update — is substrate-neutral.

Broad Use

• Behavioral ecology: sea-turtle hatchlings crawling toward beachfront lights, mayflies laying eggs on glossy asphalt.
• Behavioral economics: hyper-palatable foods and smartphone notifications exploiting cues that once marked scarce value.
• Machine learning: a policy climbing a learned proxy reward, or a classifier learning the scanner brand instead of the disease.
• Engineering safety: a controller tuned to a test-condition sensor turning pathological in deployment.
• Institutional systems: hospitals optimizing patient satisfaction at the expense of clinical outcomes.
• Public health: vaccine-fear cues outweighing diffuse benefit cues; satisfaction-driven antibiotic over-prescription.

Clarity

Separates agent stupidity from environmental novelty plus proxy decoupling — the agent is doing exactly what worked — and distinguishes incidental decoupling (a hotel light) from active manipulation (dark patterns, reward hacking).

Manages Complexity

Compresses a wide class of failures into a four-question triage: what cue, what it historically tracked, whether that correlation still holds, and how fast the agent updates versus how fast the environment changed.

Abstract Reasoning

Exposes the rate-mismatch and yields the non-obvious prediction that better optimization on the cue makes the failure worse — a stronger agent tracks the now-misleading cue more reliably and fails harder.

Knowledge Transfer

• Across substrates: three archetypes ride along — restore the proxy, speed up the update, add a sanity check.
• Conservation ↔ ML safety: a hatchling toward hotel lights and an RL agent climbing a stale reward are one failure (amber lighting ↔ reward shaping; hatchery release ↔ human feedback).
• ML ↔ behavioral design: distribution-shift mitigation and online learning mirror friction and reflection prompts.

Example

A sea-turtle hatchling's phototaxis — "crawl toward the brightest horizon" — reliably found the ocean for millions of years; beachfront lighting reverses the cue in decades while the response updates over generations, so the most reliably cue-tracking hatchlings fare worst.

Relationships to Other Abstractions

Current abstraction Evolutionary Trap Prime

Parents (2) — more general patterns this builds on

• Evolutionary Trap is a kind of Cue Outcome Decoupling Prime

  Evolutionary trap is cue-outcome decoupling specialized to an environment-driven coupling break outrunning a deeply installed rule's update rate.

• Evolutionary Trap is a kind of, typical Natural Selection Prime

  Evolutionary Trap is typically a specialization of Natural Selection, retaining the parent's defining structure while adding the child's specific commitments.

Children (1) — more specific cases that build on this

• Ecological Trap Domain-specific is a kind of Evolutionary Trap

  An ecological trap is evolutionary trap specialized to an organism's evolved habitat, mate, or oviposition decision and a fitness-valued outcome.

Hierarchy paths (2) — routes to 2 parentless roots

• Evolutionary Trap → Cue Outcome Decoupling → Proxy–Target Fidelity → Representation → Abstraction

Show alternative path (1)

Not to Be Confused With

• Evolutionary Trap is not Campbell's Law (measure-as-target collapse) because the trap is environment-driven — the world moves while the agent and proxy stay fixed — whereas Campbell's law is agent-driven: an agent games a staked measure.
• Evolutionary Trap is not Coevolution because it is a one-sided rate mismatch where the agent cannot update fast enough, whereas coevolution is reciprocal adaptive change between interacting parties.
• Evolutionary Trap is not Concept Drift because it is the behavioral failure a fast-cue/slow-update loop produces, whereas concept drift is merely the statistical fact that a distribution moved.