Feature-Flag Experimentation¶
Software tool — instantiates Convex Exposure Gain Design
Wraps each change in a runtime toggle so a new variant reaches only a scoped slice of users and can be ramped up or killed instantly — turning every release into a bounded, reversible bet.
A feature flag is a runtime switch that decouples shipping code from exposing behaviour. Feature-Flag Experimentation uses that switch as a bounded-exposure instrument: a new variant is deployed to production but revealed only to a chosen slice — 1% of traffic, one region, internal staff — and the flag can widen, narrow, or cut that exposure in seconds without a redeploy. Its defining move — as distinct from a canary that injects a fault to test whether guards fire — is that it delivers a genuine change to real users under a dial you keep your hand on: the same toggle that ramps a winner from 1% to everyone also retracts a loser to zero in one click. That makes each release a capped, reversible experiment rather than an all-or-nothing event.
Example¶
A SaaS company is rolling out a rewritten checkout flow. Instead of a big-bang launch, they put the new flow behind a flag: on for internal employees first, then 2% of real shoppers in a single country, watching conversion and error rate side by side with the old flow. At 2% the JavaScript error rate ticks up — but only on one browser. They flip the flag off for that browser in one click while leaving it on everywhere else, ship a fix, and resume the ramp: 2% → 10% → 50% → everyone over about two weeks. The blast radius of that bug was a fraction of one market for a few hours, not the entire customer base — and no rollback deploy was ever needed, because the variant had never been turned on for most users in the first place.
How it works¶
- Decouple deploy from release. Ship the variant dark; the flag, not a deployment, controls who actually sees it.
- Scope by cohort. Target the exposure to a segment — percentage, geography, plan tier, named user list — so only that slice bears the risk.
- Ramp on health, not on calendar. Widen the flag while live metrics stay clean; hold or narrow the moment they don't.
- Kill in place. A bad variant is retracted by flipping one switch — no rollback, no redeploy — so time-to-mitigate is seconds, not a release cycle.
Tuning parameters¶
- Exposure percentage / cohort — how wide the flag opens and to whom; larger slices gather signal faster but widen the blast radius of a bad variant.
- Ramp schedule — fixed steps versus metric-gated auto-promotion; aggressive ramps ship value sooner but shorten the window to catch harm.
- Targeting granularity — whole-population versus per-attribute (browser, region, tier); finer targeting isolates damage but multiplies the flag matrix you must reason about.
- Guardrail thresholds — the live metrics (error rate, latency, conversion) that auto-halt or auto-revert a ramp; tight thresholds trip on noise, loose ones let harm run.
- Flag lifetime — how long a flag lives before cleanup; long-lived flags accumulate as config debt (see below).
When it helps, and when it misleads¶
Its strength is that it converts every release into a bounded, reversible experiment: the downside of a bad change is capped at the exposed slice and undone in seconds, while a good change is ramped to everyone on evidence. It is the delivery half of progressive delivery, and its blast-radius discipline is what keeps continuous experimentation survivable.[n1]
Its failure modes follow from the same mechanism. Flags that are never retired pile up into flag debt — a combinatorial tangle of stale toggles that is itself a new fragility, the opposite of what the tool is for. A flag also only caps the harm it is wired to detect; a variant that degrades something unmeasured — a slow trust erosion, a downstream data problem — ramps out "clean." The classic misuse is treating the flag as a substitute for a decision: leaving a half-finished feature on for a favoured customer indefinitely, or promoting to 100% because the calendar said so rather than because the metric did. The discipline is to pair every flag with a removal date and a metric-based stop rule, and to treat "the dashboard looks fine" as necessary, not sufficient.
How it implements the components¶
bounded_exposure_envelope— the flag's cohort and percentage are the envelope: they fix exactly who is exposed to the variant and how widely.contagion_firewall— because the variant is confined to the flagged slice, a defect cannot propagate to unexposed users; the toggle boundary is the firewall.adaptive_dose_controller— ramping the flag up or down on live health signals is dose control: the exposure "dose" tracks the observed response.
It does not map where the system is fragile in the first place (fragility_surface_map — Red-Team Stress Exercise), set the standing loss limit and stop rule (downside_cap_and_stop_rule — Volatility Budget with Loss Limit), or decide which variants to keep and scale across the portfolio (selection_and_retention_filter — Small-Bet Option Ladder); the flag is the delivery-and-containment tool those mechanisms steer.
Related¶
- Instantiates: Convex Exposure Gain Design — feature flags give the archetype its scoped, instantly-reversible exposure surface.
- Consumes: Volatility Budget with Loss Limit sets the loss limit each ramp must stay inside; the flag enforces the stop that the budget defines.
- Sibling mechanisms: Small-Bet Option Ladder · Canary Perturbation · Chaos Engineering Game Day · Red-Team Stress Exercise · Volatility Budget with Loss Limit · Supplier Stress Rotation · Progressive Overload Protocol · Controlled Burn or Ecological Disturbance · Deliberate Practice with Desirable Difficulty · After-Action Learning Harvest
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: Feature-Flag Experimentation operates as a bounded trial, probe, simulation, or rehearsal that generates evidence from performance because it wraps each change in a runtime toggle so a new variant reaches only a scoped slice of users and can be ramped up or killed instantly — turning every release into a bounded, reversible bet.
Independent corroboration: The frozen evidence defines Feature-Flag Experimentation as 'Wraps each change in a runtime toggle so a new variant reaches only a scoped slice of users and can be ramped up or killed instantly — turning every release into a bounded, reversible bet', so its operative form is Experiment, Test & Rehearsal.
Nearest alternative: Control, Automation & Runtime — Scoped cohort exposure is deliberately varied to learn whether a release works, even though a runtime toggle delivers and kills the treatment.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Specialized
Rationale: Feature-flag exposure control developed in software release and site-reliability practice.
Related originating lineages:
- Statistics & Experimental Design — Randomized controlled exposure and inference on variants materially shape its experimentation use.
Review outcome: Independent reviewer agreement; high confidence.
Notes¶
A feature flag caps exposure, not commitment — it can hold a variant at 5% forever. The convex gain only lands when a separate step banks the winner into the mainline and deletes the loser; until then the flag is a paused bet, not a resolved one. Treating a live flag as a decision already made is how a containment tool quietly becomes permanent config debt.
[n1] Blast radius — the extent of a system affected when a change goes wrong. Progressive-delivery practice deliberately keeps it small (a low exposure percentage) so a bad variant reaches few users before it is detected and cut. Feature flags are the standard way to hold that radius small and adjustable at runtime. ↩