Skip to content

Chaos Exposure Testing

Intentionally introduce controlled disruption to reveal weaknesses before uncontrolled chaos exposes them.

The Diagnostic Story

Symptom: The system appears stable, but that stability has never been tested under disruption. Backups and recovery procedures exist on paper; dependency maps were drawn once and not revisited; runbooks describe steps that no one has actually run. Postmortems keep turning up surprises that, in hindsight, were entirely predictable—and stress tests get avoided because the current calm feels reassuring.

Pivot: Form a specific fragility hypothesis, select a bounded perturbation that targets it, constrain the blast radius, instrument the system's response, run the exposure under authorization, and stop or roll back when needed. Observations feed directly into repairs and repeated validation.

Resolution: Hidden dependencies and brittle recovery paths become visible while they can still be fixed, rather than during a real incident. The system accumulates verified evidence of behavior under disruption instead of untested optimistic assumptions.

Reach for this when you hear…

[SRE] “Our runbook says to promote the replica in under five minutes, but nobody has actually done it since we wrote it, and the replica config has changed three times since then.”

[hospital operations] “We think we can reroute patients to the backup wing if the main OR goes down, but we've never walked through it and I'm pretty sure the staffing assumptions are wrong.”

[supply chain] “We listed three alternate suppliers in our continuity plan, but we don't know if any of them can actually absorb our volume on short notice.”

Mechanisms / Implementations

  • Failure Injection: Deliberately disables, delays, corrupts, or degrades a component.
  • Chaos Engineering Experiment: A chaos engineering experiment is a technical mechanism, often used in software and infrastructure, for validating resilience assumptions under realistic conditions.
  • Game Day Exercise: A game day exercise rehearses disruption with operators and dependent teams.
  • Tabletop Exercise: A tabletop exercise simulates a disruption through guided discussion.
  • Fire Drill: A fire drill practices emergency behavior under bounded conditions.
  • Red-Team Stress Test: A red-team stress test uses an adversarial or independent actor to probe weaknesses.
  • Disaster Exercise: A disaster exercise coordinates many actors around a major disruption scenario.
  • Canary Perturbation: A canary perturbation introduces disruption to a small monitored slice first.
  • Runbook Rehearsal: A runbook rehearsal executes documented procedures under simulated or bounded stress.
  • Observability Dashboard: An observability dashboard displays system response during exposure.

Abstractions this archetype builds on — directly (a source ingredient) or as a related pattern. Links follow the typed catalog namespace.

Built directly on (3)

Also references 12 related abstractions

Variants

Narrower or domain-specific specializations that share this archetype's core structure. Recognized variants are established; candidate variants are provisional.

Live-System Perturbation Test · domain variant · recognized

Introduce a bounded disruption into a live or production-like system to test whether actual operating conditions match resilience assumptions.

Rehearsal Drill Exposure · implementation variant · recognized

Expose people, procedures, and coordination routines to a rehearsed disruption so readiness gaps appear before a real incident.

Adversarial Stress Probe · risk or failure variant · candidate

Use an adversarial or independent actor to probe weaknesses that ordinary operators, planners, or designers tend to miss.