Intermittent Failure Capture¶
Capture evidence during irregular failure episodes so elusive problems can be diagnosed after the episode disappears.
Essence¶
Intermittent Failure Capture is the pattern of making elusive failures diagnosable by preserving evidence while the failure is happening. The target problem is not simply that a system fails. The target problem is that the system fails irregularly, then returns to normal before the relevant state, context, sequence, or witness detail can be inspected.
The archetype works by turning a temporary episode into a durable episode record. A trigger condition opens a capture window; snapshots, traces, logs, and contextual details are preserved; the evidence is protected from ordinary cleanup; and a follow-up diagnostic path turns the record into repair, prevention, or better instrumentation.
Compression statement¶
When failures occur intermittently and vanish before ordinary inspection, define episode triggers, capture windows, state snapshots, contextual traces, evidence preservation rules, recurrence analysis, and follow-up diagnostic paths so rare or transient failures become analyzable rather than anecdotal.
Canonical formula: intermittent_episode + trigger_condition + capture_window + state_snapshot + contextual_trace + evidence_preservation + follow_up_analysis -> diagnosable_failure_record
When This Archetype Applies¶
Partial catalog groundingSome structural conditions are represented by existing abstractions, but no sufficient condition set is fully represented.
Diagnostic problem
A failure, symptom, anomaly, or breakdown appears irregularly and cannot be reliably reproduced or inspected during ordinary investigation, so the system repeatedly returns to normal before enough evidence is captured to explain what happened.
What this problem means
The structural problem is an evidence timing mismatch. Diagnosis needs information from the failure state, but the failure state is intermittent and self-erasing. Logs rotate, systems reset, symptoms fade, witnesses forget, cleanup routines remove evidence, and people reinterpret episodes after the fact.
This creates recurring root-cause ambiguity. Each episode is treated as isolated or anecdotal because the common sequence across episodes is unavailable. The system may accumulate reports without accumulating diagnosis.
Applicability expression4 distinct conditions
′ context guard? connective not recorded∅ no catalog witness yet
groundedpartly groundedopen
4 conditions, all required.
4Required in every casenumbered 1–4
These hold no matter which pattern applies.
Sparse episodic failure · grounded
Failure appears in sparse, irregular, high-deviation episodes separated by ordinary behavior.
The source archetype describes the situation as follows: Failure is intermittent. The normalized requirement above isolates the load-bearing portion used in this condition set.
primeIntermittency— Irregular bursts.
Inspection misses episodes · grounded · any one of 2
Ordinary inspection is unlikely to coincide with the failure episode.
The source archetype describes the situation as follows: Ordinary inspection misses the episode. The normalized requirement above isolates the load-bearing portion used in this condition set.
domainHeisenbug— Name the software defect whose failure vanishes under observation because attaching a debugger, adding a print, or disabling optimization perturbs the timing window or interleaving it rides on — so disappearance-under-observation is the diagnostic tell, not evidence the bug is gone.
domainMandelbug— Reclassify a chaotic, irreproducible software defect as deterministic-but-undersampled — triggered by a rare joint configuration of hidden state — so effort shifts from finding the right breakpoint to sampling that joint state until the configuration is caught.
How this was matched — 4 requirements, all needed
Routine inspection rarely coincides with an intermittent failure.
All of
- roleA failure occurs in intermittent episodes.
- roleThe system is examined through ordinary inspection.
- modalityInspection is unlikely to coincide with the failure episode.
- timingThe inspection and failure must overlap for the episode to be observed directly.
Episode context required · open
The episode's local sequence, state, and environment are necessary for diagnosis.
The source archetype describes the situation as follows: Episode context matters. The normalized requirement above isolates the load-bearing portion used in this condition set.
Ephemeral diagnostic evidence · grounded · any one of 2
Diagnostic evidence from the episode disappears or degrades rapidly.
The source archetype describes the situation as follows: Evidence is ephemeral. The normalized requirement above isolates the load-bearing portion used in this condition set.
domainHeisenbug— Name the software defect whose failure vanishes under observation because attaching a debugger, adding a print, or disabling optimization perturbs the timing window or interleaving it rides on — so disappearance-under-observation is the diagnostic tell, not evidence the bug is gone.
domainFieldnote Backfill— The validity threat by which a delayed field write-up silently fuses direct observation, inference, and gap-filling reconstruction into one undifferentiated register, collapsing the attribution that tells the strongest evidence from the weakest.
context guardThe field encounter being recorded is an intermittent failure episode.
suppliesAn intermittent failure episode produces diagnostic evidence.
How this was matched — 1 shared + 2 branches
Intermittent-failure evidence disappears or degrades rapidly.
All of
- relationAn intermittent failure episode produces diagnostic evidence.
…and any one of
- relationThe diagnostic evidence can disappear.
- timingThe diagnostic evidence can degrade rapidly.
Other requirements and context (1)
Why these sit outside the expression
Solution feasibility — it describes whether the intervention can work, not whether the diagnostic problem exists.
Solution feasibilityCapture is possible without unacceptable harm.
The source states: Capture is possible without unacceptable harm. The normalized feasibility condition is: Capture is possible without unacceptable harm. It identifies something that must be possible or available for the intervention to be workable.
Coverage
3 of 4 conditions grounded · 1 open.
When to Use This Archetype¶
Use this archetype when failures or symptoms are real but hard to reproduce, when ordinary inspection arrives after the episode has disappeared, and when diagnosis depends on transient state or context. It is especially useful when continuous full monitoring is too costly, too intrusive, or too noisy, but targeted capture during episodes is feasible.
Good fits include intermittent production bugs, episodic health symptoms, rare equipment defects, safety near misses, field events, and organizational process breakdowns that look normal during scheduled reviews. The pattern is weaker when the failure is continuously visible, when safe reproduction is easy, or when useful capture would violate privacy, consent, safety, or proportionality constraints.
Structural Problem¶
The structural problem is an evidence timing mismatch. Diagnosis needs information from the failure state, but the failure state is intermittent and self-erasing. Logs rotate, systems reset, symptoms fade, witnesses forget, cleanup routines remove evidence, and people reinterpret episodes after the fact.
This creates recurring root-cause ambiguity. Each episode is treated as isolated or anecdotal because the common sequence across episodes is unavailable. The system may accumulate reports without accumulating diagnosis.
Intervention Logic¶
The intervention is to design a capture loop around the episode itself. First, define what counts as the failure and what signal should trigger capture. Next, decide what state, context, sequence, and observations would make the episode explainable. Then create a capture window that preserves evidence before, during, and after the visible symptom.
Once evidence is captured, it must be protected. If normal recovery overwrites logs, if records are detached from their episode, or if no one owns review, the pattern fails. The final step is recurrence analysis: compare captured episodes and turn the findings into corrective action, safer reproduction, prevention, or better capture rules.
Key Components¶
Intermittent Failure Capture turns self-erasing episodes into preserved evidence packets by designing a capture loop around the failure itself. The Trigger Condition defines the observable sign, threshold, anomaly, or report that opens evidence collection — specific enough to avoid constant noise, sensitive enough to catch episodes before important state is lost. The Capture Window defines how much evidence before, during, and after the visible symptom should be retained, because causes often appear before the failure and recovery evidence often explains how the system returned to normal. Within that window, three components preserve different layers of the episode: the State Snapshot captures internal state such as configuration, queue contents, variable values, and resource levels at the moment of failure; the Contextual Trace records surrounding conditions like timing, workload, dependency state, environmental readings, and recent changes; and the Event Log reconstructs the chronological sequence of signals, actions, and transitions so investigators do not have to rely on memory.
Three further components ensure that captured evidence becomes diagnosis rather than data hoarding. The Evidence Preservation Rule protects captured material from being overwritten, normalized away, or detached from its episode by ordinary cleanup routines — the difference between a useful record and a temporary diagnostic artifact. Recurrence Analysis compares episodes to find common preconditions, sequences, triggers, or hidden causes, since a single capture may make the problem visible while several captures often make it diagnosable. The Follow-up Diagnostic Path assigns review, escalation, reproduction boundaries, and corrective action, ensuring captured evidence routes into repair, prevention, or improved capture rules rather than accumulating unread. Together these components let intermittent failures become analyzable instead of anecdotal.
| Component | Description |
|---|---|
| Trigger Condition ↗ | defines the observable sign, threshold, anomaly, or report that starts evidence capture. It must be specific enough to avoid constant noise and sensitive enough to catch episodes before important state is lost. |
| Capture Window ↗ | defines how much evidence before, during, and after the episode should be retained. The window matters because causes often appear before the visible failure and recovery evidence may explain how the system returned to normal. |
| State Snapshot ↗ | preserves the internal state at or near the failure moment: configuration, queue state, variable values, resource levels, user/session state, environmental readings, or other domain-specific details. |
| Contextual Trace ↗ | captures surrounding conditions such as timing, workload, dependency state, user action, medication timing, staffing, weather, or recent changes. The context explains why the episode occurred in this instance. |
| Event Log ↗ | provides the chronological sequence of signals, actions, transitions, and observations. It lets investigators reconstruct the episode rather than relying on memory. |
| Evidence Preservation Rule ↗ | prevents captured evidence from being overwritten, normalized away, discarded, or altered before analysis. It is the difference between a useful episode record and a temporary diagnostic artifact. |
| Recurrence Analysis ↗ | compares episodes to find common preconditions, sequences, triggers, or hidden causes. A single capture may make the problem visible; several captures often make it diagnosable. |
| Follow-up Diagnostic Path ↗ | assigns review, escalation, reproduction boundaries, and corrective action. Without this path, capture becomes data hoarding rather than diagnosis. |
Common Mechanisms¶
Mechanisms implement the archetype; they are not the archetype itself. A Flight Recorder or Black Box Log maintains recent state so evidence survives the episode. A Trigger-Based Debug Trace or Automatic Diagnostic Capture turns on richer evidence collection when a symptom or threshold fires. An Incident Snapshot packages state, context, and logs into one preserved record. A Symptom Diary lets a person capture episodic human experience when sensors are unavailable. A Rare Event Monitor stays ready for low-frequency episodes, and a Post-Episode Evidence Review turns captured material into analysis while context is still fresh.
The mechanism choice depends on episode frequency, diagnostic need, privacy risk, cost, and whether evidence is best captured by automation, resilient recording, human reporting, or review workflow.
8 documented mechanisms across 4 implementation forms.
The grouping reflects forms represented among the mechanisms currently documented for this archetype; an absent form is not necessarily an impossible implementation.
Assessment, Review & Assurance · 1 mechanism
- Post-Episode Evidence Review — Reviews captured evidence soon after the episode while context is still recoverable and corrective action is still timely.
Control, Automation & Runtime · 1 mechanism
- Automatic Diagnostic Capture — Automatically saves diagnostic details when a trigger condition is met, reducing dependence on the investigator being present.
Monitoring, Sensing & Alerting · 2 mechanisms
- Rare Event Monitor — Watches for low-frequency events and preserves evidence when they occur rather than relying on continuous human attention.
- Trigger-Based Debug Trace — Turns on richer logging or tracing when a symptom, threshold, or anomaly indicates that an intermittent episode may be occurring.
Record, Log & Register · 4 mechanisms
- Black Box Log — Maintains tamper-resistant or failure-resistant traces for situations where ordinary records may be lost during the event itself.
- Flight Recorder — Continuously or selectively retains recent operational state so evidence is available when a rare episode occurs.
- Incident Snapshot — Packages state, context, logs, timestamps, and human observations into a preserved record for later analysis.
- Symptom Diary — Lets a person record timing, context, preceding conditions, symptoms, and recovery details for episodic problems that are not directly instrumented.
Parameter / Tuning Dimensions¶
Important tuning dimensions include trigger sensitivity, trigger specificity, pre-event buffer length, post-event capture duration, snapshot depth, contextual breadth, storage retention, access control, review urgency, false-positive tolerance, and capture overhead.
The central tuning problem is balancing diagnostic power against cost and intrusion. More evidence can reveal causes, but broad capture can create privacy risk, analysis burden, performance impact, and false certainty. The best design captures enough to distinguish the episode from normal operation while minimizing unnecessary exposure.
Invariants to Preserve¶
The first invariant is that episode evidence must survive beyond the episode. The second is that state, context, timing, and observations must remain linked as one record. The third is that ordinary operation, recovery, and human safety must not be compromised by the capture setup.
A fourth invariant is that capture must remain tied to diagnosis. The intervention is not justified by accumulation alone; it should support review, learning, repair, prevention, or a decision to retire the capture mechanism.
Target Outcomes¶
The desired outcome is diagnosable failure episodes. Instead of vague reports that a problem “happened again,” the system produces preserved evidence: what state it was in, what conditions surrounded it, what sequence led into it, and what changed afterward.
Successful use reduces root-cause ambiguity, shortens the path to corrective action, reveals recurrence patterns, and reduces dependence on unsafe reproduction attempts or excessive continuous monitoring.
Tradeoffs¶
Intermittent Failure Capture trades simplicity for diagnostic power. It can add instrumentation overhead, storage cost, workflow burden, and governance complexity. It can also create ethical concerns when captured evidence includes sensitive human behavior, personal data, or security details.
The archetype also trades immediacy for accuracy. Capturing and preserving evidence may slow cleanup or require additional review. In safety-critical contexts, evidence collection must never delay emergency response.
Failure Modes¶
A common failure mode is a trigger that fires too late or too narrowly, missing the relevant state. Another is thin capture: the system records that the episode occurred but omits the context needed to explain it. Capture can also overwhelm review capacity when triggers are too broad or evidence packets are too large.
Other failure modes include privacy violations, instrumentation that changes the behavior being diagnosed, evidence stored without analysis, and overinterpretation of a small number of captured episodes. These can be mitigated through capture budgets, minimization, trigger review, access control, and explicit diagnostic ownership.
Neighbor Distinctions¶
Intermittent Sampling detects intermittent states by choosing observation windows or triggers. Intermittent Failure Capture preserves richer state and context during actual failure episodes so diagnosis remains possible afterward.
Observability Instrumentation broadly exposes system state. Intermittent Failure Capture is narrower: it is episodic, evidence-preserving, and diagnosis-centered.
Perturbation Testing tries to induce or reproduce behavior under controlled change. Intermittent Failure Capture waits for naturally occurring episodes and captures evidence when they happen.
Intermittent Burst Absorption protects operations from irregular spikes. Intermittent Failure Capture diagnoses elusive failures; it may not absorb or mitigate the episode at all.
Recurrence Pattern Detection can analyze captured records, but it is usually a component or follow-on analysis rather than the capture archetype itself.
Cross-Domain Examples¶
In software operations, a rare production bug can trigger expanded traces and state snapshots because ordinary logs are too thin after recovery. In healthcare, a symptom diary can preserve timing, context, and recovery details for episodes absent during appointments. In manufacturing, a defect detector can freeze sensor state, material batch metadata, operator actions, and environmental readings when an intermittent flaw appears.
In safety engineering, black-box logs preserve pre-incident state for post-event investigation. In organizational reliability, teams can record workload, handoff state, decision context, and dependency signals whenever an episodic escalation failure occurs.
Non-Examples¶
A daily dashboard of aggregate error rates is not this archetype unless it preserves episode-level failure evidence. A generic bug report saying “it happened again” is not enough because it lacks state and context. Random spot checks for compliance are usually Intermittent Sampling. A burst queue that holds demand during a spike is Intermittent Burst Absorption, not failure capture.
Related Abstractions¶
Abstractions this archetype builds on — directly (a source ingredient) or as a related pattern. Links follow the typed catalog namespace.
Built directly on (2)
- Intermittency: Irregular bursts.
- Observability: Infer internal state externally.
Also references 11 related abstractions
- Accountability: Responsibility for actions.
- Causality: Cause-effect relationships.
- Fault Tolerance: Continue operating under failure.
- Feedback: Outputs influence inputs.
- Irreversibility: Cannot revert state.
- Resilience: Absorb shocks and adapt.
- Sampling (Representativeness): Representative subset selection.
- Signaling: Revealing hidden information.
- State and State Transition: Captures system condition and evolution.
- Threshold: Safe vs harmful levels.
Variants¶
Narrower or domain-specific specializations that share this archetype's core structure. Recognized variants are established; candidate variants are provisional.
Trigger-Based Diagnostic Capture · recognized
- Common mechanisms: Trigger-Based Debug Trace, Automatic Diagnostic Capture
Episodic Symptom Capture · recognized
- Common mechanisms: Symptom Diary, Post-Episode Evidence Review
Flight-Recorder Failure Capture · recognized
- Common mechanisms: Flight Recorder, Black Box Log
Rare Event Evidence Capture · recognized
- Common mechanisms: Rare Event Monitor, Automatic Diagnostic Capture
Near names: Episodic Failure Capture, Intermittent Bug Capture, Rare Event Capture, Debug Trace on Trigger, Incident Snapshot, Symptom Diary, Flight-Recorder Logging.
Editorial Notes¶
Problem Classification¶
Classification: Observability, Measurement & Feedback Gaps → Temporal Sampling, Decay & Transition Resolution
Problem kernel: intermittent failure vanishes before investigation
Rationale: The system returns to normal between episodes, so ordinary observation never captures the short-lived state needed for diagnosis.
Independent corroboration: The earliest necessary condition in the frozen evidence is: A failure, symptom, anomaly, or breakdown appears irregularly and cannot be reliably reproduced or inspected during ordinary investigation, so the system repeatedly returns to normal before enough evidence is captured to explain what happened. That is a temporal sampling decay and transition resolution problem because Observation cadence or anchoring does not match state dynamics, so brief events, decaying effects, accumulated drift, and approaching transitions are missed.
Review outcome: Independent reviewer agreement; high confidence.