# Audit-set: S2_service_reliability — Cascading failure in a high-traffic online service

Below are 4 reasoning transcripts (W1..W4) for the same problem, each a different solver run. For EACH, classify whether the solver performed a COVERAGE CHECK, defined as: it (a) took a specific solution pattern it was relying on and named that pattern's FULL set of components/elements, AND (b) went through them and checked, component by component, whether the design instantiates each (or deliberately declines it / leaves it missing). Casually mentioning a few features is NOT a coverage check; the test is explicit, enumerated, component-by-component verification.

Classify each as: FULL (clear enumerated component-by-component check), PARTIAL (some enumeration but not a systematic per-component presence check), or NONE. One-line justification each. You do not know how these were produced; judge only what the transcript shows.

## Transcript W1
```
# Working (discarded before grading)

## State

**problem.** Intermittent total outages. A modest demand spike slows a few backend nodes; clients/internal callers auto-retry failed requests, multiplying load on already-struggling nodes, pushing more nodes over the edge until the whole service saturates. Recovery requires manual traffic-shedding. Team has good monitoring, can change client/server behavior, set limits, add capacity, but cannot indefinitely over-provision.

**meta_model.** A small local perturbation feeds a *reinforcing* feedback loop (the corrective response — retrying — is itself the amplifier), which propagates across tightly coupled units past a tipping threshold into total collapse, with no balancing loop to pull the system back, so it cannot self-recover.

## Verbs run (in order, with one-line why)

1. **match** (search_prime) — anchor the meta-model. Surfaced: circular_causality, homeostasis, instability, cascade, threshold_and_criticality, perturbation, resonance/amplification, self_organization.
2. **match** (search_by_facets) — oblique, multi-structure problem; decompose into 7 facets (amplifying loop / chain-reaction spread / response-increases-load / no-slack coupling / no self-recovery past threshold / rate-limiting a shared resource / isolating a failing part). Per-facet surfaced system_archetypes, systemic_risk, coupling, load_balancing, controlled_reentry, robustness, functional_redundancy, plus stress_rupture/regime_change at the fusion top.
3. **salience-rank** — load-bearing core: (a) circular_causality — the retry loop is a *positive* loop where the response IS the amplifier; (b) homeostasis — the missing balancing loop + envelope limits (T2); (c) controlled_reentry — self-recovery must be staged, not a toggle (T1, T4); (d) coupling/cascade — propagation across nodes; threshold the tipping point.
4. **prune** — drop resonance/synchronization/self_organization (descriptive, not actionable here); keep the four operative primes + the cascade/threshold framing.
5. **get_prime** circular_causality — confirms: positive loop → exponential blow-up; intervene at *loop structure/leverage points*, not the visible hot node; cites retry-storm/circuit-breaker as the canonical loop-interruption.
6. **get_prime** homeostasis — confirms the cure is a *negative* (balancing) loop: sensor→comparator→actuator with finite correction authority and response time; envelope must reject the disturbance class; T5 warns of overshoot/oscillation (over-throttling = the failure criterion).
7. **get_prime** controlled_reentry — confirms recovery is asymmetric to failure: staged ramp with go/no-go gates and re-suspend capability; T4 warns reentry can re-create the original overload (so ramp must back off on the same signal that tripped it).
8. **match** (search_archetype) — concrete mechanism set: cycle_breaking, recovery_interval_design, coordination_across_reentry_phases, intermittent_burst_absorption, **cycle_staggering** (de-synchronize retries — the jitter mechanism), tapering_strategy, elastic_capacity_scaling.
9. **compose / lower** — assemble into the typed model and instantiate concrete mechanisms, rules, thresholds, paths, and the operating rule below.
10. **evaluate-fit (gate)** — check against the explicit failure mode (over-aggressive throttling routinely degrading normal traffic). Resolved by load-shedding by *priority class* + adaptive limits keyed to live latency, not static caps; and by hysteresis/jittered ramp to prevent oscillation (homeostasis T5).

**model (typed).** Nodes: clients/callers, edge/admission layer, backend node pool, shared dependency. Edges: request→node (load), node-slow→client (failure signal), client→node (RETRY = the reinforcing edge to cut), operator→system (manual shed = the missing balancing edge to automate). Leverage point = the retry edge + an admission/balancing loop, NOT the hot node.

COST: ~9 operations, ~22k tokens.
```

## Transcript W2
```
# c22 — runaway-retry cascade / self-recovery design

## STATE

**problem.** Intermittent total outages. A modest demand spike slows a few backend nodes; automatic client/internal retries multiply load on the struggling nodes; more nodes tip over; the service saturates. Recovery needs manual traffic-shedding. Levers available: client + server behavior, limits, capacity, good monitoring; cannot indefinitely over-provision.
- success: a local slowdown stays local AND the system self-recovers within minutes without manual shedding.
- failure: a single hot spot can still cascade to total outage, OR safeguards throttle so hard that normal traffic is routinely degraded.

**meta_model (L1, domain-stripped).** Roles: a perturbation source (small load increase); a population of coupled processing units sharing capacity; agents that react to slow/failed responses by re-issuing demand; a shared exhaustible resource. Relations: a *reinforcing* loop (failure → retry → more load → more failure) overwhelms the system's weak *balancing* loops; tight coupling lets local failure propagate; no boundary isolates a failing unit; the system has no autonomous return path to a viable range (recovery is exogenous/manual). Invariants to hold: bound the amplification of small disturbances; keep failure local; provide an autonomous balancing loop that returns to a viable operating range; preserve responsiveness to genuine demand.

**operative_primes.**
- reinforcing-loop amplification / instability — *load_bearing* (the retry storm IS the disease).
- propagation / cascade across coupled units (systemic_risk, no isolation) — *load_bearing*.
- ultra-stability / homeostasis — *load_bearing* (autonomous return to a viable *range*, not a fixed point; requisite variety of regulatory responses).
- interference_and_contention on a shared resource — *context* (the substrate the loop runs on).
- hysteresis (asymmetric leave/return thresholds) — *load_bearing for the recovery half*.

**candidates (verbs: match via search_by_facets + search_archetype).**
- `instability_dampening` (overlap: instability, damping, feedback; related: amplification, threshold, homeostasis, resilience) — central; literally names retry-storms, backoff, circuit-breaker, bulkhead isolation, safe-fallback-with-reentry.
- `balancing_loop_stabilization` (circular_causality, feedback, homeostasis) — the autonomous self-correction loop with anti-overshoot tuning.
- supporting recognition only: `hysteresis_management` (separate trip/re-admit thresholds), `boundary_permeability_control` (the isolating boundary), `equilibrium_restoration` (self-recovery target).

**VERBS RUN (history).**
1. match (search_by_facets, 5 facets) — oblique multi-structure problem; surfaced reinforcing-loop, cascade/systemic_risk, ultra-stability, contention.
2. match (search_archetype) — `instability_dampening` 0.79 top.
3. get_archetype ×2 + get_prime — pulled full component lists + ultra-stability signature.
4. match (search_archetype, isolation/recovery facet) — confirmed hysteresis + boundary + equilibrium-restoration as recognition, not new structure.
5. compose — assembled the reinforcing/balancing/boundary model (referential integrity: every operative prime annotated).
6. lift (L2 coverage check, my choice — yes) over `instability_dampening` + `balancing_loop_stabilization` components.

**coverage_map (L2 — every component statused).**
- Amplification-pathway map → *instantiated*: retry-on-failure is the gain path; attack it directly.
- Gain / cadence control → *instantiated*: bounded retries + exponential backoff with jitter + a per-caller retry budget.
- Propagation boundary → *instantiated*: bulkheads (per-dependency concurrency pools) + per-node circuit breakers.
- Buffer / absorptive capacity → *instantiated*: bounded queues + fast load-shedding (reject early, cheaply).
- Balancing loop / correction rule → *instantiated*: adaptive concurrency limit driven by latency/error deviation; autoscaling as slow loop.
- Hysteresis / reentry → *instantiated*: breaker half-open probe; separate trip vs. re-admit thresholds; gradual ramp.
- Safe fallback state → *instantiated*: degraded-but-up responses (cached/partial/defaults), not failure.
- Over-damping guard (the over-lift / over-throttle guard) → *instantiated*: prioritize by criticality, size limits to spare capacity, idempotency keys, observe both cascade AND false-throttle.
- Saturation / escalation → *correctly_declined as a primary lever*: manual escalation kept only as last-resort floor; cannot over-provision, so not the design.
- Residual-instability monitor → *instantiated*: track shed rate + breaker state + retry-budget exhaustion.
No component `missing` → STOP permitted.

COST: ~6 operations (5 MCP calls + 1 write), ~14k tokens.
```

## Transcript W3
```
# Working — c24 (control policy trace)

## STATE: problem
- **Statement.** A high-traffic service suffers intermittent total outages. A modest request spike slows a few backend nodes; clients/internal callers auto-retry, multiplying load on the struggling nodes, pushing more nodes over the edge until the whole service saturates. Recovery requires operators to manually shed almost all traffic and bring the system back slowly.
- **Constraints.** Can change client + server behavior, set limits, add capacity, has good monitoring; cannot indefinitely over-provision.
- **Success.** A localized slowdown stays localized AND the system self-recovers to normal within minutes without manual shedding.
- **Failure.** A single hot spot can still cascade to total outage, OR safeguards throttle so aggressively that normal traffic is routinely degraded.

## Verbs run (in order, with why)

**1. match / transport (facet retrieval).** Problem is oblique — it instantiates at least four structures at once (positive-feedback amplification, a multiplying-retry regeneration mechanism, a no-autonomous-return tipping/hysteresis trap, and fault-isolation). Ran `search_by_facets` over six domain-stripped facets + a blended `search_prime`. Surfaced, with high confidence: `circular_causality` (the retry→overload→more-retry reinforcing loop; its own text names "retry storms / cascading failure / circuit-breaker patterns as loop interruption"), `cascade`, `tipping_points`/`hysteresis` (asymmetric up/down thresholds → no self-recovery), and the recovery family containing `circuit_breaker` and `controlled_reentry`. Archetypes: `instability_dampening`, `cycle_breaking`.

**2. salience-rank.** Load-bearing: `circular_causality` (names the loop to cut), `circuit_breaker` (the trip-isolate-reset skeleton), `instability_dampening` (the lever menu: rate-limit, backoff, bulkhead, buffer, staged escalation, fallback), `controlled_reentry` (the autonomous-return half — staged ramp, half-open probe, hysteresis), `cycle_breaking` (the retry is the *regeneration point*). Context: `hysteresis` (threshold-gap rationale), `load_shedding`/`bounded_backlog` (priority shedding + bounded queues).

**3. compose → model.** Nodes: client/caller, retry policy, queue, backend node, dependency edge, monitor, breaker, admission controller. Edges: retry→queue (amplifies), queue→node (loads), node-slow→retry (closes the reinforcing loop), breaker⊣loop (cuts), admission⊣queue (shed). Every operative prime annotates an edge — referential integrity holds.

**4. lift L1 (situation→meta-model).** Skeleton: *a shared bounded resource under a reinforcing loop whose gain rises with stress; the loop has an asymmetric activation/recovery threshold so it cannot self-clear; remedy = cut loop gain at its regeneration point, contain locally, then re-engage on a staged, monitored, hysteretic path.* No domain terms remain; every role has a witness in the model.

**5. lift L2 — coverage check (CHOSE TO RUN: yes).** Posed each pattern's components against the draft and checked instantiation:
- *Circuit_breaker* → trip threshold = INSTANTIATED (per-dependency error/latency budget); isolation scope = INSTANTIATED (per-dependency + per-shard bulkheads); reset policy = INSTANTIATED (half-open probe); **over-lift guard / T5** = degraded-but-useful service should not be amputated → handled by serving cached/fallback rather than hard-open where possible.
- *Cycle_breaking* → regeneration point = INSTANTIATED (the automatic retry); interruption lever = INSTANTIATED (retry budget + backoff+jitter); replacement pathway = INSTANTIATED (fail-fast fallback so callers still get an answer); recurrence monitor = INSTANTIATED (retry-ratio + saturation alarms).
- *Instability_dampening* → gain reduction = INSTANTIATED (token-bucket rate limits, concurrency caps); buffer = INSTANTIATED (bounded queues, shed on overflow); propagation boundary = INSTANTIATED (bulkheads); preserve true-signal = INSTANTIATED (priority classes so health-checks/critical paths bypass shedding) — this is the anti-over-throttle guard for the failure criterion.
- *Controlled_reentry* → staged ramp = INSTANTIATED; telemetry gate = INSTANTIATED; re-suspend on relapse = INSTANTIATED; T4 "re-creates original problem" = guarded by ramping at a rate below measured headroom; **hysteresis** (recover threshold stricter than trip threshold) = INSTANTIATED to stop breaker chatter.
- *Load_shedding/bounded_backlog* → admission control at edge + queue caps = INSTANTIATED.
- Correctly DECLINED: indefinite over-provisioning (constraint-barred); broad market-wide-style global halt (isolation must be fine-grained, not whole-service) — declining it IS the fix for the cascade.
- No component left `missing` → STOP condition met.

**6. evaluate-fit (gate).** Anti-signatures walked: "problem is a one-time shock" — not tripped (recurs). "Already-large disturbance needs recovery not sensitivity-reduction" — partially tripped → that's why a recovery (reentry) arm is included alongside the dampening arm. "Damping suppresses legitimate signal" — guarded by priority classes + health-check exemption. Candidates kept.

**7. reconcile.** Concrete model says "cut the retry loop"; meta-model says "reduce loop gain at the regeneration point AND restore via a hysteretic staged path." Divergence: a pure breaker (concrete) would satisfy containment but not autonomous recovery; the meta-model forces the reentry/hysteresis arm. Both readings retained → recommendation carries both arms. Meta-model not decorative.

STOP — sound recommendation ready.
```

## Transcript W4
```
# Working (control policy: free verb order; discarded before grading)

STATE.problem: intermittent total outages; a modest demand spike slows a few backend nodes; automatic client/internal retries multiply load on the struggling nodes, pushing more over the edge until full saturation; recovery needs manual traffic-shed. Need: keep local slowdown local AND self-recover within minutes, without routinely degrading normal traffic.

VERBS RUN (in order, with one-line why):
1. **match** — search_prime with the domain-stripped meta-model ("local perturbation triggers an amplifying feedback response that escalates to total collapse; cannot self-recover"). Surfaced the amplifying-loop/collapse neighborhood.
2. **match (search_by_facets)** — six structural facets (amplifying-overload loop; contagious local→total failure; uncoordinated demand exhausting a shared resource; retries multiplying load; cannot self-reset; rate-limit to protect downstream). Per-facet lists pulled the operative spread: cascade/systemic_risk, tragedy_of_the_commons/scarcity, buffering/bottleneck/load_balancing, controlled_reentry/resilience/fault_tolerance, hysteresis.
3. **salience-rank** — load-bearing core = the re-emission/amplification loop (cascade) + the retry-incentive misalignment (tragedy_of_the_commons); recovery half = controlled_reentry + hysteresis. Periphery (instability/perturbation/tipping_points) = vocabulary, pruned.
4. **prune** — cut to: cascade, tragedy_of_the_commons, controlled_reentry, hysteresis (+ archetypes instability_dampening, balancing_loop_stabilization).
5. **lift** — abstract structure: each node re-emits the disturbance (retry = re-emission); criticality parameter k = amplification factor per failed request. Containment menu (from cascade): cut coupling edges, raise neighbor thresholds, insert firebreaks that absorb re-emission before it regenerates. Keep k<1.
6. **transport (+map)** — instability_dampening archetype: map "where is the gain?" → the retry loop is the gain. Its mechanisms (rate limiter, exponential backoff, bulkhead/segment isolation, circuit breaker, staged-escalation gate) align 1:1 to the target. Roles: perturbation=demand spike; amplification path=retries×fan-out; firebreak=load-shed/circuit break.
7. **compose** — typed model: clients/internal-callers --(retry edge, the amplifier)--> backend nodes --(shared dependency edges)--> downstream pools. Annotate: retry edge = tragedy_of_commons (private-rational retry, collective-destructive); node→pool edges = cascade coupling; recovery path = controlled_reentry; trip-vs-recover asymmetry = hysteresis.
8. **lower** — instantiate concrete mechanisms/thresholds (below).
9. **reconcile** — recovery loop tuned via balancing_loop_stabilization: separate activation/deactivation thresholds (hysteresis band) to prevent flapping; deadband + effect-monitoring so it does not over-damp normal traffic.
10. **evaluate-fit (gate)** — checked instability_dampening anti-signatures: NOT mere capacity absence (it is genuine amplification) ✓; NOT a case where amplification is wanted ✓. Failure modes guarded: over-damping (→ the failure criterion "routinely degraded normal traffic": use deadband, prioritized shedding, fast recovery); displaced instability (→ bulkheads per dependency, not one global throttle); hidden accumulation (→ alert on sustained shed/breaker-open as a redesign signal). Gate PASSED.

COST: ~9 operations, ~24k tokens.
```
