{"schema_version":1,"experiment_id":"eoa_inverse_innovation_exp09_archetype_breadth150_20260804","cell_id":"feedback_loop_redirection__computer_science","arm":"BREADTH_PROBE_ONE_SHOT","candidate_id":"feedback_loop_redirection__computer_science__P1","proposal_index":1,"version":0,"title":"Saturation-Aware Diagnostic Retries for CI Flake Storms","problem":"In a continuous-integration system, a failed test automatically triggers retries on the same shared runner pool. When failures are timing-sensitive, retries increase runner saturation and timing variance, which produces more timing-sensitive failures and therefore more retries. The CI system's corrective response can thus recreate and amplify the condition that elicited it.","actors":["CI platform owner","Repository maintainers","Developers awaiting build results","Shared-runner scheduler","Test jobs sharing runner capacity"],"observable_state":"A failure burst is followed by rising retry-job arrivals, runner occupancy, queue delay, and additional failures with signatures that disappear on later executions; retry-generated work constitutes an observable portion of the load during the burst.","consequence":"The retry loop delays trustworthy build results, consumes capacity needed by unaffected jobs, and can blur the distinction between deterministic regressions and load-sensitive test failures.","affected_objective":"Produce timely, trustworthy CI verdicts while preserving diagnostic evidence and bounded shared-runner utilization.","intervention":"Replace the fixed failure-to-immediate-retry rule with an auditable saturation-aware diagnostic response. A first failure retains its raw failed verdict and is classified using its normalized failure signature, recent matching outcomes, and current runner saturation. The system may schedule at most one low-priority diagnostic replay when capacity is below a declared threshold. As saturation or repeated nondeterminism rises, the retry gain falls to zero and the response changes from execution to evidence preservation, an explicit unstable-test annotation, and owner notification. The target of the loop becomes diagnostic information gained per replay rather than obtaining a passing rerun; no retry may convert the original failure into a pass.","structural_mapping":[{"archetype_element":"Existing feedback loop","domain_realization":"Test failure triggers retry work; retry work raises shared-runner load and timing variance; elevated variance creates further failures that trigger more retries."},{"archetype_element":"Observable signal","domain_realization":"Normalized failure signature, recent reproducibility of that signature, retry-origin job count, queue delay, and runner saturation."},{"archetype_element":"Signal modification","domain_realization":"A bare failure event is replaced as the retry-control signal by failure evidence combined with reproducibility history and current saturation."},{"archetype_element":"Response-rule change","domain_realization":"Failure no longer always causes an immediate same-priority retry; it causes either one capacity-gated diagnostic replay or a non-executing evidence-and-notification response."},{"archetype_element":"Gain adjustment","domain_realization":"Permitted diagnostic retries decrease from one to zero as saturation or repeated nondeterminism crosses declared thresholds."},{"archetype_element":"Target redefinition","domain_realization":"The retry controller optimizes for additional diagnostic information without obscuring the original verdict, rather than for a passing rerun."},{"archetype_element":"Monitoring and adjustment","domain_realization":"Operators compare retry load, queue behavior, verdict latency, diagnostic yield, and missed-regression indicators, with thresholds versioned and reversible."}],"mechanism_mapping":[{"mechanism_slug":"control_loop_retuning","role":"Retunes retry gain and timing according to runner saturation so the corrective action does not intensify the sensed condition.","counterfactual_removal":"Without retuning, every qualifying failure continues to add retry load even when the pool is already unstable."},{"mechanism_slug":"response_rule_change","role":"Redirects a failure from automatic re-execution toward either one diagnostic replay or evidence preservation and notification.","counterfactual_removal":"Without the response-rule change, added observability would describe the storm but would not alter it."},{"mechanism_slug":"target_redefinition","role":"Changes the retry target from securing a passing execution to obtaining bounded diagnostic information while retaining the failed verdict.","counterfactual_removal":"Without target redefinition, the controller remains pressured to rerun until success and can still reward nondeterministic passing outcomes."}],"causal_chain":["A timing-sensitive test fails on a shared runner.","The baseline rule launches an immediate retry into the same pool.","Retry work raises occupancy, queueing, and timing variance.","Higher variance induces additional timing-sensitive failures.","Those failures trigger further retries, reinforcing the load-failure loop.","The revised controller combines failure evidence with reproducibility and saturation signals.","It reduces retry gain under stress and substitutes evidence preservation plus notification for additional execution.","Less retry-origin work is injected into the stressed pool while original failures remain visible.","The reinforcing path is weakened without treating a rerun pass as authoritative."],"baseline":"Each eligible failed test is retried immediately at ordinary priority on the shared runner pool, commonly with a fixed retry count; a passing rerun may dominate how the result is presented, and retry behavior does not respond to pool saturation.","nearest_rivals":["A static retry cap limits total attempts but does not redirect the response as saturation changes.","A circuit breaker stops retries after a threshold but does not retarget retries toward bounded diagnostic information or incorporate signature reproducibility.","A separate retry-runner bulkhead contains resource impact but leaves the failure-to-retry reinforcement rule intact inside that compartment.","Flaky-test quarantine removes selected tests from gating but can conceal current failures and acts after classification rather than retuning the live loop.","Additional retry dashboards improve observability but do not change the signal-to-action relation."],"remaining_contrastive_claim":"The candidate's defining move is to make retry execution a decreasing, capacity-gated response to combined reproducibility and saturation evidence while preserving the original failed verdict. Its causal distinction from static caps, isolation, quarantine, or monitoring is the explicit rewiring of the recurring failure-to-retry rule and its target.","authority_safety":{"decision_authority":"The CI platform owner may implement scheduler behavior, but each repository's maintainers must approve its classification and notification policy; protected-branch verdict policy remains with the repository's existing authorized owners.","authorized_first_step":"Perform a read-only trace replay for one repository using historical job events; compute what the proposed controller would have done without changing scheduling, verdicts, branch protection, or developer notifications.","excluded_actions":["Converting an original failed test into a passing verdict because a retry passes","Automatically quarantining or disabling tests","Changing protected-branch requirements","Running more than one diagnostic replay per original failure","Raising job priority or taking capacity from unrelated tenants","Using undisclosed failure classifications to evaluate individual developers"],"halt_rollback":"Halt evaluation if required signals cannot be reconstructed reliably or if the shadow rule would suppress evidence of deterministic regressions. Any later pilot must be feature-flagged; rollback restores the prior retry rule immediately while retaining logs and original verdicts."},"negative_tests":{"strongest_counterevidence":"The apparent storm may be caused entirely by a deterministic bad change, runner infrastructure failure, or external dependency outage, with retries merely coincident with rather than causal to saturation and subsequent failures.","problem_falsifier":"The feedback-loop diagnosis is falsified for the sampled episodes if retry jobs do not precede or contribute to increased saturation, or if removing retry-origin demand in trace replay leaves the subsequent queue and failure sequence unchanged.","intervention_falsifier":"The intervention is falsified if shadow replay shows that the redirected rule still produces reinforcing retry bursts, loses failure evidence, increases time to identify deterministic regressions, or yields no usable diagnostic distinction between reproducible and non-reproducible failures.","risks":["Signature grouping may merge unrelated failures or split identical ones.","Tests or teams may adapt to annotations in ways that conceal instability.","Reducing retries may delay collection of evidence needed for rare failures.","Thresholds may oscillate near saturation boundaries.","A low-priority diagnostic lane may create a hidden secondary queueing loop.","Maintainers may interpret unstable-test annotations as permission to ignore failures.","Repository-specific workloads may make a shared threshold illegitimate.","Historical replay may omit timing interactions needed to predict scheduler behavior."]},"next_evidence_step":"For one consenting repository, reconstruct ten consecutive business days of CI job events and identify failure bursts containing original attempts, retry arrivals, saturation, queue delay, normalized signatures, and final verdicts. Run the baseline and proposed response rule in shadow trace replay, inspect every case where actions differ, and reject the candidate unless the retry-generated load path is temporally present, deterministic failures remain visible, and the revised rule avoids generating a secondary retry burst. This step changes no production behavior and ends after the fixed trace window and case review.","prior_art_status":"UNSEARCHED","diversity_from_prior_proposals":"Not assessed against other proposals because runtime isolation forbids inspecting them; this candidate is independently anchored in the CI scheduler's failure-retry-load feedback loop.","revision_record":{"parent_version":null,"progress_targets_addressed":[],"conceptual_changes":[],"operational_changes":[],"evidence_changes":[],"claim_changes":[]}}