{"schema_version":1,"experiment_id":"eoa_inverse_innovation_exp09_archetype_breadth150_20260804","research_id":"eoa_inverse_innovation_exp09_light_prior_art_20260804","cell_id":"feedback_loop_redirection__computer_science","search_lanes":{"direct_problem_and_intervention":{"queries":["CI flaky test retries resource consumption runner saturation shared runners research","CI test retry storm saturation capacity aware retries flaky tests","CI capacity-aware retries failure signature saturation diagnostic replay"],"source_ids":["SRC1","SRC4"],"no_result_note":null},"synonyms_and_historical_terms":{"queries":["CI \"resource-affected flaky tests\" rerun retries resource starvation","test retry circuit breaker runner saturation preserve original failure verdict","retry storm thundering herd load amplification"],"source_ids":["SRC1","SRC4"],"no_result_note":null},"products_practices_and_standards":{"queries":["Buildkite automatic retry documentation exit status limit","flaky test retry load shedding shared runner queue","Chromium CQ failed test shard retry policy"],"source_ids":["SRC2","SRC3"],"no_result_note":null},"component_combination":{"queries":["CI capacity-aware retries failure signature saturation diagnostic replay","test retry circuit breaker runner saturation preserve original failure verdict","flaky test retry load shedding shared runner queue"],"source_ids":["SRC1","SRC2","SRC3","SRC4"],"no_result_note":"No retained source disclosed the complete conjunction of saturation- and reproducibility-controlled retry gain, at most one low-priority diagnostic replay, evidence-and-notification fallback, and an original failure that cannot become a passing verdict."}},"sources":[{"source_id":"SRC1","title":"The Effects of Computational Resources on Flaky Tests","publisher":"IEEE Transactions on Software Engineering / arXiv","url":"https://arxiv.org/abs/2310.12132","source_type":"PRIMARY_RESEARCH","claims_supported":["CI tests compete for limited CPU, memory, disk, and network resources.","In a study of 52 projects and 27 resource configurations, 46.5% of identified flaky tests were resource-affected.","Resource starvation can substantially increase nondeterministic test-failure rates, supporting the proposed load-to-flakiness path."]},{"source_id":"SRC2","title":"Chromium Docs — Commit Queue","publisher":"Chromium Project","url":"https://chromium.googlesource.com/chromium/src/%2B/refs/heads/main/docs/infra/cq.md","source_type":"FIRST_PARTY_PRODUCT","claims_supported":["Chromium's commit queue retries failed test shards using the same configuration as the original execution.","A successful retry causes the initial failure to be ignored, directly illustrating retry-to-green verdict handling.","Chromium permits retries to be disabled and explicitly describes retry policy as a tradeoff involving flakiness and compute-intensive CI operation."]},{"source_id":"SRC3","title":"Retry","publisher":"Buildkite","url":"https://buildkite.com/docs/pipelines/configure/retry","source_type":"FIRST_PARTY_PRODUCT","claims_supported":["Buildkite supports automatic retries using failure conditions and fixed per-rule limits, including a zero limit.","Retry history and failed-job information are preserved and immutable while a new job is created.","The documented retry inputs include exit status, signal, and signal reason, but not live queue saturation or historical failure-signature reproducibility."]},{"source_id":"SRC4","title":"Retry Storm Antipattern","publisher":"Microsoft Azure Architecture Center","url":"https://learn.microsoft.com/en-us/azure/architecture/antipatterns/retry-storm/","source_type":"OFFICIAL_GUIDANCE","claims_supported":["Frequent retries against a busy or unavailable service can add load, prevent recovery, and intensify the original problem.","Retry storms are also called a thundering herd and are visible through bursts of attempts, long processing times, and cascading failures.","Established mitigations include bounded attempts, backoff, circuit breakers, throttling, bulkheads, and retry-after signals."]}],"problem_evidence":{"status":"PARTLY_SUPPORTED","finding":"The constituent mechanism is visible: primary research shows that constrained computational resources can increase flaky-test failures, Chromium and Buildkite show that failed CI work is commonly rerun, and Microsoft documents the general retry-storm feedback mechanism in which corrective retries intensify overload. The retained sources do not directly demonstrate a CI episode where retry-origin jobs measurably caused shared-runner saturation and then induced further timing-sensitive failures, so the proposal's complete causal loop remains to be established on CI traces.","source_ids":["SRC1","SRC2","SRC3","SRC4"]},"closest_prior_art":[{"name":"Chromium Commit Queue shard retry workflow","source_ids":["SRC2"],"overlap":"Retries failed test shards in the same configuration, uses reruns to distinguish patch failures from intermittent or pre-existing failures, permits disabling retries, and may ignore an initial failure after a successful rerun.","remaining_difference":"The disclosed workflow does not gate retry execution on current runner saturation or recent normalized-signature reproducibility, assign a single replay a diagnostic-information target and low priority, or require the original failed verdict to remain authoritative."},{"name":"Buildkite conditional automatic retries with immutable history","source_ids":["SRC3"],"overlap":"Offers auditable conditional retry rules, bounded or zero retry limits, preserved immutable attempt history, retry reporting, and distinct treatment of infrastructure-related failure reasons.","remaining_difference":"The documented controller uses configured exit, signal, and signal-reason conditions rather than live saturation and signature-history evidence; it does not describe monotonically decreasing retry gain under load or an evidence-and-owner-notification fallback."},{"name":"Retry-storm protection using limits, backoff, throttling, bulkheads, and circuit breakers","source_ids":["SRC4"],"overlap":"Recognizes the same reinforcing failure-retry-load loop and redirects or suppresses retry work when continued execution would worsen saturation.","remaining_difference":"The guidance concerns service requests generally, not CI verdict semantics; it does not combine test-signature reproducibility, a low-priority diagnostic replay, preserved failure evidence, and repository-owner notification."},{"name":"Resource-Affected Flaky Test identification","source_ids":["SRC1"],"overlap":"Classifies nondeterministic tests using failure behavior under differing resource conditions and establishes that resource availability can affect flakiness.","remaining_difference":"It is an experimental identification method rather than a live CI retry controller and does not use shared-pool saturation to choose between one diagnostic replay and no execution."}],"prior_art_disposition":"ADJACENT_PRIOR_ART","contrastive_claim_remaining":"For an original CI failure, combining recent normalized-signature reproducibility with current shared-runner saturation to reduce permitted replay work from one low-priority diagnostic execution to zero—while keeping the original failure authoritative and substituting preserved evidence plus owner notification—will reduce retry-origin burst load relative to fixed conditional retries without delaying recognition of deterministic regressions.","contrastive_claim_falsifier":"The claim is falsified if closer art discloses that complete control rule, or if a fixed-window shadow replay finds that the rule does not reduce retry-origin burst load, still creates a reinforcing or secondary replay queue, overwrites or suppresses original failure evidence, provides no additional diagnostic discrimination, or increases time to identify deterministic regressions compared with the repository's current retry policy.","gates":{"adequate_source_search":{"status":"PASS","rationale":"The bounded search covered direct CI retry-saturation formulations, resource-affected and retry-storm terminology, deployed CI retry practices, generic overload controls, and combinations involving signatures, saturation, circuit breakers, diagnostic replay, and verdict preservation. Exactly four opened sources span four publishers and include primary research, first-party documentation, and official guidance.","source_ids":["SRC1","SRC2","SRC3","SRC4"]},"supported_problem":{"status":"PASS","rationale":"The evidence directly supports resource-sensitive flakiness, established CI reruns, and the general retry-amplification mechanism. Although the end-to-end CI loop is only partly supported, it is observable and falsifiable with the proposed trace data.","source_ids":["SRC1","SRC2","SRC3","SRC4"]},"distinct_testable_claim":{"status":"PASS","rationale":"Adjacent art separately covers CI reruns, immutable retry history, resource-sensitive flakiness, and overload-responsive retry suppression. The remaining conjunction specifies measurable inputs, a zero-or-one replay action, priority and verdict constraints, and comparative outcomes suitable for falsification.","source_ids":["SRC1","SRC2","SRC3","SRC4"]},"bounded_next_test":{"status":"PASS","rationale":"A read-only replay of ten consecutive business days for one consenting repository is time-, tenant-, and action-bounded. It can compare retry-origin arrivals, saturation, queue delay, failure signatures, diagnostic yield, secondary bursts, verdict retention, and deterministic-regression recognition while manually reviewing every divergent decision.","source_ids":["SRC1","SRC2","SRC3"]},"no_obvious_safety_or_authority_stop":{"status":"PASS","rationale":"The proposed first test changes no job scheduling, verdict, branch protection, quarantine state, or notification. Existing product practices demonstrate that retry policy and attempt history are administrable CI concerns; repository consent, reliable signal reconstruction, case review, and a halt on regression-evidence suppression bound the shadow evaluation.","source_ids":["SRC2","SRC3"]}},"screen_survival":true,"world_novelty_boundary":"This bounded public-web screen found adjacent research, CI implementations, and retry-storm controls but cannot establish world novelty, patentability, market size, expert acceptance, or realized value. The surviving boundary is only that the four retained sources did not disclose the proposal's complete combination; unsearched patents, papers, source code, vendor features, and private CI practices may contain closer prior art."}