{"schema_version":1,"experiment_id":"eoa_inverse_innovation_exp04_retrieval_first_paired20_20260802","cell_id":"layer_decay_and_expiration_management__computer_science","round_index":0,"assessments":[{"hypothesis_id":"H1","search_queries":["feature flag expiration TTL stale flags code references automatic cleanup documentation","LaunchDarkly flag expiration code references archive stale flags","Unleash feature flag stale expiration dependency code references","feature flags renewable lease TTL quarantine deletion dependency"],"sources":[{"source_id":"H1_S1","title":"Archiving flags","publisher":"LaunchDarkly","url":"https://launchdarkly.com/docs/home/flags/archive","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["LaunchDarkly identifies flags ready to archive using lifecycle checks.","Code references help determine whether flag-conditioned code remains.","Prerequisite dependencies block archiving.","Archiving is reversible, while permanent deletion requires a later explicit action."]},{"source_id":"H1_S2","title":"Feature flags","publisher":"Unleash","url":"https://docs.getunleash.io/concepts/feature-flags","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["Unleash assigns expected lifetimes by flag type and marks flags potentially stale after that lifetime.","Stale-state events can trigger CI failures or automated cleanup pull requests.","Parent flags with child dependencies cannot be archived until dependencies are resolved."]},{"source_id":"H1_S3","title":"On the Removal of Feature Toggles: A Study of Python Projects and Practitioners Motivations","publisher":"Empirical Software Engineering","url":"https://suhaib.ca/papers/Hoyos_EMSE2021.pdf","source_class":"PRIMARY_RESEARCH","claims_supported":["The study analyzed feature-toggle removal in open-source projects and surveyed practitioners.","It found that 75% of studied toggle components were removed within 49 weeks, while some unintended long-lived toggles remained.","Practitioners reported lifecycle completion and scheduled audits as removal triggers."]}],"closest_analogue":"LaunchDarkly's ready-to-archive workflow, supplemented by Unleash's expected-lifetime stale-state mechanism.","overlap":"Existing products already combine age or expected-lifetime signals, runtime-use checks, code-reference discovery, flag-dependency gates, archival, and restoration. The empirical literature also establishes persistent toggles as a real lifecycle problem.","remaining_difference":"The searched systems do not clearly require every temporary flag to carry a creation-time lease that can remain active only through evidence-backed explicit renewal. Their age signals generally recommend cleanup or mark staleness rather than automatically withholding continued authority pending renewal.","classification":"POSSIBLE_DISTINCTION","disposition":"ADVANCE","rationale":"Most lifecycle components are established, but mandatory renewable leases appear testably stronger than existing advisory lifetime and archive-readiness workflows. A controlled comparison against dashboard-only review remains meaningful."},{"hypothesis_id":"H2","search_queries":["RAG documentation version-aware retrieval superseded chunks stale documentation coding assistant","coding assistant index obsolete documentation chunks reindex version metadata current API","Azure AI Search RAG document version supersede delete old chunks tombstone","LlamaIndex document update delete old chunks ref_doc_id versioning"],"sources":[{"source_id":"H2_S1","title":"VersionRAG: Version-Aware Retrieval-Augmented Generation for Evolving Documents","publisher":"arXiv / Zurich University of Applied Sciences","url":"https://arxiv.org/abs/2510.08109","source_class":"PRIMARY_RESEARCH","claims_supported":["VersionRAG explicitly models technical-document version sequences, content boundaries, and changes between versions.","Its retrieval stage performs version-aware filtering and change tracking.","VersionQA evaluates version-sensitive questions over technical documentation; VersionRAG reports 90% accuracy versus 58% for naive RAG and 64% for GraphRAG."]},{"source_id":"H2_S2","title":"Document Management","publisher":"LlamaIndex","url":"https://llamaindexxx.readthedocs.io/en/latest/module_guides/indexing/document_management.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["LlamaIndex supports document insertion, deletion, update, and refresh.","Updating a document by stable identifier removes or suppresses its prior nodes from retrieval.","Document-to-chunk relationships are tracked through reference-document identifiers."]},{"source_id":"H2_S3","title":"HoH: A Dynamic Benchmark for Evaluating the Impact of Outdated Information on Retrieval-Augmented Generation","publisher":"ACL / arXiv","url":"https://arxiv.org/abs/2503.04800","source_class":"PRIMARY_RESEARCH","claims_supported":["HoH is a benchmark specifically targeting outdated information coexisting with current information in RAG sources.","Its experiments report that outdated evidence reduces answer accuracy and can mislead generation even when current evidence is available."]}],"closest_analogue":"VersionRAG and its VersionQA benchmark.","overlap":"VersionRAG already models successor-like version sequences and document-state changes, filters retrieval by version, targets evolving technical and API documentation, and measures whether version-aware retrieval prevents obsolete answers. LlamaIndex already supplies identity-based replacement and removal of old chunks.","remaining_difference":"The hypothesis additionally leaves a durable chunk-level tombstone that resolves directly to a successor and evaluates generated code for currently supported APIs rather than versioned-document question answering. Those are narrower implementation and evaluation differences.","classification":"OBVIOUS_COLLISION","disposition":"REJECT","rationale":"The central causal claim—explicit supersession or version structure preventing obsolete technical guidance without simply discarding version history—is already embodied and benchmarked by VersionRAG. Successor-resolving tombstones do not create enough separation for this shallow screen."},{"hypothesis_id":"H3","search_queries":["ML model registry disaster recovery restore test archived model artifacts reproducibility","MLflow model registry archived models restore artifacts documentation","machine learning reproducibility archived model environment dataset end-to-end restore test","NIST AI model lineage reproducibility artifacts archive restoration testing"],"sources":[{"source_id":"H3_S1","title":"Model Store Back Up and Retention","publisher":"Oracle Cloud Infrastructure","url":"https://docs.oracle.com/en-us/iaas/Content/data-science/using/model-store-backup-retention.htm","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["OCI supports age-based model archival, later deletion, cross-region backup, and restoration of archived model artifacts before download.","The policy is explicitly presented as a way to control storage expense while retaining disaster-recovery capability."]},{"source_id":"H3_S2","title":"ML Models","publisher":"MLflow","url":"https://mlflow.org/docs/latest/ml/model/index.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["MLflow packages model artifacts with model metadata, dependency files, runtime information, signatures, and input examples.","MLflow provides save, load, log, and dependency-validation mechanisms intended to recreate model environments."]},{"source_id":"H3_S3","title":"Machine Learning Pipelines: Provenance, Reproducibility and FAIR Data Principles","publisher":"arXiv","url":"https://arxiv.org/abs/2006.12117","source_class":"PRIMARY_RESEARCH","claims_supported":["End-to-end ML reproducibility requires versions and relationships for datasets, code, packages, hyperparameters, preprocessing, and other provenance entities.","Missing or outdated components beyond model weights are documented causes of reproduction failure."]},{"source_id":"H3_S4","title":"Automated Modernization of Machine Learning Engineering Notebooks for Reproducibility","publisher":"Proceedings of the ACM on Software Engineering / arXiv","url":"https://arxiv.org/abs/2602.07195","source_class":"PRIMARY_RESEARCH","claims_supported":["Environment erosion makes historical ML artifacts non-reproducible even when code remains available.","The study reports that only 35.4% of 12,720 sampled notebooks remained reproducible and evaluates restoration through actual execution rather than checksums."]}],"closest_analogue":"OCI Model Store Backup and Retention combined with MLflow's self-describing model packages.","overlap":"Existing platforms archive and restore model artifacts, capture model and environment metadata, and support lower-cost lifecycle states. Research directly documents environment erosion and validates reproducibility by executing historical ML workloads.","remaining_difference":"The located platform documentation does not show a recurring operational drill that samples complete release-specific bundles across age and format strata and proves retrieval, parsing, environment creation, dataset and feature reconnection, and bounded-time reproduction end to end.","classification":"POSSIBLE_DISTINCTION","disposition":"ADVANCE","rationale":"Archival, tiering, and reproducibility packaging are established, but the proposed stratified recurring restore audit across a multi-artifact lineage bundle remains a concrete, testable operational distinction."},{"hypothesis_id":"H4","search_queries":["database optimizer plan baseline evolve verify unused plans auto purge retention Oracle documentation","SQL plan management automatic evolve plan baseline benefit verification stale plans","query hints expiration automatic disable measured performance regression database","PostgreSQL query plan hints stale automatic expiration quarantine"],"sources":[{"source_id":"H4_S1","title":"Function reference for Aurora PostgreSQL query plan management","publisher":"Amazon Web Services","url":"https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Optimize.Functions.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["Aurora can execute captured queries to compare each managed plan with other baselines or with using no managed plan.","It records each plan's incremental benefit or disadvantage and a last-verified timestamp.","Managed plans can be reversibly enabled or disabled and assigned approval states."]},{"source_id":"H4_S2","title":"Best practices for Aurora PostgreSQL query plan management","publisher":"Amazon Web Services","url":"https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Optimize.BestPractice.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["Aurora evolves plan baselines by experimentally comparing discovered plans and automatically approving faster plans.","The process is intended to prevent plan regressions when optimizers, schemas, or workloads change."]},{"source_id":"H4_S3","title":"SQL Plan Management in Oracle Database 19c","publisher":"Oracle","url":"https://www.oracle.com/docs/tech/database/technical-brief-sql-plan-mgmt-19c.pdf","source_class":"OFFICIAL_GUIDANCE","claims_supported":["Oracle periodically re-verifies candidate plans using measured execution performance and records last-verification time.","Plan evolution can automatically accept demonstrably better alternatives while retaining reports and reversible enablement controls.","A scheduled task purges unused plan baselines after a configurable retention period, 53 weeks by default."]}],"closest_analogue":"Aurora PostgreSQL Query Plan Management's evolve-plan-baselines experiment and reversible plan disablement.","overlap":"Existing database systems already collect recent verification and usage timestamps, measure the counterfactual benefit of a managed plan—including comparison with no managed plan—reconsider alternatives periodically, enable or disable plans reversibly, and purge unused baselines by retention age.","remaining_difference":"The hypothesis expresses these established signals as a single explicit age-discounted score focused on demoting accepted overrides and proposes a particular tail-latency evaluation. That is primarily a scoring and experimental framing difference.","classification":"OBVIOUS_COLLISION","disposition":"REJECT","rationale":"Aurora and Oracle already provide the substantive machinery: measured plan benefit, periodic revalidation, reversible demotion, and age or nonuse-based retirement. Repackaging those inputs as a decay score is insufficient separation at this screen."},{"hypothesis_id":"H5","search_queries":["CI test quarantine expiry owner expiration date flaky tests quarantine","Buildkite test quarantine expiration date owner flaky test","\"quarantine\" \"expiration\" flaky test CI","test quarantine owner deadline CI official documentation"],"sources":[{"source_id":"H5_S1","title":"Test Quarantine Process","publisher":"GitLab","url":"https://handbook.gitlab.com/handbook/engineering/testing/quarantine-process/","source_class":"COMMERCIAL_FIRST_PARTY","claims_supported":["GitLab treats quarantine as temporary and assigns quarantine merge requests to an identified responsible owner.","Owners must investigate, update the issue weekly, and resolve or remove the test within three months.","If no action is taken, an owning-team approval and three-month deletion countdown govern long-term quarantine."]},{"source_id":"H5_S2","title":"Test state and quarantine","publisher":"Buildkite","url":"https://buildkite.com/docs/pipelines/configure/tests/test-suites/test-state-and-quarantine","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["Buildkite models enabled, muted, and skipped test lifecycle states.","Muted tests continue executing while failures become non-blocking and telemetry remains available.","Workflows can automatically quarantine tests and later transition their state."]},{"source_id":"H5_S3","title":"Quarantine your non-deterministic tests with a time limit","publisher":"Arkency","url":"https://blog.arkency.com/weekly-quarantine/","source_class":"AUTHORITATIVE_SECONDARY","claims_supported":["A working CI pattern encoded an expiration date directly in a quarantined test.","When the time limit elapsed, the test returned to the failing CI path unless the team fixed, removed, or explicitly extended the quarantine."]},{"source_id":"H5_S4","title":"Flaky tests","publisher":"pytest","url":"https://docs.pytest.org/en/stable/explanation/flaky.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["pytest documents non-strict expected failure as a form of manual quarantine.","The official guidance warns that using this mechanism permanently is dangerous because genuine failures may be overlooked."]}],"closest_analogue":"GitLab's owned quarantine lifecycle, together with the established expiring-quarantine CI pattern documented by Arkency.","overlap":"Existing practice already combines non-blocking quarantine states, continued visibility or separate execution, responsible owners, tracking issues, response timelines, periodic progress evidence, fixed resolution windows, and automatic return to failure after an encoded expiry.","remaining_difference":"Class-specific maximum durations, formal preservation-exception categories, and a standardized evidentiary renewal form are more prescriptive governance details, but they do not materially change the established owner-and-expiry quarantine lifecycle.","classification":"OBVIOUS_COLLISION","disposition":"REJECT","rationale":"The core intervention—temporary CI suppression that must be owned, reviewed, fixed, removed, or explicitly extended by a deadline—has direct operational precedents. The remaining differences are policy parameterization."}],"nominated_ids":["H1","H3"],"replenishment_recommended":false}