{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"predictive_residual_processing__mathematics:P4:v0","cell_id":"predictive_residual_processing__mathematics","search_queries":["site:github.com/leanprover-community mathlib port Lean 4 migration status dependencies theorem","proof assistant change impact analysis theorem dependency regression testing paper","Coq incremental compilation dependency graph reverse dependencies official","Isabelle session dependency build system official documentation","site:leanprover-community.github.io mathlib breaking changes maintainers migration deprecation policy","site:github.com/leanprover-community/mathlib4 \"breaking change\" review maintainer","site:rocq-prover.org regression proof selection dependency Coq build official","formal proof library evolution migration dependency impact maintainers paper theorem","site:lean-lang.org/doc/reference latest kernel trusted proof checker axioms declarations dependencies Lean","site:docs.rocq-prover.org coqdep dependency official reference manual","site:leanprover-community.github.io mathlib maintainer review burden formal mathematics library","proof library evolution theorem dependency analysis Isabelle paper","ICOQ Regression Proof Selection for Large-Scale Verification Projects 2017 DOI","Maintaining a Library of Formal Mathematics 2020 DOI CICM","Practical Machine-Checked Formalization of Change Impact Analysis 2020 DOI"],"sources":[{"source_id":"S1","title":"Maintaining a Library of Formal Mathematics","publisher":"Vrije Universiteit Amsterdam / Springer","url":"https://research.vu.nl/en/publications/maintaining-a-library-of-formal-mathematics/","source_class":"PRIMARY_RESEARCH","publication_date":"2020-07-17","accessed_at":"2026-08-03","claims_supported":["Mathlib is a community-maintained formal-mathematics library.","Its developers explicitly sought tools that detect subtle proof-development mistakes and lessen contribution-review burden.","Formal-library maintainers are a plausible beneficiary group, although this source does not measure migration-report burden specifically."]},{"source_id":"S2","title":"mathlib4: The Math Library of Lean 4","publisher":"Leanprover Community","url":"https://github.com/leanprover-community/mathlib4","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"2026-08-03 (live repository snapshot)","accessed_at":"2026-08-03","claims_supported":["Mathlib is an active formal-mathematics corpus with named maintainers and component expertise.","The entire library was migrated from Lean 3 to Lean 4 using mathport.","Dependency updates are managed through versioned manifests and pull requests, demonstrating a concrete migration workflow and adopter setting."]},{"source_id":"S3","title":"RFC: Add left actions and right actions to expression tree elaborator and make ^ be a right action","publisher":"Lean Project","url":"https://github.com/leanprover/lean4/issues/2854","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"2023-11-09","accessed_at":"2026-08-03","claims_supported":["A change to Lean elaboration was explicitly identified as breaking and as affecting mathlib because of library-wide workarounds.","The RFC documents downstream fixes, regressions, timeouts, and elaboration interactions, supporting the existence of nontrivial effects beyond simple textual dependency reachability."]},{"source_id":"S4","title":"Pull Request Review Guide","publisher":"Leanprover Community","url":"https://leanprover-community.github.io/contribute/pr-review.html","source_class":"OFFICIAL_GUIDANCE","publication_date":"Undated; accessed 2026-08-03","accessed_at":"2026-08-03","claims_supported":["Mathlib maintainers are the only users authorized to merge pull requests.","Mathlib reviewers provide review input, establishing identifiable authorizers and a human-review workflow for changes."]},{"source_id":"S5","title":"iCoq: Regression Proof Selection for Large-Scale Verification Projects","publisher":"IEEE/ACM Automated Software Engineering; author-hosted by the University of Texas at Austin","url":"https://users.ece.utexas.edu/~gligoric/papers/CelikETAL17iCoq.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"2017","accessed_at":"2026-08-03","claims_supported":["Large evolving proof projects create a documented need to find failures quickly after revisions.","iCoq tracks fine-grained dependencies among Coq definitions, propositions, and proofs and ignores semantically irrelevant changes.","Its evaluation reported proof-checking speedups of up to 10 times over full checking and up to 3 times over timestamp-based incremental checking.","This is the closest located proof-specific analogue, but it selects proofs to check rather than checking the full corpus and compressing the subsequent human migration review."]},{"source_id":"S6","title":"Practical Machine-Checked Formalization of Change Impact Analysis","publisher":"Springer Nature / PubMed Central","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC7480691/","source_class":"PRIMARY_RESEARCH","publication_date":"2020-04-17","accessed_at":"2026-08-03","claims_supported":["Change-impact analysis based on explicit dependency graphs is established research used in regression selection, build systems, and compilers.","The authors formalized impact analysis in Coq and integrated executable versions with regression-test selection, regression-proof selection, and a build system.","Dependency-based impact analysis has known trustworthiness requirements and is not a novel foundation for the proposal."]},{"source_id":"S7","title":"The Isabelle System Manual (Isabelle2022)","publisher":"Isabelle Project, Technical University of Munich","url":"https://isabelle.in.tum.de/website-Isabelle2022/dist/Isabelle2022/doc/system.pdf","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"2022","accessed_at":"2026-08-03","claims_supported":["Isabelle's established build tooling manages dependencies among sessions, theory sources, auxiliary files, and build images.","It supports descendant and requirement selection, fresh builds, dependency-only status checks, source listings, and complete session builds.","Dependency-aware build and status reporting are established practices adjacent to, but less elaborate than, the proposed predictive residual review ledger."]},{"source_id":"S8","title":"Axioms — The Lean Language Reference","publisher":"Lean Project","url":"https://lean-lang.org/doc/reference/latest/Axioms/","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"2026-08-03 (current manual snapshot)","accessed_at":"2026-08-03","claims_supported":["Lean automatically tracks the axioms on which proofs depend, including transitive dependencies exposed by the print-axioms command.","Axiom additions can undermine proof foundations, so trust-dependency changes warrant protected full-record treatment.","Compiler-trust and generated native-decision axioms demonstrate that trust-boundary metadata is technically extractable but semantically consequential."]}],"problem_evidence":{"support":"MODERATE","rationale":"Formal-library review burden is explicitly acknowledged, large proof projects need faster identification of revision-induced failures, and a concrete Lean elaboration change affected mathlib through library-specific workarounds. These sources establish the broader maintenance problem and show that realized effects can involve elaboration and timeouts. They do not establish the proposal's narrower prevalence claim that full theorem-level migration reports are commonly dominated by predictable records or that human inspection of those records is a binding bottleneck.","source_ids":["S1","S3","S5"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Mathlib provides an identifiable adopter setting: named maintainers oversee the corpus, only maintainers can merge changes, and reviewers participate in approval. The community has expressed a desire to lessen review burden. No source expresses demand for a frozen predictive impact model, residual-only review interface, independent raw audits, or willingness to fund and operate this architecture.","source_ids":["S1","S2","S4"]},"prior_art":{"proximity":"ADJACENT_PRIOR_ART","closest_analogues":[{"name":"iCoq regression proof selection","similarity":"Tracks fine-grained proof dependencies across revisions, identifies affected proofs, ignores semantically irrelevant edits, and was empirically evaluated on large Coq projects.","remaining_difference":"iCoq reduces proof-checking work by selecting proofs. The candidate instead preserves complete checker execution and proposes prediction-plus-typed-residual reconstruction to reduce human review, with protected-event bypasses, random audits, and component fallback.","source_ids":["S5"]},{"name":"Verified dependency-graph change-impact analysis","similarity":"Computes affected components from changes and explicit dependency relations; has a machine-checked formulation and integration with regression-proof selection and build systems.","remaining_difference":"It supplies a trustworthy dependency-analysis substrate but not a learned theorem-state predictor, frozen complete migration ledger, human attention experiment, protected trust-event channel, or raw-record audit scheme.","source_ids":["S6"]},{"name":"Isabelle session build management","similarity":"Maintains dependency-aware build graphs, selects descendants or requirements, lists sources, performs fresh builds, and reports status for large theory collections.","remaining_difference":"It is session-level build and dependency management, not theorem-level prediction of migration outcomes or residual-governed reviewer attention with reconstructible records and blinded audits.","source_ids":["S7"]},{"name":"Mathlib maintenance tooling and review workflow","similarity":"Uses automated checks and documentation to reduce review burden while named maintainers and reviewers govern changes.","remaining_difference":"The located practice does not predict every theorem's migration state or suppress predicted records through an audited residual interface.","source_ids":["S1","S2","S4"]}],"distinctive_claim_remaining":"Relative to full migration reports, static dependency grouping, and iCoq-style impact analysis, a frozen theorem-level predictor plus complete independent checking, typed residual presentation, protected full-record bypasses, random raw-record audits, and component fallback will reduce report volume and reviewer time without reducing protected-event recall, exact ledger reconstruction, or blinded reviewer classification accuracy. This is contrastive and falsifiable; novelty, patentability, freedom to operate, market size, and realized impact remain unmeasured.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"The constituent data and workflow capabilities are credible: fine-grained proof dependencies have been implemented in iCoq; dependency-aware builds are standard in Isabelle; Lean exposes transitive axiom dependencies; mathlib has versioned builds, migrations, maintainers, and review procedures. No located source demonstrates accurate theorem-level prediction of obligation, diagnostic, dependency, trust, timeout, and survival outcomes together, exact prediction-plus-residual ledger reconstruction, or safe human-review compression. The predictor, residual schema, audit-rate calibration, and reviewer interface therefore remain experimental. A read-only archived replay avoids changing canonical proofs and presents no apparent legal or formal-acceptance barrier, subject to repository permission and confidentiality controls for unreleased revisions.","source_ids":["S2","S5","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Large proof revisions and review burden are real, but the amount of human time currently lost to predictable theorem records is unmeasured; impact could be material only for sufficiently large migrations.","source_ids":["S1","S3","S5"]},"stakeholder_pull":{"score":3,"rationale":"Mathlib has identifiable maintainers with merge authority and an expressed interest in lower review burden, but there is no expressed demand or funding commitment for this specific architecture.","source_ids":["S1","S2","S4"]},"incremental_advantage":{"score":3,"rationale":"Keeping full checking while targeting human attention could improve safety relative to iCoq-style proof selection, but superiority over grouped full reports or static impact views has not been tested.","source_ids":["S5","S6","S7"]},"distinctiveness_plausibility":{"score":3,"rationale":"No close match was found for the full frozen-ledger, typed-residual, protected-bypass, audit, and decompression combination; most underlying mechanisms are established and the search cannot establish world novelty.","source_ids":["S5","S6","S7"]},"technical_implementability":{"score":4,"rationale":"Proof dependency extraction, complete builds, revision tracking, trust-dependency inspection, and authoritative review workflows already exist. The uncertain element is prediction quality and schema completeness rather than basic systems feasibility.","source_ids":["S2","S5","S7","S8"]},"adoption_authority_feasibility":{"score":4,"rationale":"Mathlib's maintainers are clearly identifiable authorizers, and a read-only shadow replay can preserve their existing merge and proof-checking authority.","source_ids":["S2","S4"]},"evidence_readiness":{"score":2,"rationale":"An archived replay is well bounded, but no baseline data were found for theorem-level report volume, reviewer time, predictable-record share, protected-event prevalence, or predictor calibration.","source_ids":["S1","S3","S5"]},"safety_net_benefit":{"score":4,"rationale":"Full-corpus checking, protected axiom/trust records, exact reconstruction, random audits, and automatic fallback directly address silent suppression risks, assuming they are implemented and tested as specified.","source_ids":["S5","S8"]},"scalability":{"score":3,"rationale":"Large proof corpora and dependency tooling support computational scalability, but prediction maintenance, theorem-level storage, independent audits, and frequent fallback may erase attention savings across heterogeneous components.","source_ids":["S2","S5","S7"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"One preregistered, read-only replay on an archived migration, including a frozen baseline predictor, typed residual generator, blinded fault insertions, exact reconstruction tests, and a small blinded reviewer study.","confidence":"LOW","assumptions":["One proof engineer and one research/evaluation lead contribute approximately four to eight person-weeks in aggregate.","Existing checker logs and dependency exports are reusable.","No production integration, canonical artifact modification, or paid large-sample recruitment is included."],"source_ids":["S2","S5","S7"]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"Reusable prototype for one proof ecosystem: ledger schema, prediction pipeline, version fingerprints, residual UI, protected-event rules, audit sampler, metrics, and full-report fallback.","confidence":"LOW","assumptions":["Two to four engineers or proof-infrastructure specialists work for roughly three to six months.","The implementation builds on existing batch checking and dependency facilities.","Security certification, multi-prover portability, and production service-level guarantees are excluded."],"source_ids":["S5","S7","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"First operational shadow deployment across several large components, including integration hardening, access controls, observability, reviewer training, independent evaluation, incident playbooks, and maintainer governance.","confidence":"LOW","assumptions":["A small cross-functional team works for six to twelve months.","Full canonical reports remain available and unchanged.","Launch remains advisory and cannot approve migrations or skip checker work."],"source_ids":["S2","S4","S5","S8"]},"annual_recurring":{"band_2026_usd":"50K_TO_250K","scope":"Model refreshes, schema and threshold review, audit sampling, regression testing, infrastructure operation, dependency/checker-version adaptation, and maintainer oversight for one ecosystem.","confidence":"LOW","assumptions":["Approximately 0.5 to 2.0 full-time-equivalent staff plus modest compute and storage.","Most migrations do not trigger prolonged full-component fallback.","Costs exclude the underlying proof-maintenance and complete checker work already required by the baseline."],"source_ids":["S2","S3","S7"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"External primary and first-party evidence establishes formal-library review burden, costly revision regression discovery, and downstream breakage caused by elaboration changes. The specific prevalence and magnitude of redundant migration-report review remain unknown.","source_ids":["S1","S3","S5"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Mathlib maintainers are identifiable corpus stewards and the sole merge authorizers; reviewers form an established supporting workflow.","source_ids":["S2","S4"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The candidate can be compared against full-report review, static dependency grouping, and iCoq-style impact views on exact reconstruction, protected recall, reviewer accuracy, time, report volume, fallback, and total cost.","source_ids":["S5","S6","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"A single archived, non-release-blocking migration replay with a frozen model, untouched outcomes, blinded insertions, and explicit comparators and falsifiers is bounded and reversible.","source_ids":["S2","S4","S5"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"A shadow replay can retain complete checker execution, canonical reports, protected axiom and trust records, and maintainer approval authority. Repository authorization and confidentiality must be confirmed before using nonpublic migrations, but no inherent legal or safety stop was found.","source_ids":["S4","S8"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"The four estimates have bounded scopes and transparent staffing assumptions, but no direct wage, vendor, or comparable deployment-cost evidence was located; predictor complexity and fallback frequency could move costs by more than one band.","source_ids":["S2","S5","S7"]}},"next_evidence_step":"With mathlib maintainer authorization, preregister one read-only replay of an archived, non-release-blocking migration containing at least 500 eligible declarations and complete retained checker records. Freeze the impact model, residual taxonomy, protected classes, thresholds, and audit sample before revealing an untouched evaluation partition. Randomly assign blinded reviewers to: (A) the complete theorem-level migration report, (B) the complete report grouped by static dependency reachability and outcome, or (C) the proposed residual interface with reconstruction and fallback. Insert blinded cases covering unexpected failure and survival, dependency addition and removal, removed obligations, admitted facts or axiom changes, checker timeouts, missing outputs, manifest gaps, and version mismatch. Require 100% protected-event recall, zero missing-as-success errors, and exact canonical ledger reconstruction. Falsify the intervention if any protected case is suppressed; any ledger cannot be reconstructed; any consequential mismatch appears in the independent random audit but not the routed/fallback view; the lower confidence bound for reviewer classification accuracy is more than five percentage points below the best comparator; median review time or report volume improves by less than 20%; structured residual drift fails to trigger fallback; or model, audit, and operating cost is not lower than baseline reviewer cost at equal fidelity. Success authorizes only a second shadow study, not selective checking or migration approval.","blocking_evidence":["No measured prevalence of foundational migrations or theorem-by-theorem migration reports was found.","No evidence quantifies what share of migration records is predictable repetition or whether reviewer attention is currently binding.","No theorem-level predictor has demonstrated calibration across failures, unexpected survivals, obligations, realized dependencies, diagnostics, trust changes, timeouts, and missingness.","No human study compares residual presentation with full or dependency-grouped migration reports.","Audit rates needed to detect rare consequential blind spots are unknown.","No direct cost benchmarks or maintainer funding commitment were found.","Repository permission and confidentiality requirements for unreleased foundational revisions remain deployment-specific."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"This eight-source evaluation found established proof-specific regression selection, verified dependency-graph impact analysis, dependency-aware proof-library builds, trust-dependency inspection, and formal-library review tooling. It did not find the exact combination of complete checking, frozen theorem-state prediction, typed residual review, exact ledger reconstruction, protected full-record bypasses, independent raw audits, and drift-triggered decompression. That absence is not a world-novelty, patentability, freedom-to-operate, or market finding; those remain unmeasured.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":false,"progress_targets":["Obtain a named formal-library maintainer partner and authorization for one archived shadow replay.","Measure baseline report size, reviewer time, predictable-record share, and protected-event prevalence on the selected migration.","Pre-register the three comparator interfaces, untouched evaluation partition, audit sampling, and all falsification thresholds.","Demonstrate exact ledger reconstruction, 100% protected-event recall, and zero missing-as-success outcomes.","Establish blinded noninferiority of reviewer classification accuracy and a material reduction in review time or report volume.","Quantify fallback frequency, audit disagreement, model-maintenance effort, and total resource-equivalent cost."],"reason":"Bounded web research establishes the broader problem, credible authorizers, adjacent prior art, and technical plausibility, but cannot answer the decisive claim. Whether residual presentation safely reduces reviewer burden requires access to a real migration corpus, construction and calibration of a predictor, blinded human review, injected or naturally occurring rare cases, independent audits, and live cost measurement. Those are fieldwork and testing requirements, so empirical research—not further general web search—is the next decision point."},"proposal_index":4}