{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"bounded_rivalry_governance__aviation_aeronautics:P3:v0","cell_id":"bounded_rivalry_governance__aviation_aeronautics","search_queries":["site:oig.dot.gov aviation contract maintenance repair stations oversight air carriers report repeat repairs documentation","site:faa.gov AC 120-16G reliability program contract maintenance repair station operator","aircraft rotable component repair contract performance repeat removal reliability MRO supplier study","airline component support contract guaranteed reliability turnaround time repair first party","site:faa.gov AC 120-79A developing implementing air carrier continuing analysis surveillance system reliability program component removal data","ATA Spec 2000 aircraft reliability data component removal official","site:easa.europa.eu Part-M reliability programme contracted maintenance operator responsibility continuing airworthiness","aircraft repair vendor scorecard repeat removal rate component MRO reliability procurement","site:oig.dot.gov \"contract maintenance\" air carrier oversight repair stations report 2015 2020","site:gao.gov aviation repair stations maintenance outsourcing oversight airlines safety report","site:ntsb.gov aircraft accident improper component overhaul repair station maintenance documentation","FAA enforcement repair station falsified maintenance records component repair","\"Impact of Performance-based Contracting on Product Reliability\" publication year authors"],"sources":[{"source_id":"S1","title":"Aviation Safety: FAA's Risk-Based Oversight for Repair Stations Could Benefit from Additional Airline Data and Performance Metrics (GAO-16-679)","publisher":"U.S. Government Accountability Office","url":"https://www.gao.gov/products/gao-16-679","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2016-07-28","accessed_at":"2026-08-03","claims_supported":["For 28 selected U.S. airlines, contracted maintenance represented 58–64 percent of annual maintenance spending during 2010–2014.","Representatives of 10 airlines identified repair-station service quality, cost, and OEM service contracts as important maintenance-contracting factors.","FAA oversight lacked some repair-station workload data and effectiveness metrics, demonstrating a recognized data and performance-measurement gap."]},{"source_id":"S2","title":"AC 120-16G: Air Carrier Maintenance Programs","publisher":"Federal Aviation Administration","url":"https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_120-16G.pdf","source_class":"OFFICIAL_GUIDANCE","publication_date":"2016-01-04","accessed_at":"2026-08-03","claims_supported":["Air carriers retain direct and primary responsibility for maintenance performed by outside maintenance providers.","Contract-maintenance policies must keep work under the certificate holder's supervision and control and include continuing analysis and surveillance procedures.","FAA guidance calls for evaluating new providers, continuing provider oversight, records, inspection requirements, and written arrangements for recurring contract maintenance.","Maintenance-program safety and airworthiness requirements cannot be displaced by commercial scoring."]},{"source_id":"S3","title":"AC 120-79A Change 1: Developing and Implementing an Air Carrier Continuing Analysis and Surveillance System","publisher":"Federal Aviation Administration","url":"https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_120-79A_CHG_1.pdf","source_class":"OFFICIAL_GUIDANCE","publication_date":"2010-09-07","accessed_at":"2026-08-03","claims_supported":["Air-carrier reliability and CASS programs use operational data collection, analysis, corrective action, statistical performance standards, reporting, and process changes.","Relevant operational data can include pilot reports, mechanical delays, condition monitoring, and teardown reports.","FAA notes that full reliability-program data requirements can exceed the resources of smaller and some medium-sized operators."]},{"source_id":"S4","title":"Easy Access Rules for Continuing Airworthiness (Regulation (EU) No 1321/2014), Revision from September 2025","publisher":"European Union Aviation Safety Agency","url":"https://www.easa.europa.eu/en/document-library/easy-access-rules/online-publications/easy-access-rules-continuing-airworthiness?erules-id=ERULES-1963177438-16440","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2025-09","accessed_at":"2026-08-03","claims_supported":["A CAMO bears responsibility for the airworthy condition of managed aircraft and must retain control over contracted maintenance.","The CAMO and operator should agree on the maintenance-organisation selection process.","Maintenance contracts should specify responsibilities and work unambiguously, and contracting does not prevent facility-level checking of work.","The rules expressly recognize component-maintenance arrangements involving multiple appropriately approved organisations."]},{"source_id":"S5","title":"Impact of Performance-Based Contracting on Product Reliability: An Empirical Analysis","publisher":"INSEAD / Management Science","url":"https://www.insead.edu/faculty-research/publications/journal-articles/impact-performance-based-contracting-product","source_class":"PRIMARY_RESEARCH","publication_date":"2012-02-10","accessed_at":"2026-08-03","claims_supported":["Using proprietary aircraft-engine manufacturer data and an econometric model addressing contract-choice endogeneity, the study estimated 25–40 percent higher reliability under performance-based contracts than time-and-material contracts.","The results support the proposition that maintenance-contract incentives can affect reliability through maintenance frequency and care.","The study is engine-level and does not test matched rotable cohorts, recurring multi-award allocation, or the candidate's complete governance bundle."]},{"source_id":"S6","title":"Component Repair and Overhaul","publisher":"Lufthansa Technik","url":"https://www.lufthansa-technik.com/en/component-repair-overhaul","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"not stated","accessed_at":"2026-08-03","claims_supported":["Lufthansa Technik markets closed-loop component repair and overhaul with reliability engineering and total-cost-of-ownership improvement.","Its Total Component Maintenance service combines per-flight-hour pricing, reliability improvement, logistics, modification monitoring, warranty handling, and guaranteed availability.","This is close commercial prior art for lifecycle-oriented component support, although the page does not describe matched supplier cohorts or rolling performance-based work-share competition."]},{"source_id":"S7","title":"Spec 2000: Electronic Logbook (Chapter 17), Revision 2018.1","publisher":"Airlines for America","url":"https://publications.airlines.org/products/spec-2000-electronic-logbook-ch-17-revision-2018-1","source_class":"STANDARD","publication_date":"2018 revision; exact date not stated","accessed_at":"2026-08-03","claims_supported":["Spec 2000 supplies an industry data model and exchange methods for electronic-logbook information across airline maintenance systems.","The revision supports aircraft and component utilization, replacement-part information, on-wing repair, and electronic work-package interoperability.","The standard supports technical plausibility of exchanging some required data but does not establish that any operator has complete serial-level repair-to-field-outcome linkage."]},{"source_id":"S8","title":"Component on Demand","publisher":"Airbus","url":"https://www.aircraft.airbus.com/en/services/maintain/flight-hour-services/component-on-demand","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"not stated","accessed_at":"2026-08-03","claims_supported":["Airbus offers component exchange, loan, and repair services with component traceability and aircraft records.","The offering includes a one-year warranty without flight-cycle or flight-hour limits and emphasizes rapid availability.","Traceability and warranty are established commercial practices, but the page does not disclose comparative supplier scoring or recurring portfolio allocation."]}],"problem_evidence":{"support":"MODERATE","rationale":"Contract maintenance is economically material, and airlines expressly balance quality and cost while regulators identify oversight-data and performance-metric gaps. Primary research further shows that maintenance-contract incentives can materially affect aerospace-product reliability. These sources support the general problem of weak coupling between visible contract terms and lifecycle performance, but they do not establish the prevalence of the candidate's specific alleged tactics—selective rejection, narrowed inspections, understated exclusions, or minimally compliant repairs—in rotable procurement.","source_ids":["S1","S2","S5"]},"stakeholder_evidence":{"support":"STRONG","rationale":"Air carriers and CAMOs are identifiable adopters and authorizers: FAA and EASA guidance assigns them continuing responsibility and control over contracted maintenance and provider selection. Ten airlines reported that quality and cost influence contracting, while Lufthansa Technik and Airbus sell reliability-, warranty-, availability-, and traceability-oriented component services, demonstrating expressed customer demand for lifecycle performance rather than price alone.","source_ids":["S1","S2","S4","S6","S8"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"Performance-based aerospace maintenance contracts","similarity":"They change maintenance-provider incentives by tying compensation to realized performance or reliability rather than individual repair inputs.","remaining_difference":"The empirical source concerns aircraft-engine support contracts and does not test matched rotable cohorts, delayed serial-outcome scoring, secured per-unit reserves, or recurring multi-supplier portfolio allocation.","source_ids":["S5"]},{"name":"Lufthansa Technik Total Component Maintenance","similarity":"Closed-loop component support combines per-flight-hour cost limits, reliability engineering, total-cost-of-ownership attention, logistics, warranty management, and availability commitments.","remaining_difference":"It is a provider-managed service offering, not an operator-run rivalry among independently scored repair shops with matched lots and recurring challenger access.","source_ids":["S6"]},{"name":"Airbus Component on Demand","similarity":"Established component repair/exchange practice includes traceability, aircraft records, warranty coverage, and rapid availability.","remaining_difference":"No evidence on the page shows field-outcome holdbacks, cross-shop matched cohorts, anti-collusion screening, or marginal work-share reallocation.","source_ids":["S8"]},{"name":"FAA/EASA contract-maintenance oversight and reliability systems","similarity":"Existing governance already requires provider evaluation, continuing oversight, operational-data analysis, clear responsibilities, approved maintenance, and operator/CAMO control.","remaining_difference":"These safety-governance systems do not prescribe the candidate's commercial tournament, matched-case score, bond, or multi-award allocation rule.","source_ids":["S2","S3","S4"]}],"distinctive_claim_remaining":"For two prespecified rotable families, relative to a lowest-evaluated-price/turnaround tender and a conventional fleet-level performance contract, adding blinded matched repair cohorts, noncompensable compliance gates, a held-back serial-level outcome score, secured warranty responsibility, and recurring multi-award allocation will produce stable risk-adjusted supplier rankings and reduce repeat-removal, documentation-correction, and unrecovered-warranty burden without increasing difficult-case rejection, total landed cost, safety noncompliance, or supplier concentration. Each endpoint and attribution rule is falsifiable; no effect is presently established.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"FAA CASS/reliability guidance demonstrates that airlines can collect and analyze maintenance operational data, A4A provides interoperable logbook and component-utilization structures, and FAA/EASA rules provide an authority framework for provider oversight. Existing component services demonstrate traceability, warranty, reliability engineering, and lifecycle-pricing workflows. Feasibility remains conditional on proprietary serial linkage, comparable service exposure, sufficient event counts, defensible causal attribution, data-protection controls, contract amendments, reserve legality, and supplier participation.","source_ids":["S2","S3","S4","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":4,"rationale":"Contract maintenance constitutes a large share of airline maintenance spending, while maintenance quality and reliability affect operational disruption and safety. Realized impact of this intervention is unmeasured.","source_ids":["S1","S2","S5"]},"stakeholder_pull":{"score":4,"rationale":"Operators have explicit regulatory responsibility and report quality and cost as contracting concerns; commercial offerings prominently sell reliability, warranty, traceability, and availability.","source_ids":["S1","S2","S4","S6","S8"]},"incremental_advantage":{"score":3,"rationale":"The bundle could improve attribution, comparison, resilience, and post-award accountability beyond ordinary tenders, but performance-based contracting and closed-loop lifecycle component services already address much of the central incentive problem.","source_ids":["S5","S6"]},"distinctiveness_plausibility":{"score":2,"rationale":"The precise matched-cohort, score-holdback, reserve, and recurring portfolio combination was not located, but its principal elements are recombinations of established reliability programs, performance contracts, warranty practices, oversight, and multisupplier procurement.","source_ids":["S2","S3","S5","S6","S8"]},"technical_implementability":{"score":3,"rationale":"Relevant data models and reliability-analysis practices exist, but serial linkage, censoring, case-mix balance, exposure normalization, rare events, and shop attribution may defeat valid ranking at a specific operator.","source_ids":["S3","S7"]},"adoption_authority_feasibility":{"score":3,"rationale":"The operator/CAMO can select and oversee maintenance providers, but prospective scoring, reserves, data access, appeals, and work-share changes require coordinated procurement, engineering, quality, legal, and possibly regulator acceptance.","source_ids":["S2","S4"]},"evidence_readiness":{"score":2,"rationale":"The proposed first step is methodologically bounded, but its necessary work-order, serial, service-exposure, warranty, and supplier data are proprietary and their completeness is unknown.","source_ids":["S3","S7"]},"safety_net_benefit":{"score":4,"rationale":"A read-only shadow study, noncompensable safety gates, preserved original records, multi-source allocation, and explicit halt conditions limit first-step harm. The eventual commercial mechanism could still create coding or case-selection incentives.","source_ids":["S2","S4"]},"scalability":{"score":3,"rationale":"Standards and existing airline reliability systems permit replication, but only component families with adequate comparable volume, outcome maturity, portable data, and at least two qualified suppliers are suitable.","source_ids":["S3","S7"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"A 12–20 week retrospective shadow study for two component families: data inventory, de-identification, linkage, preregistration, matching, sensitivity analysis, engineering review, procurement review, and an independent methods audit. Excludes live routing and underlying repair expenditure.","confidence":"LOW","assumptions":["One operator supplies an analysis-ready extract under an existing data-use authority.","The study uses existing records and no new hardware.","A small team includes a maintenance reliability engineer, data analyst/statistician, procurement specialist, and part-time legal/privacy and audit review.","The band is a resource-equivalent estimate, not a vendor quote."],"source_ids":["S3","S7"]},"initial_deployment_startup":{"band_2026_usd":"250K_TO_1M","scope":"Design and contracting for a prospective two-family pilot: rulebook, data interfaces, supplier amendments, case-allocation controls, reserve mechanism, audit protocol, appeals, security/privacy controls, and regulator/quality review. Excludes repair invoices and reserve principal.","confidence":"LOW","assumptions":["Existing maintenance and procurement systems can be extended rather than replaced.","Two or three already-approved shops participate.","No aircraft maintenance instructions or airworthiness approvals are altered.","External legal, actuarial, and independent scoring support is required."],"source_ids":["S2","S3","S4","S7"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"First live allocation period for two component families, including supplier onboarding, matched-lot administration, scoring operations, outcome holdback administration, audits, appeal capacity, and post-period review. Excludes normal repair spend, inventory, and bond or warranty-reserve principal.","confidence":"LOW","assumptions":["Launch uses existing approved shops and component logistics.","The operator already captures most required serial and work-order data.","Rare-event assurance remains in existing safety programs rather than the commercial score.","No custom enterprise platform is procured."],"source_ids":["S2","S4","S6","S8"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Annual resource equivalent for data quality, independent scoring, supplier audits, reserve and appeal administration, challenger qualification, metric-governance review, cybersecurity/privacy controls, and periodic rule recalibration across a small component portfolio. Excludes repair volume and reserve principal.","confidence":"LOW","assumptions":["Two allocation cycles per year or less.","A small permanent analytics/governance function is supplemented by independent audit.","Scaling to many fleets or component families could exceed this band.","FAA guidance that full reliability programs can be resource-intensive is treated as a caution, not a cost benchmark."],"source_ids":["S2","S3"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"The economic importance of outsourced maintenance, the joint importance of service quality and cost, documented oversight-data gaps, and evidence that contract form affects aerospace reliability support the underlying problem. The frequency of the candidate's specific gaming behaviors remains unverified.","source_ids":["S1","S2","S5"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"An air carrier's procurement authority and its continuing-airworthiness, engineering, and quality functions are identifiable adopters; FAA and EASA guidance assigns the operator/CAMO responsibility and control over contracted maintenance and provider selection.","source_ids":["S1","S2","S4"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The candidate can be compared against lowest-price/turnaround, fixed-share, and fleet-level performance-contract rules using predefined linkage, matching, ranking-stability, repeat-removal, documentation, warranty, rejection, cost, and concentration endpoints.","source_ids":["S3","S5","S6"]},"bounded_next_evidence_step":{"status":"YES","reason":"A read-only retrospective study of two component families can be time-boxed, preregistered, blinded, and stopped without changing live maintenance, supplier standing, payments, or aircraft decisions.","source_ids":["S2","S3","S7"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"For the shadow study, original maintenance and procurement decisions remain controlling, compliance is noncompensable, and anomaly screens produce no misconduct findings. Prospective deployment would require separate contractual and authority review.","source_ids":["S2","S4"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"The four estimates state inclusions, exclusions, and resource assumptions, but no direct benchmark was found for operator data integration, independent aviation scoring, contract amendments, reserve administration, or appeals. Actual costs depend heavily on data quality and enterprise systems.","source_ids":["S3","S7"]}},"next_evidence_step":"With one operator under a data-use agreement, preregister a 16-week, read-only shadow study covering exactly two rotable families, no more than 1,000 closed work orders, a fixed historical repair window, and a fixed subsequent service-observation window. Before unblinding shops, define compliance gates, incoming-condition and exposure covariates, attribution rules, censoring treatment, landed-cost and field-outcome weights, reserve simulation, and a portfolio rule. Compare hypothetical allocations with (1) recorded lowest-evaluated-price/turnaround decisions, (2) fixed work shares, and (3) a fleet-level performance-contract rule. Report linkage yield, unmatched cases, difficult-case rejection, repeat removals, documentation corrections, warranty recovery, rank uncertainty, concentration, and sensitivity across all predeclared matching specifications. Falsify progression if required serial/exposure linkage falls below the preregistered data-adequacy threshold, statistical power is inadequate, rankings reverse across reasonable predeclared specifications, attribution cannot separate shop effects from installation and operations, the rule rewards easier case mix, or it provides no decision-relevant improvement over the comparators. Do not route live components or alter supplier status based on results.","blocking_evidence":["No external source establishes how prevalent price- or turnaround-induced inspection narrowing, strategic rejection, exclusion understatement, or minimally compliant repair is in rotable contracting.","The completeness and accuracy of operator serial-level work-order, installation, removal, service-exposure, warranty, and documentation-correction data are unknown.","Causal attribution of repeat removal to a shop rather than aircraft condition, installation, troubleshooting, route, environment, or latent design defects is unvalidated.","Adequate sample size and observation duration for rare but consequential component failures are unknown.","Contractual enforceability and jurisdiction-specific treatment of score holdbacks, warranty reserves, reallocations, audit access, and supplier appeals require operator-specific legal review.","Supplier willingness, distributional effects on smaller approved shops, and the risk of strategic fault coding or difficult-case rejection require field evidence.","The proposed cost ranges lack external implementation quotations or operator labor estimates."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The bounded search found substantial adjacent and colliding practice—performance-based aerospace maintenance contracts, closed-loop component services, reliability programs, warranty and traceability offerings, and formal contracted-maintenance oversight—but did not locate the exact full combination of matched repair cohorts, delayed serial-outcome scoring, secured per-unit responsibility, recurring multi-award allocation, challenger access, and collusion screening. Absence from eight sources is not evidence of world novelty. World novelty, patentability, freedom to operate, market size, and realized impact remain unmeasured.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Obtain an operator data-use agreement and complete a blinded field-level inventory of work-order, serial, exposure, warranty, removal, and documentation data.","Preregister data-adequacy thresholds, matching variables, attribution rules, outcome windows, comparators, decision metrics, and falsifiers before supplier identities are revealed.","Execute the bounded retrospective shadow study and publish all unmatched cases, censoring, uncertainty intervals, and ranking-sensitivity results.","Establish whether the exact alleged procurement-gaming behaviors are prevalent enough to justify intervention, rather than merely theoretically possible.","If retrospective thresholds pass, obtain operator legal, procurement, continuing-airworthiness, quality, privacy, and supplier review for a separately authorized prospective pilot and independent cost quotation."],"reason":"Web evidence establishes a meaningful contracting domain, an empowered adopter, established lifecycle-performance contracting, and a technically plausible shadow-analysis path. It cannot determine the candidate's incremental effect, behavior prevalence, attribution validity, data adequacy, supplier response, or implementable cost. Those questions require proprietary operator records and, ultimately, prospective field testing; therefore the evaluation must stop for empirical research."},"proposal_index":3}