{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"predictive_residual_processing__aviation_aeronautics:P2:v0","cell_id":"predictive_residual_processing__aviation_aeronautics","search_queries":["FAA composite aircraft ultrasonic inspection C-scan repeat inspection guidance NDI manual","phased array ultrasonic composite aircraft change detection prior scan C-scan subtraction research","automated defect recognition aircraft composites ultrasonic C-scan inspector workload human factors","commercial ultrasonic inspection software compare C-scans change detection composites","site:sae.org NAS410 nondestructive testing personnel qualification certification aerospace","FAA human factors nondestructive inspection reliability visual data review inspector workload NDI","NASA ultrasonic C-scan composites automated inspection human inspector","EASA AI roadmap aviation maintenance inspection machine learning authority human oversight","\"Automated defect detection for ultrasonic inspection of CFRP aircraft components\" PDF","site:publications-cnrc.canada.ca \"Automated defect detection\" CFRP"],"sources":[{"source_id":"S1","title":"AC 43-214A: Repairs and Alterations to Composite and Bonded Aircraft Structure","publisher":"U.S. Federal Aviation Administration","url":"https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_43-214A.pdf","source_class":"OFFICIAL_GUIDANCE","publication_date":"2016-07-23","accessed_at":"2026-08-03","claims_supported":["Ultrasonic inspection, including A-, B-, and C-scan displays, is an accepted NDI technique for aircraft composite structures.","Maintenance organizations include operators' facilities, MROs, and certificated repair stations.","Changes to FAA-approved procedures, processes, or data require FAA approval."]},{"source_id":"S2","title":"AC 65-31B: Training, Qualification, and Certification of Nondestructive Inspection Personnel","publisher":"U.S. Federal Aviation Administration","url":"https://www.faa.gov/regulations_policies/advisory_circulars/index.cfm/go/document.information/documentID/1023552","source_class":"OFFICIAL_GUIDANCE","publication_date":"2014-02-24","accessed_at":"2026-08-03","claims_supported":["Aircraft NDI personnel require documented experience, training, qualification, examination, and certification.","Organizations should maintain written qualification and certification programs.","Technical adequacy remains the responsibility of qualified personnel and their supervising organizations."]},{"source_id":"S3","title":"PRC-6501 Rev. G: Process Specification for Ultrasonic Inspection of Composites","publisher":"NASA Johnson Space Center","url":"https://www.nasa.gov/wp-content/uploads/2023/03/prc-6501-current.pdf","source_class":"STANDARD","publication_date":"2020-01-07","accessed_at":"2026-08-03","claims_supported":["The specification applies to in-process, final, and in-service ultrasonic inspection of composite materials.","It requires calibration controls, traceable records for high-consequence inspections, complete coverage unless the design authority specifies otherwise, and manual A-scan verification of indications.","Inspection reports and relevant C-scans are retained as permanent quality records, and deviations require appropriate engineering authority."]},{"source_id":"S4","title":"Automated defect detection for ultrasonic inspection of CFRP aircraft components","publisher":"NDT & E International / Elsevier","url":"https://www.researchgate.net/publication/352100302_Automated_Defect_Detection_for_Ultrasonic_Inspection_of_CFRP_Aircraft_Components","source_class":"PRIMARY_RESEARCH","publication_date":"2021-06-03","accessed_at":"2026-08-03","claims_supported":["High-resolution automated ultrasonic inspection of large aircraft composite structures produces tremendous data volumes whose interpretation remains time-consuming, costly, and dependent on highly trained operators.","A commercial-software-based automated analysis pipeline already used geometry-aware segmentation, reference-envelope subtraction, C-scan reconstruction, and A-scan defect extraction.","Coupling problems, synchronization errors, holes, ply drop-offs, stringers, and other geometry produced false calls or required special handling.","Testing reported an a90/95 of 6.8 mm, approximately threefold analysis-time reduction on one component family, and projected fivefold reduction on another, demonstrating substantial adjacent prior art but not the proposed repeat-scan reader workflow."]},{"source_id":"S5","title":"Human-machine collaborative automation strategies for ultrasonic phased array data analysis of carbon fibre reinforced plastics","publisher":"NDT & E International / University of Strathclyde","url":"https://pureportal.strath.ac.uk/en/publications/human-machine-collaborative-automation-strategies-for-ultrasonic-/","source_class":"PRIMARY_RESEARCH","publication_date":"2025-09-30","accessed_at":"2026-08-03","claims_supported":["Manual or rule-based NDE analysis remains common and can be inefficient and error-prone under complex, noisy conditions.","AI-assisted through supervisory automation workflows were tested on two complex CFRP specimens, detecting 36 manufactured defects.","Multiple-model analysis improved F1 by up to 17.2% over single models and processed the two samples in 94.03 and 57.01 seconds, while the authors describe manual inspection of large components as taking hours.","Industrial adoption remains limited by model trust, black-box behavior, and sparse practical workflow guidance."]},{"source_id":"S6","title":"Automated Defect Evaluation Software for Ultrasonic NDI","publisher":"M.C. Gill Composites Center, University of Southern California","url":"https://composites.usc.edu/automated-defect-evaluation-software-for-ultrasonic-ndi/","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["USC states that expanded aerospace-composite use creates strong demand for manufacturing and in-service NDI.","The project reports a shortage of qualified ultrasonic NDI personnel and describes automated C-scan evaluation as addressing an urgent aerospace need.","Korean Air, Korea Aerospace University, and Inha University are identified project partners, providing identifiable potential adopter and research-partner classes."]},{"source_id":"S7","title":"TomoView Ultrasonic Acquisition and Analysis Software","publisher":"Evident Scientific","url":"https://ims.evidentscientific.com/en/products/software/tomoview","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["Commercial offline ultrasonic-analysis software already supports UT and phased-array data, multiple-file and C-scan merging, filtering, SNR analysis, indication annotation, reporting, A-scan resynchronization, and composite-inspection applications.","Existing products therefore supply much of the workstation, visualization, provenance, and raw-data-access substrate.","The product page does not document a versioned expected-scan residual queue, independent hidden raw-tile audits, or risk-triggered decompression."]},{"source_id":"S8","title":"Artificial Intelligence Roadmap: A human-centric approach to AI in aviation","publisher":"European Union Aviation Safety Agency","url":"https://www.easa.europa.eu/en/domains/research-innovation/ai","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2023-05-10","accessed_at":"2026-08-03","claims_supported":["EASA maintains a human-centric aviation-AI program addressing safety, ethics, trustworthiness, conceptual guidance, and anticipated rulemaking.","The regulatory framework is still evolving through research, stakeholder discussion, guidance, and rulemaking activity.","A human-assistance system is more authority-plausible than an autonomous structural-disposition system, but this roadmap does not itself approve the proposed application."]}],"problem_evidence":{"support":"STRONG","rationale":"The problem is visible at the general task level: primary research describes tremendous ultrasonic data volumes, manual analysis that is time-consuming and costly, complex geometry and acquisition effects that create false calls, and large-component review that can take hours. USC independently reports qualified-personnel scarcity and urgent demand. What remains unmeasured is the prevalence and magnitude of the narrower bottleneck in eligible repeat in-service panel inspections, rather than manufacturing inspection generally.","source_ids":["S4","S5","S6"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Identifiable adopters and funders exist: FAA-recognized maintenance organizations and operators are the authority-bearing user class; Korean Air participates in an automated C-scan evaluation project; Bombardier contributed to the closest primary study; and Spirit AeroSystems supported the 2025 research. These demonstrate institutional pull for automated ultrasonic analysis, but no source expresses demand for this exact repeat-scan residual-first, hidden-audit workflow or commits data, budget, or approval.","source_ids":["S1","S4","S5","S6"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"Ultis-based automated CFRP defect-analysis pipeline","similarity":"Automates analysis of full ultrasonic aircraft-composite datasets using geometry-aware C-scan projection, segmentation, reference-envelope subtraction, reconstructed C-scans, defect extraction, and false-call controls.","remaining_difference":"It targets automated defect detection on production components, not qualified-reader attention ordering across repeat inspections with a versioned prior-scan model, concealed raw audit tiles, reconstruction checks, and mandatory decompression against a full-review comparator.","source_ids":["S4"]},{"name":"Human-machine collaborative multi-model PAUT analysis","similarity":"Combines supervised, unsupervised, and self-supervised models in AI-assisted through supervisory NDE workflows and directly measures detection and analysis time.","remaining_difference":"It does not establish repeat-scan residual representation, model-version synchronization, independent sampling of suppressed raw regions, or noninferiority of a residual-first reader interface.","source_ids":["S5"]},{"name":"USC Automated Defect Evaluation Software","similarity":"Explicitly develops automated evaluation of ultrasonic C-scan images for aerospace structures in response to workforce and inspection demand.","remaining_difference":"The public description does not document prediction-plus-residual reconstruction, repeat-scan registration governance, concealed raw audits, or regional/full-scan fallback.","source_ids":["S6"]},{"name":"Evident TomoView","similarity":"A commercial offline inspection workstation already supports phased-array and conventional UT, C-scan merging, filters, SNR analysis, reporting, resynchronization, and raw-data analysis.","remaining_difference":"The documented product features do not provide a model-governed expected-scan suppression architecture or test the claimed attention and safety benefit against fixed registered differences and full review.","source_ids":["S7"]}],"distinctive_claim_remaining":"For an eligible, provenance-complete repeat inspection, a frozen versioned expected-scan model plus uncertainty- and consequence-weighted residual ordering, concealed independent raw-tile insertion, and mandatory decompression will reduce qualified-reader review burden relative to complete traversal while remaining noninferior on protected indication capture and outperforming a simpler registered-difference display after all modeling, audit, fallback, and configuration-control costs are counted. This fails if either comparator meets the same preregistered safety and attention criteria with lower total burden.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"Offline implementation is technically credible because commercial workstations already expose C-scans, raw ultrasonic data, filtering, resynchronization, and reporting, while primary studies demonstrate automated geometry-aware and multi-model analysis. NASA and FAA guidance show that calibration, coverage, traceability, qualified-personnel authority, permanent records, and engineering-controlled deviations can be specified. However, no source validates cross-inspection registration fidelity, reconstruction tolerances, hidden audit sampling, fallback thresholds, reader anchoring effects, or performance on natural in-service changes. Operational procedure approval remains organization- and jurisdiction-specific.","source_ids":["S1","S2","S3","S4","S5","S7","S8"]},"scores":{"meaningful_impact":{"score":4,"rationale":"If noninferiority is maintained, concentrating attention could materially reduce review time and false-call handling in a documented high-data-volume task. The exact eligible repeat-inspection burden has not been quantified.","source_ids":["S4","S5","S6"]},"stakeholder_pull":{"score":3,"rationale":"Airlines, manufacturers, universities, and aerospace funders visibly support automated ultrasonic analysis, but no adopter has requested or committed to this exact intervention.","source_ids":["S4","S5","S6"]},"incremental_advantage":{"score":2,"rationale":"Automated C-scan analysis, reference subtraction, reconstruction, geometry segmentation, AI assistance, and commercial offline workstations are already mature analogues. Advantage rests on an untested governance and reader-ordering combination.","source_ids":["S4","S5","S7"]},"distinctiveness_plausibility":{"score":3,"rationale":"No opened source documented the complete combination of repeat-scan expected modeling, reconstructive signed residuals, consequence-weighted reader ordering, concealed independent raw audits, and decompression. Absence in eight sources is not world-novelty evidence.","source_ids":["S4","S5","S6","S7"]},"technical_implementability":{"score":3,"rationale":"Core data-processing and interface components exist, but robust registration across inspections, configuration compatibility, natural-defect generalization, and reliable fallback logic remain unvalidated.","source_ids":["S3","S4","S5","S7"]},"adoption_authority_feasibility":{"score":2,"rationale":"A retrospective decision-support study can preserve existing authority, but operational use would intersect approved procedures, qualified-personnel programs, complete-coverage obligations, quality records, engineering authority, and evolving aviation-AI assurance.","source_ids":["S1","S2","S3","S8"]},"evidence_readiness":{"score":2,"rationale":"Relevant adjacent studies and tools exist, but there is no direct paired-scan reader study, natural in-service change dataset, adopter commitment, or validated safety case for the proposed workflow.","source_ids":["S4","S5","S7"]},"safety_net_benefit":{"score":5,"rationale":"Retention of authoritative raw data, qualified-reader control, concealed raw sampling, protected-zone bypass, reconstruction checks, and automatic reversion to full review directly address the proposal's principal suppression risks, although their effectiveness still requires testing.","source_ids":["S2","S3","S8"]},"scalability":{"score":2,"rationale":"Software processing could scale, but models, registration tolerances, repair histories, acquisition compatibility, procedure approval, and validation would likely be panel-, method-, and organization-specific.","source_ids":["S1","S3","S4","S7"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"250K_TO_1M","scope":"One preregistered retrospective reader study of at most 40 paired scans from one panel/method family, including a frozen research interface, data de-identification and registration, three counterbalanced review arms, qualified readers, adjudication, fault injection, and statistical analysis.","confidence":"MODERATE","assumptions":["The MRO or asset owner contributes existing full-resolution paired scans and adjudicated outcomes under a research agreement.","Six to ten qualified readers are compensated for counterbalanced sessions.","No aircraft, maintenance record, approved procedure, or operational disposition is changed.","The range is a bottom-up resource-equivalent estimate; none of the eight sources provides a project quote."],"source_ids":["S3","S4","S5","S7"]},"initial_deployment_startup":{"band_2026_usd":"1M_TO_5M","scope":"Production-grade implementation for one inspection-method and panel family at one organization, including secure data ingestion, workstation integration, model and configuration control, audit logging, cybersecurity, verification datasets, human-factors evaluation, and quality-system documentation.","confidence":"LOW","assumptions":["Existing inspection hardware and raw-data export remain usable.","The system remains advisory and does not reduce mandated coverage.","The range includes software engineering and assurance but not new fleet-wide scanning hardware.","No vendor pricing or regulator-approved compliance plan was publicly verified."],"source_ids":["S1","S3","S7","S8"]},"operational_launch":{"band_2026_usd":"1M_TO_5M","scope":"Limited operational shadow launch for one MRO program across one or a few sites, including procedure-change review, training, validation and qualification evidence, parallel full review, support, incident response, and launch governance.","confidence":"LOW","assumptions":["Existing qualified inspectors and structural-engineering authorities retain all decisions.","Shadow operation continues full review until formal evidence supports any workflow change.","Aircraft-specific data conversion and registration do not require replacing acquisition equipment.","Certification and approval costs vary materially by operator, design approval holder, and jurisdiction."],"source_ids":["S1","S2","S3","S8"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Annual support for one deployed panel/method family: model and baseline configuration control, raw-audit review, drift and fallback monitoring, software maintenance, periodic revalidation, reader training, quality audits, and incident investigation.","confidence":"LOW","assumptions":["Deployment remains limited to a small number of sites and one inspection family.","Model updates are batch-reviewed rather than performed online.","Qualified-reader time and engineering review dominate recurring cost.","No public maintenance contract or software price was available in the eight-source record."],"source_ids":["S2","S3","S7","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Independent research and an aerospace university project document high data volume, time-consuming manual analysis, false-call sources, workforce scarcity, and demand for automated ultrasonic interpretation.","source_ids":["S4","S5","S6"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"FAA-recognized MROs and operators are identifiable authorizers/users; Korean Air, Bombardier, and Spirit AeroSystems are directly associated with adjacent automated ultrasonic-analysis work.","source_ids":["S1","S4","S5","S6"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The residual-first workflow can be compared against complete review and fixed registered differences on indication capture, review burden, reconstruction fidelity, false prioritization, audit disagreement, and fallback frequency.","source_ids":["S4","S5","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"A maximum-40-case, single-family, frozen-model, retrospective three-arm reader study is bounded and does not change aircraft status or approved procedures.","source_ids":["S2","S3","S4","S5"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"For the proposed retrospective step, authoritative raw scans, existing records, qualified-reader authority, complete coverage, and engineering disposition remain unchanged. This gate does not authorize operational use or reduced coverage.","source_ids":["S1","S2","S3","S8"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"The four ranges are scoped bottom-up resource-equivalent estimates, but no public vendor quote, reader-study budget, proprietary data-preparation estimate, or approval plan was verified.","source_ids":["S3","S7","S8"]}},"next_evidence_step":"With an aircraft operator or MRO data partner, preregister a retrospective, offline, blinded reader study using no more than 40 paired full-resolution inspections from one documented panel and inspection-method family. Freeze eligibility, registration, model version, protected zones, audit sampling, residual budget, reconstruction tolerance, fallback rules, and indication-level noninferiority margins before outcomes are exposed. Qualified readers should review each case in counterbalanced sessions under (A) existing complete full-scan traversal, (B) a fixed registered-difference display, and (C) the residual-first interface. Primary endpoints should be protected-indication sensitivity and reader decision concordance; secondary endpoints should include review time, regions examined, raw-data reveals, false prioritizations, reconstruction error, concealed random-audit disagreement, fallback rate, and total model/audit burden. Inject bounded registration, coupling, calibration, and model-version faults to test reversion. Falsify the claim if arm C misses any preregistered protected indication beyond the noninferiority margin, violates reconstruction tolerance, yields systematic concealed-audit disagreement, anchors readers toward incorrect dispositions, or fails to reduce total burden after model and fallback costs; also falsify incremental advantage if arm B or A meets all criteria at lower burden.","blocking_evidence":["No direct measurement establishes how often eligible repeat aircraft-composite scans exist or how much full-scan traversal delays their disposition.","No proprietary paired-scan dataset with natural new, changed, stable, repaired, and acquisition-artifact cases was available in the web evidence.","No qualified-reader experiment compares residual-first ordering with both complete review and fixed registered differences.","No evidence shows that concealed raw-tile auditing and decompression reliably catch subtle indications that the predictive model suppresses.","Cross-inspection registration, acquisition compatibility, reconstruction tolerance, rare-event recall, and reader anchoring remain unvalidated.","No operator, MRO, design approval holder, or regulator has committed data, budget, procedure authority, or an operational approval path for this exact system.","Cost ranges lack vendor quotes and organization-specific quality, cybersecurity, integration, and approval estimates."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The eight-source search found substantial adjacent automated C-scan analysis, AI-assisted ultrasonic interpretation, reference-envelope subtraction, commercial offline workstations, and aviation assurance guidance, but did not find the complete proposed repeat-scan residual-review and independent-audit workflow. This is only a bounded prior-art observation. World novelty, patentability, freedom to operate, market size, realized impact, and exhaustive standards or product coverage remain unmeasured.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure a data-use agreement for up to 40 provenance-complete paired scans from one operator or MRO and one panel/method family.","Predefine eligibility, registration tolerances, protected indications, reconstruction limits, noninferiority margins, audit sampling, and fallback thresholds with Level III NDI, structural engineering, and quality representatives.","Implement and freeze the three reader-study comparators: complete review, fixed registered difference, and residual-first review.","Run counterbalanced blinded sessions with qualified readers and report indication-level sensitivity, concordance, time, raw reveals, false prioritization, audit disagreements, and fallback frequency.","Demonstrate fault-triggered reversion under injected registration, coupling, calibration, and version mismatches.","Obtain organization-specific estimates for integration, validation, procedure control, training, and recurring audit costs before any operational-adoption inquiry."],"reason":"Bounded web research verifies a material general problem, identifiable stakeholders, strong adjacent prior art, and a technically plausible offline path. The remaining incremental and safety claims cannot be resolved by further public-web research: they require proprietary paired scans, qualified-reader fieldwork, and controlled empirical testing. Under the stated controller rule, this requires an empirical-research stop."},"proposal_index":2}