{"schema_version":1,"experiment_id":"eoa_inverse_innovation_exp04_retrieval_first_paired20_20260802","cell_id":"computability_boundary_mapping__engineering_design","hypothesis_id":"H5","search_queries":["multidisciplinary design optimization external simulation failure handling failed analysis infeasible","engineering optimization hidden simulation failures retry abort recover unavailable experiment","human in the loop optimization abstain unavailable evaluator preference oracle failure","FMI standard co-simulation status discard pending error external solver","site:patents.google.com optimization external simulation failure retry infeasible design evaluation","NASA systems engineering handbook decision analysis uncertainty missing information design decision gate","simulation-based optimization hidden constraints failed function evaluations crash constraints research","black-box optimization failed evaluations missing values hidden constraints engineering design"],"sources":[{"source_id":"PA1","title":"Simulation Failure Capturing","publisher":"Sandia National Laboratories, Dakota","url":"https://snl-dakota.github.io/docs/6.19.0/users/usingdakota/advanced/simulationfailurecapturing.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["Dakota provides explicit failure detection, communication, and mitigation for external simulations.","Its declared routes include abort, retry, recovery using dummy function values, and continuation.","Retry expressly addresses external infrastructure failures such as unavailable licenses, networks, and shared storage.","Recovery with dummy objective or constraint values shows that existing practice can intentionally translate missing evaluations into optimization penalties rather than preserve an unresolved gate status."]},{"source_id":"PA2","title":"Raising an AnalysisError","publisher":"OpenMDAO Development Team","url":"https://openmdao.org/newdocs/versions/latest/advanced_user_guide/analysis_errors/analysis_error.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","claims_supported":["OpenMDAO exposes invalid analysis evaluations through a distinct AnalysisError path.","Its documentation warns that optimizer behavior is contingent: some optimizers may fail to find a solution or may return an incorrect solution after evaluation errors."]},{"source_id":"PA3","title":"Functional Mock-up Interface Specification","publisher":"Modelica Association Project FMI","url":"https://fmi-standard.org/docs/main/","source_class":"STANDARD","claims_supported":["FMI standardizes typed statuses including OK, Warning, Discard, Error, and Fatal for externally coupled simulation components.","For Discard and Error, output values are explicitly undefined; Error prohibits continued simulation, while Discard permits only controlled alternatives or termination.","The standard therefore establishes a neighboring-domain contract that prevents a failed external model call from being silently treated as a valid model result."]},{"source_id":"PA4","title":"Optimizing a Simulation or Ordinary Differential Equation","publisher":"MathWorks","url":"https://www.mathworks.com/help/optim/ug/optimizing-a-simulation-or-ordinary-differential-equation.html","source_class":"COMMERCIAL_FIRST_PARTY","claims_supported":["MathWorks documents objective and constraint evaluation failures as an ordinary simulation-optimization condition.","It recommends returning NaN so supported solvers can attempt a different step, demonstrating established propagation of an indeterminate evaluation rather than immediate classification as infeasible."]},{"source_id":"PA5","title":"Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints","publisher":"INFORMS Journal on Computing","url":"https://pubsonline.informs.org/doi/10.1287/ijoc.2018.0864","source_class":"PRIMARY_RESEARCH","claims_supported":["The research defines hidden constraints as cases where a black-box simulation returns no objective value for a parameter vector.","It introduces an evaluability predictor and contrasts that approach with optimizers that assign artificial values to failed evaluations, showing that missing-evaluation routing is an established research problem."]},{"source_id":"PA6","title":"NASA Systems Engineering Handbook","publisher":"National Aeronautics and Space Administration","url":"https://www.nasa.gov/wp-content/uploads/2018/09/nasa_systems_engineering_handbook_0.pdf","source_class":"OFFICIAL_GUIDANCE","claims_supported":["NASA guidance calls for decision reports to capture evaluation methods, assumptions, uncertainties, sensitivities, recommendations, and the final decision rationale.","This substantially anticipates the proposed decision-record and guarantee-provenance components at engineering design gates, although it is not an executable oracle router."]},{"source_id":"PA7","title":"The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures","publisher":"Association for Computing Machinery authors' manuscript","url":"https://sven-mayer.com/wp-content/uploads/2022/07/ou2022human.pdf","source_class":"PRIMARY_RESEARCH","claims_supported":["A three-month professional deployment and follow-up controlled study found inconsistent or inexplicable human ratings in a human-in-the-loop Bayesian optimization workflow.","The authors report that preference optimization lacked mechanisms for inconsistent and contradictory human judgments, directly supporting the nominated problem beyond purely numerical simulators."]}],"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"Dakota simulation-failure capture","similarity":"Very close to the executable core: an optimizer receives explicit failure signals from an external analysis and routes them through declared abort, retry, recover, or continuation policies.","remaining_difference":"It is simulation-specific, permits user-supplied dummy objective or constraint values, and does not impose an end-to-end rule preserving an unresolved status at a design gate across laboratories, vendors, and human evaluators.","source_ids":["PA1"]},{"name":"FMI typed external-model status contract","similarity":"Standardizes the contract boundary for external multidisciplinary models, distinguishes recoverable rejection from error and fatal failure, and declares failed-call outputs undefined.","remaining_difference":"It governs simulation interoperability rather than candidate optimization verdicts, heterogeneous non-computational evaluators, or escalation to a design-review board.","source_ids":["PA3"]},{"name":"OpenMDAO AnalysisError and MathWorks NaN handling","similarity":"Both expose failed objective or analysis evaluations to optimizers and support controlled attempts to move away from unevaluable regions instead of treating the call as ordinary data.","remaining_difference":"They provide solver-level error handling but no mechanically enforced promise contract or final-gate invariant forbidding unqualified feasible, infeasible, or optimal verdicts after unresolved evaluation failures.","source_ids":["PA2","PA4"]},{"name":"Hidden-constraint and human-preference optimization research","similarity":"The literature treats missing simulator outputs and inconsistent human judgments as explicit failure modes of black-box or human-in-the-loop optimization.","remaining_difference":"The examined methods predict evaluability or study judgment inconsistency; they do not supply one auditable router spanning simulation, experiment, vendor, and expert capabilities through a systems-engineering gate.","source_ids":["PA5","PA7"]},{"name":"NASA decision-analysis record","similarity":"Captures assumptions, uncertainty, sensitivities, evaluation methods, recommendations, and final rationale for engineering-board decisions.","remaining_difference":"It is documentation guidance rather than an executable contract that blocks definitive optimization labels when required evidence is absent or outside promise.","source_ids":["PA6"]}],"overlapping_components":["Explicit external-evaluation failure signaling","Declared abort, retry, recovery, and continuation routes","Typed separation of successful, warning, discarded, error, and fatal outcomes","Undefined-output semantics after failed external calls","Input-domain bounds and hidden-constraint or evaluability handling","Optimizer-dependent safeguards against failed evaluations","Decision records containing assumptions, uncertainty, methods, and rationale","Recognition of inconsistent human judgment as an optimization failure mode"],"remaining_contrastive_claim":"No examined source implements one mechanically enforced contract spanning simulator, laboratory, vendor, and human evaluations that checks each input promise and preserves a labeled unresolved or escalated state through the final multidisciplinary design-gate verdict.","claim_falsifier":"A product, standard, patent, or deployed workflow would falsify the contrastive claim if it demonstrably covers all four evaluator classes, mechanically checks their preconditions, treats delay, refusal, inconsistency, and failure as typed non-results, and prevents every unqualified feasible, infeasible, or optimal gate verdict until valid evidence is restored.","problem_support":"STRONG","recommendation":"RESEARCH","world_novelty_boundary":"The bounded eight-query, seven-source search establishes substantial prior art for nearly every constituent mechanism, especially simulation-failure contracts, typed undefined results, retry or abort routing, evaluability modeling, and decision provenance. It did not locate the claimed heterogeneous-oracle, end-to-end gate invariant; that narrow integration remains the only plausible novelty boundary, and absence from this ordinary-web search is not evidence of worldwide novelty or patentability."}