{"schema_version":1,"research_id":"eoa_inverse_innovation_exp05_external_evaluation_20260803","source_assessment_id":"computability_boundary_mapping__human_computer_interaction:P1:v0","cell_id":"computability_boundary_mapping__human_computer_interaction","search_queries":["site:w3.org/WAI/WCAG22/Understanding/error-prevention-legal-financial-data reversible checked confirmed","site:nist.gov SP 800-218 code review analysis testing software verification","Rice 1953 Classes of recursively enumerable sets and their decision problems PDF","abstract interpretation Cousot 1977 POPL PDF","CBMC official documentation bounded model checker loops unwind assertions","SPIN official model checker finite state systems verification documentation","OASIS SARIF 2.1 result kind pass fail open notApplicable executionSuccessful standard","official design system destructive action confirmation undo guidance government","site:docs.cedarpolicy.com validation errors unknown policies Cedar official validator sound","site:cedarpolicy.com formal verification automated reasoning policy validation official","site:infer.deepsemantic.com docs sound analysis false positives official Infer","site:developer.apple.com destructive action confirmation human interface guidelines undo","Turing 1936 On Computable Numbers PDF official university halting undecidable","Rice theorem official university course page program semantic properties undecidable","halting problem undecidable primary paper accessible PDF"],"sources":[{"source_id":"S1","title":"Understanding Success Criterion 3.3.4: Error Prevention (Legal, Financial, Data)","publisher":"World Wide Web Consortium, Web Accessibility Initiative","url":"https://www.w3.org/WAI/WCAG22/Understanding/error-prevention-legal-financial-data.html","source_class":"OFFICIAL_GUIDANCE","publication_date":"Undated WCAG 2.2 Understanding document","accessed_at":"2026-08-03","claims_supported":["Important submissions involving legal, financial, or user-controlled data should be reversible, checked, or reviewable and confirmable.","Mistakes involving irreversible transactions or data deletion can have serious consequences, particularly for disabled users.","The protected-action interaction objective is externally recognized, although this informative page does not require confirmation, cancellation, and undo simultaneously."]},{"source_id":"S2","title":"Button – GOV.UK Design System","publisher":"Government Digital Service, United Kingdom","url":"https://design-system.service.gov.uk/components/button/","source_class":"OFFICIAL_GUIDANCE","publication_date":"Undated living guidance","accessed_at":"2026-08-03","claims_supported":["Warning buttons are intended for serious destructive actions that cannot easily be undone.","GOV.UK recommends an additional confirmation step for such actions and warns against communicating seriousness through color alone.","A government design-system team is an identifiable potential authorizer for protected-action component rules."]},{"source_id":"S3","title":"Guidelines on Minimum Standards for Developer Verification of Software","publisher":"National Institute of Standards and Technology","url":"https://www.nist.gov/publications/guidelines-minimum-standards-developer-verification-software","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2021-07","accessed_at":"2026-08-03","claims_supported":["NIST recommends multiple complementary verification techniques, including automated testing, static scanning, structural tests, historical tests, and fuzzing.","The guidance expressly does not address the totality of software verification, supporting bounded claims rather than an implication that finite testing proves universal behavior.","Software owners have an externally expressed need for verification evidence, but this source does not demand the proposed HCI-specific architecture."]},{"source_id":"S4","title":"On Computable Numbers, with an Application to the Entscheidungsproblem","publisher":"Proceedings of the London Mathematical Society; CMU-hosted facsimile","url":"https://www.cs.cmu.edu/~odonnell/15455-s17/turing-paper.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"1936","accessed_at":"2026-08-03","claims_supported":["There are general decision questions over unrestricted effective computation for which no uniform effective solution exists.","An impossibility conclusion must be tied to a declared computation model and formal reduction rather than inferred from timeouts.","This foundational result supports the plausibility, not the correctness, of the proposal's specific plugin-to-protected-state reduction."]},{"source_id":"S5","title":"Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints","publisher":"ACM SIGPLAN-SIGACT; author-hosted bibliographic and technical summary","url":"https://cs.nyu.edu/~pcousot/COUSOTpapers/POPL77.shtml","source_class":"PRIMARY_RESEARCH","publication_date":"1977","accessed_at":"2026-08-03","claims_supported":["Abstract interpretation derives information about concrete computations through an abstract semantic domain.","Abstract analysis can be sound relative to formal semantics while remaining incomplete or imprecise.","A reviewed over-approximation is established prior art for one-sided program assurance, not a novel mechanism of this proposal."]},{"source_id":"S6","title":"The CPROVER Manual: Loop Unwinding","publisher":"CPROVER Project","url":"https://www.cprover.org/cprover-manual/cbmc/unwinding/","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"Undated living documentation","accessed_at":"2026-08-03","claims_supported":["CBMC analyzes executions only to an explicit loop-unwinding bound unless sufficiency is separately established.","Insufficient bounds can miss deeper bugs, while unwinding assertions can expose that limitation.","Bounded counterexample search plus explicit bound-sufficiency status is mature prior art for the proposal's bounded-search mode."]},{"source_id":"S7","title":"Static Analysis Results Interchange Format (SARIF) Version 2.1.0","publisher":"OASIS Open","url":"https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html","source_class":"STANDARD","publication_date":"2020-03-26","accessed_at":"2026-08-03","claims_supported":["SARIF already distinguishes pass, fail, open because of insufficient information, not-applicable, informational, and human-review results.","SARIF separately records whether the analysis invocation succeeded and supports tool execution notifications.","Machine-readable separation of proof, inconclusive analysis, inapplicability, review, and tool failure is substantially established practice."]},{"source_id":"S8","title":"Policy validation against schema","publisher":"Cedar Policy Project","url":"https://docs.cedarpolicy.com/policies/validation.html","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"Undated living documentation","accessed_at":"2026-08-03","claims_supported":["Cedar validates policies against an explicit schema and states a formally proved, scoped validation-soundness guarantee.","The guarantee has enumerated residual error classes and depends on requests adhering to schema expectations.","Schema changes alter the authorization model and trigger revalidation, closely paralleling enforceable fragments, scoped guarantees, and recheck triggers."]}],"problem_evidence":{"support":"MODERATE","rationale":"W3C and GOV.UK establish that consequential or destructive interface actions require reversal, checking, confirmation, and clear cancellation-related design, while NIST cautions that ordinary verification techniques do not constitute total verification. SARIF's standardized open, not-applicable, review, and execution-failure states show that non-Boolean analysis outcomes are a recognized tooling need. No direct source demonstrates the proposal's more specific prevalence claim that design-system pipelines currently accept unrestricted executable plugins while collapsing timeouts or bounded non-findings into universal-safe Boolean verdicts.","source_ids":["S1","S2","S3","S7"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"The GOV.UK Design System team is an identifiable design-system authorizer expressing a need for confirmation around serious destructive actions; W3C supplies an accessibility-facing requirement context, and NIST supplies an institutional software-verification mandate. These sources support the objectives and relevant authority roles, but none expresses demand or funding for a computability-boundary router for executable interface plugins.","source_ids":["S1","S2","S3"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"SARIF result and invocation taxonomy","similarity":"Already standardizes pass, fail, inconclusive/open, not-applicable, human-review, and tool-execution status distinctions for static-analysis workflows.","remaining_difference":"It is an interchange format, not an HCI property checker; it does not enforce a finite interaction language or prove a protected-action monitor.","source_ids":["S7"]},{"name":"Cedar schema validation and revalidation","similarity":"Combines an enforceable typed language/schema boundary, a formally scoped soundness guarantee, documented residual errors, and revalidation after model changes.","remaining_difference":"It addresses authorization-policy validity rather than reachability of protected UI actions and does not provide the proposal's exact protected-action status alphabet.","source_ids":["S8"]},{"name":"CBMC bounded model checking","similarity":"Finds replayable assertion violations within declared bounds and exposes insufficient unwinding rather than treating a bounded non-finding as universal proof.","remaining_difference":"It is a general C verifier; the proposal adds an HCI-specific grammar, monitor, routing policy, and separation from usability review.","source_ids":["S6"]},{"name":"Abstract interpretation","similarity":"Provides the proposed one-directional, potentially imprecise proof mode through a sound abstraction of concrete executions.","remaining_difference":"The proposal's remaining work is selecting and reviewing an abstraction faithful to executable interface plugins and their event semantics.","source_ids":["S5"]}],"distinctive_claim_remaining":"The remaining falsifiable claim is application-specific: for protected-action interface extensions, an ingestion-enforced finite-state grammar plus exact checking, reviewed one-sided analysis for executable plugins, bounded witness search, and persistent guarantee labels will produce fewer overstated release verdicts than a Boolean lint/test baseline without causing unacceptable UNKNOWN rates, review burden, or semantic mismatch. No searched source establishes this integrated HCI implementation or its comparative workflow performance.","confidence":"HIGH"},"implementation_evidence":{"support":"STRONG","rationale":"The core ingredients are technically mature: model-relative impossibility arguments, abstract interpretation, bounded model checking, schema-enforced validation, revalidation triggers, and standardized multi-state analysis results. A shadow fixture audit is therefore implementable. Remaining feasibility risks are proposal-specific: whether the plugin language supports the claimed reduction, whether fragment membership is bypass-proof, whether the abstraction really over-approximates the deployed semantics, whether state growth is manageable, and whether the formal monitor represents what users perceive. Existing sources do not resolve those implementation and semantic-fidelity obligations.","source_ids":["S4","S5","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Preventing misleading assurance around consequential actions could avoid serious user harm and blocked releases, but the prevalence and realized effect size of the specific pipeline failure are unmeasured.","source_ids":["S1","S2","S3"]},"stakeholder_pull":{"score":2,"rationale":"Relevant standards and design-system authorities express adjacent needs, but no adopter requests this exact architecture or commits staff, funding, or deployment access.","source_ids":["S1","S2","S3"]},"incremental_advantage":{"score":3,"rationale":"The integrated routing contract could improve a Boolean baseline, but SARIF, Cedar, CBMC, and abstract interpretation already supply most constituent practices; comparative advantage remains untested.","source_ids":["S5","S6","S7","S8"]},"distinctiveness_plausibility":{"score":2,"rationale":"Distinctiveness is limited to the HCI-specific integration and protected-action monitor; the main technical and status-label mechanisms substantially collide with established prior art.","source_ids":["S5","S6","S7","S8"]},"technical_implementability":{"score":4,"rationale":"A bounded prototype can reuse mature formal-analysis patterns and result schemas, although production soundness and state growth remain material risks.","source_ids":["S5","S6","S7","S8"]},"adoption_authority_feasibility":{"score":3,"rationale":"A design-system release owner can authorize a non-blocking shadow audit, and government design-system practice demonstrates such governance roles; production adoption would require cross-functional formal, accessibility, and release approval.","source_ids":["S1","S2","S3"]},"evidence_readiness":{"score":4,"rationale":"The candidate supplies explicit fixtures, statuses, comparators, halt conditions, and falsifiers, while prior art provides concrete implementation references. It still lacks the actual reduction, grammar, monitor, and test corpus.","source_ids":["S4","S6","S7","S8"]},"safety_net_benefit":{"score":4,"rationale":"Preserving UNKNOWN, out-of-scope, human-review, and tool-failure states is a strong safety feature aligned with an existing analysis-results standard; the benefit disappears if downstream release tooling remaps them to pass or fail.","source_ids":["S1","S7"]},"scalability":{"score":3,"rationale":"Machine-readable routing and reusable grammars can scale across components, but finite-state explosion, abstraction maintenance, and independent review costs may grow sharply with language expressiveness.","source_ids":["S5","S6","S8"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"A four-week, offline audit by one verification engineer with part-time independent formal and interaction reviewers; define one monitor and fragment, implement a thin router, and run approximately 30 frozen fixtures against three comparators.","confidence":"MODERATE","assumptions":["Existing component source and event semantics are available without procurement.","Open-source verification and result-format tooling is reused.","The estimate is a resource-equivalent labor band, not a vendor quotation.","No production pipeline or user-facing behavior is changed."],"source_ids":["S6","S7","S8"]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"Build and review a minimum viable fragment parser, exact checker, executable-plugin abstraction, bounded-search adapter, guarantee-label schema, replay support, and shadow CI integration for one design system.","confidence":"LOW","assumptions":["Approximately three to eight loaded engineer-months plus independent review.","The host plugin language and build pipeline expose stable intermediate representations.","No new theorem prover or model checker is developed from scratch.","Accessibility and usability validation remains a separate workstream."],"source_ids":["S5","S6","S7","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Production hardening across multiple protected-action components, semantic-version governance, security review, performance engineering, documentation, reviewer training, monitoring, rollback controls, and release-policy integration.","confidence":"LOW","assumptions":["Approximately eight to twenty-four cross-functional loaded person-months.","Launch includes state-space and abstraction tuning, auditability, and pipeline reliability work.","The analyzer remains advisory until seeded-case and shadow-run acceptance criteria are met.","This excludes organization-wide migration of arbitrary legacy plugins."],"source_ids":["S3","S6","S7","S8"]},"annual_recurring":{"band_2026_usd":"50K_TO_250K","scope":"Maintain grammar, semantics, monitors, abstractions, fixtures, revalidation triggers, documentation, and independent reviews as plugin and design-system versions change.","confidence":"LOW","assumptions":["Approximately 0.5 to 1.5 loaded full-time equivalents annually.","Major language redesigns or organization-wide expansion would be separately funded.","Tooling is self-hosted or uses existing CI capacity.","Cost is highly sensitive to the frequency of recheck triggers and UNKNOWN triage."],"source_ids":["S7","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Authoritative accessibility and government design guidance establishes the harm and need around consequential actions, while NIST and SARIF support honest limits and non-Boolean verification outcomes. The exact prevalence of the proposed pipeline pathology remains unmeasured.","source_ids":["S1","S2","S3","S7"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"A design-system release owner is a credible internal authorizer, and the GOV.UK Design System provides an identifiable real-world analogue that governs destructive-action components. Exact adopter pull for formal boundary mapping remains absent.","source_ids":["S2","S3"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The proposal can be compared against a Boolean lint/test baseline and a label-only SARIF mapping on correct label assignment, false-safe outcomes, UNKNOWN rate, bypass resistance, replayability, and review burden.","source_ids":["S6","S7","S8"]},"bounded_next_evidence_step":{"status":"YES","reason":"A four-week offline audit of one archived component and a frozen synthetic suite is bounded, reversible, comparator-based, and has explicit technical and semantic falsifiers.","source_ids":["S6","S7","S8"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"The first step is non-blocking and offline, leaves release authority with the existing owner, and must halt on monitor mismatch, fragment bypass, unsound safe labels, non-replayable witnesses, or status collapse.","source_ids":["S1","S7"]},"credible_cost_scope_and_range":{"status":"YES","reason":"All four estimates name a bounded deliverable and labor assumptions and use deliberately broad resource-equivalent bands. Confidence is low beyond the first audit because language complexity, state growth, and integration scope are unknown.","source_ids":["S5","S6","S7","S8"]}},"next_evidence_step":"Run a four-week, non-blocking shadow audit on one archived protected-action component and approximately 30 preregistered fixtures. Freeze the plugin semantics, protected-action monitor, finite grammar, analysis bounds, and expected labels before execution. Compare (A) current Boolean lint/tests, (B) the same tools mapped only into SARIF-like pass/fail/open/failure states, and (C) the full proposal with enforced fragment membership, exact checking, reviewed over-approximation, bounded witness search, and routing. Require an independent reviewer to check the source-to-target reduction and abstraction obligation and an interaction reviewer to judge monitor fidelity. Continue only if all seeded finite cases receive correct exact labels, every violation witness replays, bound exhaustion remains UNKNOWN, malformed inputs and tool failures remain distinct, no fragment bypass succeeds, and no abstraction-based safe result omits a seeded concrete behavior. Falsify incremental advantage if the full architecture produces no fewer overstated verdicts than comparator B, if more than half of executable fixtures remain UNKNOWN without actionable explanation, or if review effort exceeds two person-hours per fixture without improving correct disposition. This test cannot establish prevalence, production-scale performance, user comprehension, or realized harm reduction.","blocking_evidence":["No direct evidence quantifies how often real design-system pipelines accept unrestricted executable plugins and collapse timeout, bounded, out-of-scope, or tool-failure states into Boolean assurance.","No named adopter has requested, funded, or committed access for the exact protected-action boundary architecture.","The proposed halting reduction has not been constructed and independently checked against a concrete plugin language and event semantics.","No implemented fragment gate, exact checker, abstraction, router, or replay harness has been tested on fixtures.","No user or accessibility study establishes that the formal confirmation, cancellation, and undo monitor is semantically faithful or perceptible in use.","No comparative field evidence measures false-safe reduction, UNKNOWN burden, release latency, bypass behavior, or production scalability."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"World novelty, patentability, freedom to operate, market size, and realized impact were not measured. This bounded search found substantial mechanism-level prior art in abstract interpretation, bounded model checking, schema-enforced validation, revalidation, and machine-readable inconclusive/failure labels. It did not find a direct source implementing the entire protected-action HCI architecture, but absence from eight sources is not evidence of world novelty.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure one design-system partner and obtain an archived executable-plugin component, baseline pipeline outputs, and authority for a non-blocking shadow audit.","Produce independently reviewable artifacts: the exact property monitor, enforceable grammar, plugin-to-halting reduction, abstraction soundness argument, frozen fixtures, and result-label mapping.","Run the preregistered three-comparator audit and report label accuracy, false-safe outcomes, UNKNOWN and out-of-scope rates, witness replayability, fragment bypass attempts, analysis time, and reviewer effort.","Conduct a separate interaction review that can falsify semantic fidelity; do not interpret formal satisfaction as evidence that users notice or understand the safeguards.","Do not advance to production unless no seeded label is misclassified, no safe abstraction omits a seeded behavior, all witnesses replay, and downstream tooling preserves every inconclusive and failure state."],"reason":"Open-web evidence establishes that the problem objective matters, the architecture is technically plausible, and most constituent mechanisms are established prior art. It cannot determine whether the specific pipeline pathology is prevalent, whether an adopter will accept the workflow, whether the proposed formalization and abstraction are sound, or whether the integrated architecture outperforms a simpler SARIF-style multi-state baseline. Those questions require proprietary artifacts, independent proof work, and live shadow testing; under the controller rule this requires an empirical-research stop, which is non-repairable within bounded web research."},"proposal_index":1}