{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"representation_independent_interface_contract__human_computer_interaction:P3:v0","cell_id":"representation_independent_interface_contract__human_computer_interaction","search_queries":["design review comments lost when prototype changes annotation anchor research","semantic anchoring design annotations evolving user interface prototypes research","Figma comments move with layers official comments design review","W3C Web Annotation selectors robust anchoring changing documents","site:dl.acm.org design annotations evolving prototypes traceability comments interface design review","\"Engineering Annotations\" interactive systems design artefacts annotation model","design rationale annotations prototype evolution traceability user interface research","official Figma REST API comments endpoint node_id client_meta"],"sources":[{"source_id":"S1","title":"Move or edit comments","publisher":"Figma","url":"https://help.figma.com/hc/en-us/articles/360041547853-Move-or-edit-comments","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["Figma comments can be pinned to frames, layers, or canvas coordinates.","Cut-and-paste can disconnect a comment from its layer; cross-file moves leave comments behind; duplicated files do not copy comments.","The documented behavior directly demonstrates representation-dependent attachment failure in a major design workflow."]},{"source_id":"S2","title":"Comments | Developer Docs","publisher":"Figma","url":"https://developers.figma.com/docs/rest-api/comments-types/","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["Figma exposes comment identity, author, creation time, resolution time, file identity, and positioning metadata.","Comment targeting is represented through absolute coordinates or offsets and regions within frames rather than semantic interaction roles.","A prototype adapter could read current comment and location information, although the documentation does not establish the write and governance capabilities needed for the full proposal."]},{"source_id":"S3","title":"Web Annotation Data Model","publisher":"World Wide Web Consortium (W3C)","url":"https://www.w3.org/TR/annotation-model/","source_class":"STANDARD","publication_date":"2017-02-23","accessed_at":"2026-08-03","claims_supported":["The W3C model standardizes annotation bodies, targets, selectors, provenance, creation and modification information, and resource states.","Selectors may be refined or offered as alternatives, and Time State can identify the representation appropriate to an annotation as a resource changes.","The model is a mature substrate for representing bindings and provenance, but it does not itself define the proposed design-concern lifecycle, ambiguity statuses, or resolver conformance rule."]},{"source_id":"S4","title":"Web Annotations Workshop Report","publisher":"World Wide Web Consortium (W3C)","url":"https://www.w3.org/2014/04/annotation/report.html","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"2014-04-02","accessed_at":"2026-08-03","claims_supported":["W3C workshop participants explicitly identified robust re-anchoring after document change as a requirement.","Hypothes.is combined multiple selectors and fallback matching strategies because XPath anchors fail after structural change.","The report identifies orphaned annotations and cross-format or cross-location change as unresolved concerns, closely analogous to changing prototype representations."]},{"source_id":"S5","title":"Engineering Annotations: A Generic Framework For Gluing Design Artefacts in Models of Interactive Systems","publisher":"arXiv / authors from Université Côte d’Azur, IRIT, SOFTEAM and collaborators","url":"https://arxiv.org/abs/2205.01333","source_class":"PRIMARY_RESEARCH","publication_date":"2022-05-03","accessed_at":"2026-08-03","claims_supported":["The authors state that annotations made on one interactive-system artifact are often relevant to other artifacts and that tools are needed to coordinate them.","They propose a model based on the W3C Web Annotation Data Model and demonstrate feasibility with a deployed plugin/tool suite across multiple artifacts and two cockpit-design iterations.","This is the closest located analogue and substantially overlaps the proposal's cross-artifact annotation-preservation objective."]},{"source_id":"S6","title":"Re-Thinking Traceability: A Prototype to Record and Revisit the Evolution of Design Artefacts","publisher":"Association for Computing Machinery, via EUSSET Digital Library","url":"https://dl.eusset.eu/items/04ec3886-5ffd-4568-b6d4-1512d57903ab","source_class":"PRIMARY_RESEARCH","publication_date":"2018","accessed_at":"2026-08-03","claims_supported":["The study characterizes tracking creative design processes as cumbersome and design traceability as important for revisiting artifact evolution.","A 15-week evaluation with six student pairs found that an annotated artifact repository aided reflection, agreement, consistency, and planning.","The evidence supports value and feasibility qualitatively but is too small and educationally situated to establish prevalence or organizational effect size."]},{"source_id":"S7","title":"Branching in Figma","publisher":"Figma","url":"https://www.figma.com/best-practices/branching-in-figma/","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["Figma recommends comments as a feedback mechanism during branch review and assigns merge authority to editors of the main file.","Merged branches are archived with their comments, demonstrating an existing review lifecycle and identifiable editor authority.","The workflow identifies plausible adopters and authorizers but contains no commitment to adopt semantic concern anchors."]},{"source_id":"S8","title":"Traceability of Multi-Stakeholder Feedback in Design Decisions of Prototype Evolution","publisher":"University of Minnesota College of Design","url":"https://design.umn.edu/research/research-creative-scholarship-engagement/research-projects-gallery/traceability-multi","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["A named University of Minnesota research team is studying how stakeholder feedback affects successive prototype iterations.","The project explicitly seeks implications for systems and strategies that streamline prototyping and identifies Kusske Design Initiative and an Imagine Grant as funders.","This supplies identifiable researchers and funders with expressed interest in feedback traceability, but not demand for the exact proposed contract."]}],"problem_evidence":{"support":"STRONG","rationale":"The problem is visible in current product behavior: Figma documents that comments can detach during cut-and-paste, remain behind across files, and disappear from duplicated-file workflows. Independent HCI research calls design-process tracking cumbersome, and the closest research explicitly identifies a need to coordinate annotations across changing interactive-system artifacts. The sources do not quantify how often consequential accessibility or usability concerns are lost or misapplied.","source_ids":["S1","S4","S5","S6"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Figma documents comments as part of branch-review work and identifies main-file editors as reviewers and merge authorities. A named University of Minnesota team and two funders are studying traceability of stakeholder feedback through prototype evolution. These are credible adopter, authorizer, and funder classes with expressed adjacent need, but no source commits to piloting this exact semantic-anchor contract.","source_ids":["S7","S8","S1"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"Engineering Annotations framework","similarity":"Coordinates annotations across heterogeneous interactive-system design artifacts using a W3C-based model and demonstrates a plugin/tool suite across design iterations.","remaining_difference":"The available record does not establish a representation-independent contract with mandatory unique/ambiguous/unresolved/superseded outcomes, human-only lifecycle authority, or a shared black-box oracle for independently implemented resolvers.","source_ids":["S5"]},{"name":"W3C Web Annotation Data Model and robust anchoring practice","similarity":"Provides annotation identity, target selectors, provenance, version states, selector refinement, and multi-strategy re-anchoring after resource change.","remaining_difference":"It addresses resource targeting and representation recovery generally, not semantic interaction-role targets, explicit non-selection under ambiguity, concern applicability, or authorized design-review lifecycle transitions.","source_ids":["S3","S4"]},{"name":"Figma comments and branch-review workflow","similarity":"A deployed design-review product provides located comments, identity, replies, resolution timestamps, branch feedback, and editor authority.","remaining_difference":"Its documented target metadata remains coordinate/frame-based, and documented transformations can detach or omit comments; it does not expose the proposed semantic target-resolution state machine.","source_ids":["S1","S2","S7"]},{"name":"Annotated design-artifact traceability repository","similarity":"Connects evolving artifacts, design rationale, and decisions and was evaluated over a 15-week design project.","remaining_difference":"It demonstrates repository-level traceability rather than resolver substitutability, explicit ambiguity behavior, or invariants preventing artifact deletion from closing a concern.","source_ids":["S6"]}],"distinctive_claim_remaining":"For a preregistered corpus of prototype moves, renames, splits, merges, deletions, node reuse, and duplicate semantic roles, two independently represented resolvers can expose identical concern identity, provenance, lifecycle, and unique/ambiguous/unresolved/superseded outcomes; neither silently selects an ambiguous target or mutates lifecycle state without named authority, and clients need not inspect raw artifact identifiers. A single divergent contract-level outcome, silent ambiguous selection, unauthorized mutation, or required raw-ID dependency falsifies the claim.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"Implementation is technically credible because Figma exposes comment identity, timestamps, resolution state, and positional metadata; W3C supplies an interoperable annotation, selector, state, and provenance model; and the closest research reports a working multi-artifact plugin suite. The difficult portions remain unverified: authoring stable semantic targets, resolving split/merge cases, preventing plausible false matches, integrating write operations across tools, and governing confidential design artifacts. The proposed synthetic sandbox avoids immediate privacy and user-safety exposure, but production retention, access-control, intellectual-property, and platform-API terms require separate review.","source_ids":["S2","S3","S4","S5"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Lost or misapplied accessibility and usability concerns could matter materially, and traceability studies report workflow benefits, but prevalence, severity, and organizational cost are unmeasured.","source_ids":["S1","S5","S6"]},"stakeholder_pull":{"score":3,"rationale":"Review comments are an established Figma workflow, editors hold review authority, and a funded research project expresses adjacent traceability need; exact-product demand is absent.","source_ids":["S7","S8"]},"incremental_advantage":{"score":2,"rationale":"Explicit ambiguity, lifecycle authority, and resolver conformance could improve safety over coordinate anchors, but substantial cross-artifact annotation and robust-anchoring prior art already exists and no comparative result is available.","source_ids":["S3","S4","S5"]},"distinctiveness_plausibility":{"score":2,"rationale":"The remaining contract-level claim is precise, but the located Engineering Annotations framework substantially collides and its full feature boundary has not been experimentally differentiated.","source_ids":["S5"]},"technical_implementability":{"score":4,"rationale":"Existing annotation standards, accessible product metadata, and a prior multi-artifact plugin make a sandbox implementation plausible; reliable semantic resolution remains the principal technical uncertainty.","source_ids":["S2","S3","S5"]},"adoption_authority_feasibility":{"score":3,"rationale":"Design-file editors and repository or review-process owners are identifiable authorities, but cross-tool deployment and ownership of accessibility concerns may require multiple approvals.","source_ids":["S7","S8"]},"evidence_readiness":{"score":4,"rationale":"A small preregistered synthetic mutation corpus can test the distinctive invariants without production data or users.","source_ids":["S3","S5"]},"safety_net_benefit":{"score":4,"rationale":"Making ambiguity and orphaning visible rather than silently selecting or closing a concern is a credible safety advantage, although its realized error reduction is untested.","source_ids":["S4","S5"]},"scalability":{"score":3,"rationale":"A common data model and contract suite could scale across resolver implementations, but semantic-target authoring, adapter maintenance, and human adjudication may grow with artifact and tool diversity.","source_ids":["S2","S3","S5"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"Specify the contract, create three synthetic prototype revisions and twelve fictional concerns, implement two minimal resolvers, seed failures, and run the offline comparator.","confidence":"MODERATE","assumptions":["Two technical contributors for approximately two to four weeks.","No commercial design-tool procurement, production data, or participant recruitment.","Resource-equivalent estimate; no external 2026 labor-rate benchmark was included among the eight sources."],"source_ids":["S2","S3","S5"]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"Build a hardened repository, one design-tool adapter, authorization and provenance controls, reviewer interface, audit logging, and a pilot migration utility.","confidence":"LOW","assumptions":["One tool ecosystem and one design team.","No automated closure of substantive concerns.","Security, privacy, and platform terms review included but not enterprise-wide certification."],"source_ids":["S2","S3","S7"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Launch across several teams with production access controls, monitoring, support, resolver evaluation, incident rollback, governance training, and cross-tool adapters.","confidence":"LOW","assumptions":["Two or three artifact systems.","Human adjudication remains available for ambiguity.","No regulated-data deployment or historical enterprise-wide migration."],"source_ids":["S3","S5","S7"]},"annual_recurring":{"band_2026_usd":"50K_TO_250K","scope":"Maintain adapters and schemas, operate the repository, review leakage and resolver drift, support users, and periodically rerun conformance and safety tests.","confidence":"LOW","assumptions":["Moderate organizational usage and artifact volume.","One part-time product owner plus engineering and security support.","Cloud, vendor, and labor prices are not externally benchmarked in this evaluation."],"source_ids":["S2","S3"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Current Figma documentation exposes concrete attachment breakage modes, and independent research identifies cross-artifact annotation coordination and design traceability as problems.","source_ids":["S1","S5","S6"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Figma's workflow identifies main-file editors as review and merge authorities, while the University of Minnesota project identifies a named research team and funders pursuing prototype-feedback traceability. This establishes credible actors, not an adoption commitment.","source_ids":["S7","S8"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The proposal can be falsified by divergent contract-level results between independent resolvers, silent ambiguity collapse, unauthorized lifecycle mutation, lost provenance, or required client dependence on raw artifact IDs.","source_ids":["S3","S5"]},"bounded_next_evidence_step":{"status":"YES","reason":"A three-revision, twelve-concern synthetic mutation test with two resolvers and a coordinate/layer baseline is bounded, reversible, preregistrable, and avoids production data.","source_ids":["S2","S3","S5"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"The first test uses fictional concerns and synthetic artifacts, grants no resolver lifecycle authority, and can halt on any silent match or mutation. Production privacy, retention, platform permissions, and intellectual-property review remain later gates.","source_ids":["S2","S3","S7"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"The four scopes and resource-equivalent bands are bounded and auditable, but none of the eight sources provides 2026 labor, platform, security-review, or operating-cost benchmarks.","source_ids":["S2","S3","S5"]}},"next_evidence_step":"Run an offline, preregistered mutation test on three synthetic versions of one interactive prototype and twelve fictional concerns. Compare (A) a Figma-style coordinate/layer anchor baseline, (B) a component-lineage resolver, and (C) a task-role-tag resolver. Before implementation, specify expected concern identity, applicability, provenance, lifecycle state, and unique/ambiguous/unresolved/superseded result for moves, renames, one-to-many splits, many-to-one merges, deletions, node-ID reuse, duplicate roles, unauthorized closure, and provenance loss. Continue only if B and C produce identical abstract histories, reject every forbidden transition without state change, surface every seeded ambiguity, and outperform A on retained correct bindings without increasing false unique bindings. Falsify the intervention if either semantic resolver silently chooses an ambiguous target, disagrees with the other at the contract level, loses provenance, permits unauthorized lifecycle mutation, or requires clients to inspect raw IDs or resolver-specific scores.","blocking_evidence":["No production estimate of the frequency or consequence of lost, detached, or misapplied design concerns.","No evidence that reviewers can author semantic target descriptions consistently and at acceptable effort.","No head-to-head mutation-test result for coordinate, lineage, and task-role anchoring.","No adopter commitment, procurement signal, or governance agreement for the exact contract.","No production legal review covering confidential artifacts, participant information, retention, access control, intellectual property, and platform API terms.","No externally benchmarked 2026 cost inputs.","The full feature boundary of the closest Engineering Annotations implementation has not been tested against the proposal's lifecycle and conformance claims."],"research_disposition":"PRIOR_ART_DIFFERENTIATION_STUDY","world_novelty_boundary":"This evaluation found substantial adjacent and overlapping prior art, but it did not measure world novelty, patentability, freedom to operate, market size, or realized impact. Absence of an exact match in these eight sources is not evidence that no such system, publication, patent, internal tool, or practice exists.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Execute the preregistered three-revision, twelve-concern mutation test with coordinate, lineage, and task-role comparators.","Demonstrate zero silent selections for seeded ambiguous cases and zero unauthorized lifecycle mutations.","Measure correct-binding retention, false-unique-binding rate, unresolved rate, adjudication time, and semantic-target authoring effort.","Differentiate the contract experimentally from the Engineering Annotations and W3C selector/state baselines.","Obtain a named design-operations owner willing to evaluate workflow fit and governance.","Benchmark 2026 labor, platform, security-review, and recurring operating costs before deployment planning."],"reason":"Bounded web research verifies the problem, credible actors, implementability, and substantial prior art, but cannot establish the remaining advantage. The decision now depends on live offline implementation and comparative mutation testing, plus later adopter and governance evidence; under the evaluation rule this requires an empirical-research stop."},"proposal_index":3}