{"schema_version":1,"research_id":"eoa_inverse_innovation_exp05_external_evaluation_20260803","source_assessment_id":"negative_space_design__rhetoric:P2:v0","cell_id":"negative_space_design__rhetoric","search_queries":["site:gov.uk deliberation options alternatives guidance open question decision making","binary choice framing options generate alternatives empirical study decision making","deliberative democracy briefing materials balanced options alternatives official guidance","choice architecture visual layout binary options perceived exhaustive experiment","partition dependence options decision making alternatives generated framing experiment","option generation decision making failure to generate alternatives empirical research","\"generate alternatives\" \"deliberation\" briefing materials","site:oecd.org deliberative process briefing materials options alternatives","official questionnaire design response options exhaustive include other option guidance","site:pewresearch.org survey question response options exhaustive other specify","site:census.gov questionnaire design response categories exhaustive other","official structured decision making generate creative alternatives government guide","OECD good practice principles deliberative processes clear task briefing materials alternatives recommendation","Council Europe deliberative democracy recommendation briefing information balanced options","USGS structured decision making alternatives generate options official guide PDF","site:gov.uk deliberative democracy toolkit options briefing materials","\"How subjective grouping of options influences choice and allocation\" APA","\"Option generation in decision making: ideation beyond memory retrieval\" Frontiers","site:frontiersin.org \"Option generation in decision making\""],"sources":[{"source_id":"S1","title":"Writing Survey Questions","publisher":"Pew Research Center","url":"https://www.pewresearch.org/writing-survey-questions/","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["Response options, their descriptions, number, and ordering influence answers.","In a Pew comparison, 43% of open-ended respondents named an issue outside five supplied choices, versus 8% of respondents given the five choices who volunteered another issue.","Closed response lists should be exhaustive; open-ended response affordances should make the expected action evident.","Randomized split-form testing can attribute response differences to presentation changes."]},{"source_id":"S2","title":"Why do some open-ended survey questions result in higher item nonresponse rates than others?","publisher":"Pew Research Center","url":"https://www.pewresearch.org/decoded/2021/10/14/why-do-some-open-ended-survey-questions-result-in-higher-item-nonresponse-rates-than-others/","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"2021-10-14","accessed_at":"2026-08-03","claims_supported":["Open-ended questions impose more effort and have materially higher nonresponse than closed-ended questions in Pew's American Trends Panel.","Across 92 open-ended questions, average nonresponse was about 18%, versus roughly 1–2% for closed-ended questions.","Nonresponse increased with cognitive burden, establishing a foreseeable failure mode for an alternative-generation field."]},{"source_id":"S3","title":"Creating More and Better Alternatives for Decisions Using Objectives","publisher":"INFORMS, Operations Research","url":"https://pubsonline.informs.org/doi/10.1287/opre.2015.1411","source_class":"PRIMARY_RESEARCH","publication_date":"2015-09-10","accessed_at":"2026-08-03","claims_supported":["Across five empirical studies, people facing important decisions generated fewer than half of their potentially desirable alternatives.","Overlooked alternatives had average quality comparable to initially identified alternatives.","Objectives-based prompts increased the number and quality of generated alternatives, showing that structured generation before evaluation is established and can outperform an unstructured invitation."]},{"source_id":"S4","title":"How subjective grouping of options influences choice and allocation: diversification bias and the phenomenon of partition dependence","publisher":"American Psychological Association, indexed by PubMed","url":"https://pubmed.ncbi.nlm.nih.gov/16316290/","source_class":"PRIMARY_RESEARCH","publication_date":"2005-11","accessed_at":"2026-08-03","claims_supported":["Six studies found that subtle changes in the grouping or presentation of options systematically altered allocations and choices.","Partition effects were weaker among experienced participants and those with stronger intrinsic preferences, identifying important moderators.","The evidence supports presentation-sensitive choice generally but does not directly test whether a visually reserved blank region prevents perceived binary exhaustiveness."]},{"source_id":"S5","title":"Option generation in decision making: ideation beyond memory retrieval","publisher":"Frontiers in Psychology","url":"https://www.frontiersin.org/articles/10.3389/fpsyg.2014.01584","source_class":"PRIMARY_RESEARCH","publication_date":"2015-01-22","accessed_at":"2026-08-03","claims_supported":["An individual-differences study with 141 participants treated option generation as a distinct pre-choice component of ill-structured decision making.","Ideation ability predicted option-generation fluency and diversity beyond memory, executive-control, intelligence, and domain-experience measures.","Generating more options increased the quality of the best option but reduced average option quality, supporting feasibility review rather than automatic endorsement of submissions."]},{"source_id":"S6","title":"Good practice principles for deliberative processes for public decision making","publisher":"OECD","url":"https://www.oecd.org/en/publications/innovative-citizen-participation-and-new-democratic-institutions_339306da-en/full-report/component-9.html","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2020-06-10","accessed_at":"2026-08-03","claims_supported":["The principles are intended for public decision makers commissioning deliberative processes and practitioners designing them.","The task should be neutral, clear, and linked to a defined public problem; participants need accurate, relevant, accessible information.","Good processes provide inclusive participation, individual reflection time, skilled facilitation, transparency, and public-authority accountability.","Commissioning public authorities are credible authorizers, but the source expresses no demand for this particular page layout."]},{"source_id":"S7","title":"Participatory modeling and structured decision making","publisher":"U.S. Geological Survey","url":"https://www.usgs.gov/publications/participatory-modeling-and-structured-decision-making","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2016-12-31","accessed_at":"2026-08-03","claims_supported":["USGS describes a transparent, defensible, replicable decision workflow used for complex natural-resource choices.","The workflow defines context and objectives, devises alternatives, evaluates consequences, assesses trade-offs, and only then implements a decision.","Stakeholders can participate in devising alternatives, providing a credible government adopter context and strong functional prior art for separating generation from evaluation."]},{"source_id":"S8","title":"What is Deliberative Polling®?","publisher":"Stanford Deliberative Democracy Lab","url":"https://deliberation.stanford.edu/what-deliberative-pollingr","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["Deliberative Polling uses carefully balanced, public briefing materials, trained moderators, small-group question development, and before/after measurement.","The established workflow demonstrates that briefing artifacts and staged deliberation are operationally implementable.","It is adjacent prior art but does not describe a proportionally significant, sponsor-inviolable residual field next to a binary briefing."]}],"problem_evidence":{"support":"MODERATE","rationale":"The underlying problem is visible and consequential: supplied response sets and subtle option grouping change expressed choices, and people overlook potentially valuable alternatives. Pew's open-versus-closed comparison is especially relevant, while decision research establishes both partition dependence and incomplete option generation. However, none of the eight sources directly observes a filled two-block deliberative briefing or tests the claimed visual-closure mechanism, so the candidate-specific prevalence and effect size remain unverified.","source_ids":["S1","S3","S4","S5"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"OECD identifies commissioning public authorities and deliberative-process practitioners as authorizers needing neutral tasks, accessible evidence, inclusion, reflection, and participant influence. USGS supplies an identifiable government workflow in which managers and stakeholders devise alternatives before evaluation, while Stanford operates briefing-based deliberation. These establish credible adopters and expressed need for alternative generation, but no organization was found requesting this specific residual-field template.","source_ids":["S6","S7","S8"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"Explicit open-ended or residual response option","similarity":"Pew's survey practice explicitly contrasts supplied choices with an open-ended response and advises making the open-response action evident; this performs the proposal's central function of admitting an unlisted possibility.","remaining_difference":"The proposal makes the affordance a proportionally significant, visually adjacent, sponsor-inviolable region and delays comparison until submissions are recorded.","source_ids":["S1","S2"]},{"name":"Objectives-guided alternative generation before evaluation","similarity":"Siebert and Keeney empirically test structured alternative creation, while USGS structured decision making devises alternatives before consequence and trade-off evaluation. This substantially overlaps the proposal's procedural sequence.","remaining_difference":"Neither source claims that protected blank page area itself changes perceived exhaustiveness beyond an equally explicit generation prompt.","source_ids":["S3","S7"]},{"name":"Deliberative Polling briefing and staged discussion","similarity":"Stanford already combines balanced briefing material, trained moderation, participant-generated questions, and pre/post measurement in a deliberative workflow.","remaining_difference":"It does not reserve an initially empty third region for participant-originated options or prohibit sponsor use of that region.","source_ids":["S8"]},{"name":"OECD good-practice deliberative process","similarity":"The OECD framework already requires neutral framing, accessible information, inclusion, reflection time, facilitation, transparency, and accountable public-authority sponsorship.","remaining_difference":"It is process-level guidance and does not specify a binary-closure diagnostic or residual-field artifact.","source_ids":["S6"]}],"distinctive_claim_remaining":"Holding the decision question, A/B content, mandatory context, prompt wording, private generation time, and response capacity constant, placing a proportionally significant and sponsor-inviolable residual field visibly adjacent to A and B will reduce perceived exhaustiveness and increase blinded-coder-rated hybrid, conditional, intermediate, or distinct alternatives relative to an equally explicit text-only/off-page alternative-generation prompt, without a material increase in missing-context or incomplete-material interpretations.","confidence":"HIGH"},"implementation_evidence":{"support":"STRONG","rationale":"The intervention requires ordinary document templating, accessible private response channels, timed facilitation, transcription, deduplication, blinded coding, and record retention. Comparable components operate in survey research, structured decision making, OECD deliberation, and Deliberative Polling. No specialized technology is required. Workflow and data risks are manageable through consent, optional participation, accessible modes, private initial responses, provenance labels, feasibility review, and audit-preserving records. Binding use would still require the decision owner to verify statutory eligibility, records, privacy, accessibility, and mandatory-disclosure requirements; the proposed nonbinding mock test avoids unresolved legal authority.","source_ids":["S1","S2","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Overlooked alternatives can be as valuable as generated ones, and supplied response sets measurably affect answers; however, prevalence and downstream decision harm in real deliberative briefings are unmeasured.","source_ids":["S1","S3","S4"]},"stakeholder_pull":{"score":3,"rationale":"Government and deliberative institutions express need for neutral framing, co-created solutions, accessible evidence, and explicit alternative generation, but no adopter requests the exact artifact.","source_ids":["S6","S7","S8"]},"incremental_advantage":{"score":2,"rationale":"Open-ended residual responses and structured pre-evaluation option generation already supply most of the functional benefit. Only the added effect of visually protected, adjacent blank space remains.","source_ids":["S1","S3","S7"]},"distinctiveness_plausibility":{"score":2,"rationale":"Presentation and grouping can influence choice, making the visual hypothesis plausible, but no direct evidence separates protected negative space from prompt, response-capacity, and generation-time effects.","source_ids":["S1","S4"]},"technical_implementability":{"score":5,"rationale":"A template, accessible response form, facilitation script, and coding sheet use mature document and research methods with no substantive technical dependency.","source_ids":["S1","S6","S7","S8"]},"adoption_authority_feasibility":{"score":4,"rationale":"Briefing authors and chairs can usually authorize a reversible mock artifact; decision owners retain eligibility authority. Binding public use may require jurisdiction-specific review.","source_ids":["S6","S7","S8"]},"evidence_readiness":{"score":4,"rationale":"The contrast can be isolated with randomized scenarios, prompt-matched comparators, private responses, blinded coding, and explicit interpretation and confusion measures. A powered effect test, not the proposed 12–16-person usability sample alone, is needed.","source_ids":["S1","S2","S3","S4"]},"safety_net_benefit":{"score":3,"rationale":"The field could surface omitted conditions and alternatives before commitment, but it can also imply endorsement or incompleteness; provenance, feasibility labels, abstention, and rollback are necessary.","source_ids":["S2","S5","S6"]},"scalability":{"score":4,"rationale":"Templates and scripts are inexpensive to reproduce across meetings, though accessible facilitation, coding, governance, and context-specific legal review do not scale to zero cost.","source_ids":["S6","S7","S8"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"Design, preregistration, accessibility review, recruitment and incentives for a powered prompt-matched randomized mock study, facilitation, blinded coding, analysis, and a short report.","confidence":"MODERATE","assumptions":["Approximately 60–100 consenting participants rather than 12–16 if estimating an incremental comparative effect.","Two or three non-sensitive scenarios and three conditions: filled binary baseline, explicit prompt-matched alternative generation, and adjacent Reserved Residual Field.","Includes researcher, facilitator, designer, accessibility, coder, and participant resource-equivalent time; excludes institutional overhead and major software development."],"source_ids":["S1","S2","S6","S8"]},"initial_deployment_startup":{"band_2026_usd":"UNDER_10K","scope":"Adapt one existing briefing template, create an accessible digital/print response channel, prepare facilitator and recorder instructions, and complete decision-owner and records review for one nonbinding deployment.","confidence":"MODERATE","assumptions":["Existing document, survey, and meeting tools are used.","No custom platform, procurement, translation program, or binding ballot integration is required.","One author, facilitator, accessibility reviewer, recorder, and decision owner contribute limited staff time."],"source_ids":["S6","S7","S8"]},"operational_launch":{"band_2026_usd":"10K_TO_50K","scope":"Launch in three to five real but nonbinding deliberations with training, accessible formats, participant support, independent evaluation, audit records, and iteration.","confidence":"LOW","assumptions":["Each event uses an existing deliberative program and briefing process.","Costs include staff time and participant support but not the host program's full deliberation budget.","Translation, travel, representative recruitment, or legally mandated consultation could move the cost above this band."],"source_ids":["S6","S7","S8"]},"annual_recurring":{"band_2026_usd":"10K_TO_50K","scope":"Maintain templates and accessibility, train facilitators, review records and submissions, monitor confusion and differential participation, and evaluate a modest annual portfolio of uses.","confidence":"LOW","assumptions":["Approximately 10–20 uses per year within one organization.","No dedicated software team or large representative assembly is included.","Annual cost is mainly staff resource-equivalent time; high-volume public consultation would require re-estimation."],"source_ids":["S6","S7","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Independent empirical and first-party evidence shows that supplied option sets and grouping alter responses and that decision makers overlook potentially desirable alternatives, although the exact briefing-layout mechanism remains untested.","source_ids":["S1","S3","S4","S5"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"OECD commissioning public authorities, USGS structured-decision managers, and briefing-based deliberation practitioners are credible adopter classes with expressed need for neutral framing, accessible evidence, stakeholder option generation, and staged deliberation.","source_ids":["S6","S7","S8"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The incremental claim isolates the adjacent, proportionally significant protected field against an equally explicit alternative-generation prompt while holding content, time, and response capacity constant.","source_ids":["S1","S4"]},"bounded_next_evidence_step":{"status":"YES","reason":"A preregistered, finite randomized mock study can compare baseline, prompt-only, and Reserved Residual Field conditions using perceived exhaustiveness, coded option novelty, interpretation, confusion, and missing-context outcomes.","source_ids":["S1","S2","S3","S4"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"A consenting, nonbinding, non-sensitive mock test is reversible and does not confer eligibility or endorsement. Optional participation, accessible modes, provenance labels, preserved context, private initial responses, and rollback address the salient risks.","source_ids":["S2","S5","S6"]},"credible_cost_scope_and_range":{"status":"YES","reason":"The intervention uses established document, survey, facilitation, and coding workflows; resource-equivalent bands are scoped with participant-volume, staffing, accessibility, and deployment assumptions. They are planning estimates rather than vendor quotations.","source_ids":["S1","S6","S7","S8"]}},"next_evidence_step":"Run a preregistered randomized or counterbalanced mock study with approximately 60–100 consenting participants and at least two matched, non-sensitive decision scenarios. Compare: (A) a filled A/B briefing followed immediately by evaluation; (B) the same briefing plus the exact non-exhaustiveness prompt, equal private generation time, and equal response capacity supplied off-page or in a conventional compact response form; and (C) the same prompt, time, and capacity presented as the visibly adjacent, proportionally significant, sponsor-inviolable Reserved Residual Field. Randomize scenario, condition, and order; preserve identical constraints, evidence, safety disclosures, typography, and A/B content. Before discussion, measure whether A/B appear exhaustive, comprehension of the response affordance, and privately proposed options. Have condition-blind coders classify duplicate, hybrid, condition, intermediate, distinct, and uninterpretable submissions and adjudicate disagreements. Also measure abstention, missing-context reports, incomplete-material interpretations, pressure, trust, and completion time. Falsify the incremental claim if condition C does not outperform prompt-matched condition B on preregistered perceived-exhaustiveness and distinct-option outcomes, if effects occur only against condition A, or if C materially increases confusion, missing-context interpretation, pressure, or accessibility failures. Do not infer decision quality, realized impact, or safe binding adoption from this mock test.","blocking_evidence":["No direct prevalence study shows how often visually filled binary briefing sheets suppress partition challenges in actual deliberations.","No experiment isolates adjacent protected negative space from the effects of an explicit open-ended prompt, response capacity, and added generation time.","The candidate's proposed 12–16-person test is suitable for usability and measurement debugging but is generally too small for a stable incremental effect estimate.","No live adopter has committed to testing this exact template or documented jurisdiction-specific accessibility, records, privacy, and decision-eligibility requirements.","No evidence yet shows that additional submissions improve final decision quality rather than merely increasing option count, confusion, or infeasible suggestions.","The 2026-USD bands are resource-equivalent planning estimates, not externally validated procurement quotes."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"This eight-source bounded web review found substantial functional prior art for open-ended residual responses, structured alternative generation before evaluation, balanced briefing materials, and deliberative-process safeguards. It did not locate a direct test or documented practice combining a proportionally significant, visually adjacent, sponsor-inviolable empty region with governed population before comparison. That absence is not proof of world novelty. World novelty, patentability, freedom to operate, market size, and realized impact remain unmeasured.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Preregister the prompt-matched three-condition design so the visual-field effect is separated from wording, response capacity, and additional thinking time.","Complete an accessible 12–16-person usability pilot, then a powered approximately 60–100-person comparative mock test with blinded coding and scenario/order randomization.","Set quantitative falsifiers for perceived exhaustiveness, genuinely distinct alternatives, confusion, missing context, pressure, trust, completion time, and accessibility failure before collecting outcomes.","Obtain one named deliberative-program or structured-decision partner and document its authorizer, records/privacy duties, mandatory context, accessibility requirements, and rule that recorded alternatives are not automatically eligible or endorsed."],"reason":"Bounded web research establishes the general problem, credible adopters, substantial prior-art collision, implementation feasibility, and a precise residual claim. The remaining question is causal and cannot be resolved by additional web search: it requires participant testing that isolates the protected adjacent field from an ordinary explicit open-ended prompt. Under the controller rule, that fieldwork requirement mandates STOP_EMPIRICAL_RESEARCH_NEEDED."},"proposal_index":2}