{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"bounded_rivalry_governance__futurism_foresight:P3:v0","cell_id":"bounded_rivalry_governance__futurism_foresight","search_queries":["site:gov.uk Government Office for Science horizon scanning toolkit prioritise signals scanning process","horizon scanning weak signals prioritization information overload research paper","strategic foresight crowdsourcing weak signals platform prioritization provenance","ISO 56006 strategic intelligence management horizon scanning standard","horizon scanning platform users submit signals vote prioritize weak signals organization","weak signal database submit evaluate score signals foresight crowdsourcing","strategic foresight signal repository voting prioritization software","horizon scanning prioritization capacity limited signals verification","idea management platform token voting limited points submit ideas innovation challenge official","official software idea submission duplicate detection anonymous review scoring innovation management","foresight platform signal repository team scoring audit provenance weak signals official product","Hines 2019 scanning the horizon systematic literature review horizon scanning methodologies transparency prioritization","site:nist.gov SP 800-53 access control audit logging separation of duties confidentiality","weak signals foresight anonymous review evaluator bias study horizon scanning","site:csrc.nist.gov/pubs/sp/800/53/r5 final Security Privacy Controls Revision 5 update 1"],"sources":[{"source_id":"S1","title":"Horizon Scanning Programme: a new approach for policy making","publisher":"UK Cabinet Office and Government Office for Science","url":"https://www.gov.uk/government/news/horizon-scanning-programme-a-new-approach-for-policy-making","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2013-07-12","accessed_at":"2026-08-03","claims_supported":["The UK Civil Service operated a distributed horizon-scanning network whose officials escalated emerging issues to senior decision-makers.","The programme explicitly sought to minimise duplication and coordinate existing resources.","A small Horizon Scanning Secretariat supported the distributed network, establishing an identifiable adopter and a capacity constraint, although not demand for warrants."]},{"source_id":"S2","title":"UK NSC: horizon scanning","publisher":"UK National Screening Committee, GOV.UK","url":"https://www.gov.uk/government/publications/uk-nsc-approach-to-horizon-scanning/uk-nsc-horizon-scanning","source_class":"OFFICIAL_GUIDANCE","publication_date":"2022-09-29","accessed_at":"2026-08-03","claims_supported":["Systematic horizon scanning uses identification, filtration, prioritisation, assessment, dissemination and follow-up/evaluation.","Prioritisation narrows signals evaluated according to available resources.","Published criteria, stakeholder participation, diverse expertise and transparency are recommended.","The UK NSC identified an opportunity to consolidate and formalise its horizon-scanning functions."]},{"source_id":"S3","title":"Scanning the horizon: a systematic literature review of methodologies","publisher":"BMJ Open","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC6549743/","source_class":"PRIMARY_RESEARCH","publication_date":"2019-05-27","accessed_at":"2026-08-03","claims_supported":["The review mapped 101 publications and found filtration, prioritisation, assessment, dissemination and evaluation to be established horizon-scanning practices.","Established prioritisation methods include rating, ranking, best-worst scaling, Delphi, expert engagement and mixed methods.","A reported Delphi marketplace already lets participants prioritise or purchase a limited number of signals, creating close prior art to scarce warrants.","Process/output audits, conflict declarations, updating, validation and sensitivity assessment are established practices.","The review found unresolved methodological challenges and called for research on comparative effectiveness."]},{"source_id":"S4","title":"Horizon Scanning and Foresight Methods","publisher":"National Academies of Sciences, Engineering, and Medicine, hosted by NCBI Bookshelf","url":"https://www.ncbi.nlm.nih.gov/books/NBK556423/","source_class":"AUTHORITATIVE_SECONDARY","publication_date":"2020","accessed_at":"2026-08-03","claims_supported":["Horizon scanning depends on expert input and operates beyond a firm evidence base.","Salience bias can cause prominent or emotionally compelling issues, especially those advocated by vocal participants, to dominate attention.","Consistently enforced advocacy rules, voting and anonymous feedback are documented mitigations.","Prediction hit rate is an inappropriate sole metric; comparisons with other tools and effects on policy consideration are more suitable evaluation approaches."]},{"source_id":"S5","title":"Building capacity in technology horizon scanning: A guide for policymakers","publisher":"OECD Publishing","url":"https://www.oecd.org/en/publications/building-capacity-in-technology-horizon-scanning_b4f0d383-en.html","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"2026-04-14","accessed_at":"2026-08-03","claims_supported":["The OECD reviewed 129 international horizon-scanning exercises conducted during 2020–2025.","Government units and international initiatives are identifiable organizational adopters.","Interpreting early signals remains challenging, and robust standards and collaboration are still needed."]},{"source_id":"S6","title":"Envisioning Signals: Foresight at the speed of change","publisher":"Envisioning","url":"https://signals.envisioning.com/","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["A current commercial foresight product stores signals with sources, scores and shared research records.","It exposes source-check states and supports configurable scoring and multiple views.","Source-grounded signal repositories are therefore existing product practice, not a distinctive element of this proposal."]},{"source_id":"S7","title":"Lumen — Better Ideas, Better Decisions","publisher":"Lumen Decisions","url":"https://www.lumendecisions.com/","source_class":"COMMERCIAL_FIRST_PARTY","publication_date":"2026","accessed_at":"2026-08-03","claims_supported":["A current decision product offers token voting in which users allocate limited points across options to force prioritisation.","Role controls, asynchronous participation and scoring are commercially implemented adjacent practices.","Limited decision rights are established generally, though this product does not govern weak-signal submission or verification."]},{"source_id":"S8","title":"NIST SP 800-53 Rev. 5: Security and Privacy Controls for Information Systems and Organizations","publisher":"National Institute of Standards and Technology","url":"https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final","source_class":"STANDARD","publication_date":"2020-09 (updates through 2025-08-27)","accessed_at":"2026-08-03","claims_supported":["NIST provides adaptable control families covering access control, audit and accountability, authorization and monitoring, incident response, privacy, integrity and supply-chain risk.","A confidential registry and audit trail are technically governable using established controls, subject to organizational tailoring and authorization.","The standard does not itself resolve employment, whistleblowing, records-retention or sector-specific legal authority."]}],"problem_evidence":{"support":"MODERATE","rationale":"Official and research sources visibly establish scarce-resource prioritisation, duplication concerns, uncertain evidence and salience bias in horizon scanning. They do not establish the proposal's stronger prevalence claims about queue flooding, duplicate relabelling, reciprocal endorsements, provenance withholding or competition for priority credit within a particular organization.","source_ids":["S1","S2","S3","S4","S5"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"The UK Cabinet Office/Government Office for Science and UK NSC are identifiable authorizers or adopters. They explicitly seek coordinated escalation, reduced duplication, efficient prioritisation and consolidated horizon-scanning practice. No source expresses demand for equal nontransferable warrants, confidential priority registration or this integrated governance package.","source_ids":["S1","S2","S5"]},"prior_art":{"proximity":"SUBSTANTIAL_COLLISION","closest_analogues":[{"name":"Limited-signal Delphi marketplace documented by Hines et al.","similarity":"Participants prioritise or purchase a limited number of signals; established horizon-scanning practice also includes ranking, audits and longitudinal evaluation.","remaining_difference":"The proposal allocates equal, resetting, nontransferable rights to submitting teams to nominate claims for scarce verification, rather than allocating participant votes or purchases across already-submitted signals. It adds confidential priority registration and protected mandatory-reporting channels.","source_ids":["S3"]},{"name":"UK NSC filtration-prioritisation-assessment cycle","similarity":"Signals are filtered, ranked under criteria, narrowed according to available resources, assessed and followed up.","remaining_difference":"The UK NSC guidance does not describe team-level nomination caps, identity masking, confidential priority credit, anti-brigading screens or equal quota resets.","source_ids":["S2"]},{"name":"National Academies bias-controlled horizon scanning","similarity":"Uses explicit criteria, diverse experts, voting, anonymous feedback and evaluation to limit salience and group bias.","remaining_difference":"It does not create scarce nomination rights, portfolio awards of verification sprints or rules against warrant pooling and priority-credit gaming.","source_ids":["S4"]},{"name":"Envisioning Signals","similarity":"Maintains a shared signal repository with sources, scoring, verification states and reusable records.","remaining_difference":"The product evidence does not show equal submission warrants, masked adjudication, portfolio selection, appeals or separation of mandatory reports.","source_ids":["S6"]},{"name":"Lumen token voting","similarity":"Limited points force prioritisation when all options cannot be selected.","remaining_difference":"The tokens express preferences over options; they are not equal organizational rights to consume an independent verification queue and provide no foresight-specific provenance or safety governance.","source_ids":["S7"]}],"distinctive_claim_remaining":"Under a fixed verification-hour budget, giving each qualified scanning team an equal, resetting and nontransferable nomination allotment, coupled with confidential priority registration, will reduce duplicate or weakly evidenced nominations and analyst triage hours without materially reducing retrieval of consequential, cross-domain signals relative to both unrestricted intake and an unlimited-nomination common-rubric process. This claim is contrastive and falsifiable; the surrounding filtration, ranking, token-allocation, anonymity, audit and source-traceability mechanisms are already known.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"The component workflow is implementable using established prioritisation, audit, anonymous-feedback, token-allocation, source-traceability and information-control practices. A no-stakes paper replay requires little new technology. Evidence is absent for integrated performance, reviewer reliability, resistance to bundling or off-registry lobbying, identity masking in small teams, mandatory-channel classification, and acceptable missed-signal risk.","source_ids":["S2","S3","S4","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Better allocation of scarce verification effort could improve strategic preparedness and reduce duplicated work, but affected volume, current losses and realized decision impact are unmeasured.","source_ids":["S1","S2","S5"]},"stakeholder_pull":{"score":3,"rationale":"Named public bodies express needs for coordination, reduced duplication and resource-aware prioritisation, but none requests the proposed warrant design.","source_ids":["S1","S2"]},"incremental_advantage":{"score":2,"rationale":"The warrant-plus-priority-registry combination might outperform unrestricted intake, but limited-signal marketplaces and standard prioritisation already address much of the same objective, and no comparative outcome data exist.","source_ids":["S2","S3","S4"]},"distinctiveness_plausibility":{"score":3,"rationale":"The remaining distinction is narrow but testable: equal submission rights tied to verification capacity, confidential priority credit and non-entrenching resets. World novelty and patentability remain unmeasured.","source_ids":["S3","S6","S7"]},"technical_implementability":{"score":4,"rationale":"Forms, timestamping, role-based access, masking, scoring, duplicate detection and audit logs are conventional capabilities. Human adjudication and information governance are harder than the software.","source_ids":["S6","S7","S8"]},"adoption_authority_feasibility":{"score":3,"rationale":"A central foresight office can authorize a no-stakes replay and recommend process changes. Live use requires operational-owner agreement plus privacy, records, employment/contracting and mandatory-reporting review.","source_ids":["S1","S2","S8"]},"evidence_readiness":{"score":3,"rationale":"A preregistered replay with clear comparators is feasible if a partner has exportable archives and effort records; availability, labels and data quality have not been demonstrated.","source_ids":["S2","S3","S4"]},"safety_net_benefit":{"score":4,"rationale":"The proposed first step leaves live intake and decisions unchanged, excludes employment consequences and retains mandatory channels. These safeguards make rollback easy, although confidentiality still needs controlled data handling.","source_ids":["S4","S8"]},"scalability":{"score":3,"rationale":"The registry and scoring workflow scale digitally, but verification analysts, provenance audits, appeals and portfolio-panel attention remain human bottlenecks.","source_ids":["S2","S3","S6"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"10K_TO_50K","scope":"One 6–8 week preregistered paper replay using 100–200 de-identified historical or synthetic signals, 8–12 simulated teams, 3–5 reviewers, fixed analyst hours and a short privacy/security review.","confidence":"LOW","assumptions":["An archive is already exportable and requires only light cleaning.","Existing staff can participate part-time without procurement.","No production software is built and no live decisions are affected.","The estimate is a resource-equivalent range, not a vendor quotation."],"source_ids":["S2","S3","S4"]},"initial_deployment_startup":{"band_2026_usd":"50K_TO_250K","scope":"Configure or build a controlled registry, scoring workflow, access roles, audit logging, duplicate review, policy documents, training and independent appeal arrangements for a limited organizational pilot.","confidence":"LOW","assumptions":["Existing identity, document-management and security infrastructure can be reused.","Pilot scale is approximately 8–20 teams and one business unit.","The range excludes the cost of conducting intensive verification sprints.","Sector-specific legal review is limited rather than litigated."],"source_ids":["S6","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Run a two-quarter parallel pilot with registry administration, masked review, portfolio panels, provenance audits, appeals, mandatory-channel monitoring and a bounded set of verification sprints.","confidence":"LOW","assumptions":["Two to four analyst-equivalents support verification and evaluation.","The old intake remains available during the pilot.","Sensitive-source controls require security and privacy staff participation.","No organization-wide integration or public disclosure is included."],"source_ids":["S2","S3","S8"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Four quarterly cycles covering registry administration, approximately 2–4 verification analysts, panels, audit samples, appeals, security monitoring, outcome review and periodic rule revision.","confidence":"LOW","assumptions":["Verification sprints are moderate research exercises rather than laboratories or commissioned field studies.","Team count remains below roughly 25.","Existing enterprise systems host the registry.","Compensation, litigation, incident response and major software replacement are excluded."],"source_ids":["S2","S3","S6","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Scarce-resource prioritisation, duplication concerns, salience bias and uncertainty are externally documented. The exact strategic gaming behaviors remain an evidence gap rather than a falsification of the broader problem.","source_ids":["S1","S2","S3","S4"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"The UK Cabinet Office/Government Office for Science and UK NSC are identifiable offices with horizon-scanning coordination and prioritisation responsibilities and expressed needs.","source_ids":["S1","S2"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The proposal can test whether equal team nomination warrants plus confidential priority registration reduce triage burden without unacceptable loss of consequential-signal recall against two specified alternatives.","source_ids":["S2","S3","S4"]},"bounded_next_evidence_step":{"status":"YES","reason":"A no-stakes, fixed-budget archival replay can be preregistered with explicit comparators, endpoints and stopping criteria without altering live decisions.","source_ids":["S2","S3","S4"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"For the replay only, live intake, mandatory reports, employment, contracts and operational decisions remain unchanged; standard access and audit controls can protect the archive. This does not authorize live deployment.","source_ids":["S4","S8"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"Broad resource-equivalent bands and scopes can be stated, but organization size, archive condition, verification-sprint intensity, wage rates, security classification and integration requirements are unspecified and no direct quotation was obtained.","source_ids":["S2","S3","S6","S8"]}},"next_evidence_step":"With an authorized organizational partner, preregister and conduct a no-stakes replay of 100–200 de-identified historical nominations spanning at least two past quarters and 8–12 simulated submitting teams. Hold verification hours constant and compare: A) archived unrestricted intake/triage; B) identity-masked common-rubric triage with unlimited team nominations; and C) equal nontransferable warrants plus confidential timestamps and portfolio selection. Have reviewers blind to treatment retrospectively label traceability, decision relevance and whether each signal merited verification using evidence available at the original date. Measure submission and duplicate counts, provenance completeness, analyst minutes, reviewer agreement, identity leakage, thematic and decision-exposure coverage, appeals, mandatory-channel classification errors, and recall of retrospectively qualifying signals. Preregister falsifiers: less than 20% reduction in triage minutes versus B; more than a 10-percentage-point loss in qualifying-signal recall versus either comparator; any mandatory-report diversion; identity inference above 10%; reviewer kappa below 0.40; or concentration/bundling that eliminates the proposed diversity benefit. Synthetic data may test workflow and security but cannot establish organizational prevalence or comparative effect.","blocking_evidence":["No external source establishes the prevalence or cost of queue flooding, reciprocal endorsements, covert pooling, provenance withholding or priority-credit conflict in the proposed adopter setting.","No comparative evidence shows that equal team warrants outperform unrestricted intake, ordinary filtration or a limited-signal Delphi marketplace.","No partner archive has been shown to contain reliable timestamps, submission identities, triage effort, duplicates, later signposts and mandatory-channel labels.","The acceptable recall-loss margin and value of thematic diversity require adopter judgment and cannot be resolved by web research.","Masking effectiveness, bundling behavior, off-registry escalation and reviewer reliability require replay or live observation.","Privacy, records-retention, whistleblowing, employment, contracting and sector-specific reporting authority require jurisdiction- and organization-specific review before live use.","Cost bands remain weak until team count, verification-sprint intensity, labor rates, data classification and integration scope are specified."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"World novelty, patentability, freedom to operate, market size and realized impact were not measured. The bounded search found substantial collision at the mechanism level: resource-aware signal prioritisation, limited-signal purchasing, anonymous feedback, audits, source-grounded repositories and token voting are established. The unmeasured remainder is the integrated use of equal resetting team nomination rights, confidential priority registration, portfolio allocation of verification sprints and protected mandatory channels.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure an authorized partner and a de-identified archival signal stream with reliable effort and outcome metadata.","Preregister the three-way comparator replay, allocation rules, noninferiority margin and falsifiers before viewing results.","Demonstrate at least a 20% reduction in triage effort without more than a 10-percentage-point loss of qualifying-signal recall.","Show zero diversion of mandatory reports, acceptably low identity leakage and at least moderate reviewer agreement.","Quantify bundling, off-registry escalation, duplicate handling, thematic concentration and appeals.","Complete organization-specific privacy, records, whistleblowing, employment/contract and security review before any live pilot.","Replace broad resource-equivalent cost bands with partner-specific labor, integration, security and verification-sprint estimates."],"reason":"Bounded web research verifies a real resource-prioritisation problem, credible authorizers and extensive adjacent practice, while also revealing substantial prior-art collision. The remaining incremental-effect, workflow-safety and cost questions require proprietary archival data and a partnered replay rather than more web search. Under the controller rule, that requires STOP_EMPIRICAL_RESEARCH_NEEDED, with repairable set false."},"proposal_index":3}