{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"predictive_residual_processing__futurism_foresight:P3:v0","cell_id":"predictive_residual_processing__futurism_foresight","search_queries":["site:gov.uk Government Office for Science Futures Toolkit horizon scanning sources bias 2024 pdf","site:oecd.org strategic foresight public sector horizon scanning weak signals evidence official","performative prediction foresight self fulfilling prophecy organizational actions academic","campaign attribution second order effects earned media attribution official documentation","strategic foresight self fulfilling prophecy horizon scanning organizational intervention evidence reflexivity paper","foresight self-generated signals horizon scanning feedback loop evidence organization action","evaluation contribution analysis theory of change causal attribution official guide pdf","W3C PROV standard provenance entities activities agents recommendation","\"The Strongness of Weak Signals\" self-reference paradox anticipatory systems pdf","\"self-reference\" \"horizon scanning\" weak signals foresight","\"performativity of strategic foresight tools\" full text","site:gov.uk horizon scanning source credibility bias external world organization","site:proceedings.mlr.press performative prediction Perdomo 2020","site:ico.org.uk data protection impact assessment monitoring public data guidance purpose limitation","site:ico.org.uk lawful basis public sources personal data monitoring guidance","\"The performativity of strategic foresight tools\" pdf","\"Horizon scanning as an activation device\" pdf","\"The performativity of strategic foresight tools\" DOI"],"sources":[{"source_id":"S1","title":"The Futures Toolkit HTML","publisher":"UK Government Office for Science","url":"https://www.gov.uk/government/publications/futures-toolkit-for-policy-makers-and-analysts/the-futures-toolkit-html","source_class":"OFFICIAL_GUIDANCE","publication_date":"2024-08-29","accessed_at":"2026-08-03","claims_supported":["Government horizon scanning seeks evidence of changes in the external world to inform policy and strategy.","The official workflow gathers source-linked, metadata-bearing scans and requires rigorous analysis.","The toolkit warns about intellectual bias, groupthink, incomplete scanning, and reliance on interpretations rather than original sources.","A typical exercise involves desk research, workshops, multiple scanners, and sustained weekly source collection, supporting the feasibility and resource assumptions."]},{"source_id":"S2","title":"Weak signals and trend analysis: horizon scanning","publisher":"UK Government Office for Science","url":"https://www.gov.uk/government/publications/weak-signals-and-trend-analysis-horizon-scanning","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2025-08-28","accessed_at":"2026-08-03","claims_supported":["The Department for Environment, Food and Rural Affairs Futures team is an identifiable prospective adopter with an operating horizon-scanning function.","Defra commissioned external work in 2024–2025 as part of a broader initiative to develop horizon-scanning capability.","A public-sector organization has demonstrated willingness to authorize and fund improvements to this workflow, although no demand for the proposed echo filter is stated."]},{"source_id":"S3","title":"The performativity of strategic foresight tools: Horizon scanning as an activation device in strategy formation within a UK financial institution","publisher":"Technological Forecasting and Social Change; manuscript hosted by University of Huddersfield Research Portal","url":"https://pure.hud.ac.uk/ws/files/28198463/TFS_2019_949_Revise_and_Resubmit_Final.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"2021-01-01","accessed_at":"2026-08-03","claims_supported":["A three-year longitudinal case study found that a horizon scan actively shaped strategy rather than merely describing an external environment.","The scan enrolled and excluded participants, consolidated authority, persuaded decision-makers, justified decisions, and stimulated action consistent with an anticipated future.","The study supports the proposed feedback mechanism—foresight can help create the organizational reality it later observes—but does not document later echoes being misclassified as independent evidence."]},{"source_id":"S4","title":"Performative Prediction","publisher":"Proceedings of Machine Learning Research","url":"https://proceedings.mlr.press/v119/perdomo20a.html","source_class":"PRIMARY_RESEARCH","publication_date":"2020-07-13","accessed_at":"2026-08-03","claims_supported":["Predictions used for decisions can influence the outcomes they predict and thereby change the observed data distribution.","Ignoring performativity can appear as distribution shift, so learning from post-intervention observations without modeling the intervention is unsafe.","The paper supplies a formal adjacent framework for action-dependent observations, but not a foresight-source attribution product."]},{"source_id":"S5","title":"Magenta Book Annex A: Analytical methods for use within an evaluation","publisher":"HM Treasury and UK Evaluation Task Force","url":"https://www.gov.uk/government/publications/the-magenta-book/magenta-book-annex-a-analytical-methods-for-use-within-an-evaluation-html","source_class":"OFFICIAL_GUIDANCE","publication_date":"2026-05-15","accessed_at":"2026-08-03","claims_supported":["Process tracing tests hypothesized causal mechanisms against observable implications and explicit alternatives.","Contribution analysis examines whether an intervention plausibly contributed to observed outcomes and requires assessment of other influencing factors.","Theory-based methods can support causal attribution but do not provide precise effect sizes and may be time-consuming and resource-intensive.","These established methods are close prior art and appropriate comparators for a retrospective study."]},{"source_id":"S6","title":"PROV-O: The PROV Ontology","publisher":"World Wide Web Consortium","url":"https://www.w3.org/TR/prov-o/","source_class":"STANDARD","publication_date":"2013-04-30","accessed_at":"2026-08-03","claims_supported":["A standardized, machine-processable representation exists for entities, activities, agents, derivation, attribution, responsibility, plans, timestamps, and collections.","The proposed action ledger, source lineage, transformation trace, and actor responsibility fields can be implemented using established provenance primitives.","PROV-O records derivation and responsibility but does not infer causal independence or supply an echo model."]},{"source_id":"S7","title":"Get started with attribution","publisher":"Google Analytics Help","url":"https://support.google.com/analytics/answer/10596866?hl=en","source_class":"OFFICIAL_PRODUCT_DOCUMENTATION","publication_date":"not stated","accessed_at":"2026-08-03","claims_supported":["A deployed first-party product attributes outcomes across multiple temporally ordered touchpoints using path data, counterfactual comparisons, and fractional credit.","The documentation demonstrates technical feasibility of time- and path-aware attribution models and holdback-informed comparisons.","Advertising attribution optimizes credit for key events; it does not protect foresight evidence from self-generated confirmation or preserve full-source reconstruction."]},{"source_id":"S8","title":"A guide to lawful basis","publisher":"UK Information Commissioner's Office","url":"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/lawful-basis/a-guide-to-lawful-basis/","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2026-04-02","accessed_at":"2026-08-03","claims_supported":["Organizations must identify and document a lawful basis before processing personal information, including information obtained indirectly.","Purposes and lawful bases must be disclosed, and changed purposes may require a new basis.","Special-category or criminal-offence information requires additional conditions, creating an authority constraint for stakeholder-level scanning and residual archives."]}],"problem_evidence":{"support":"MODERATE","rationale":"The exact failure—second-order organizational echoes later being counted as independent horizon-scan evidence—was not directly measured in the opened literature. Nevertheless, official guidance confirms that scans are intended to represent external change and are vulnerable to bias and incomplete source interpretation, while a longitudinal study directly shows that horizon scanning can shape strategy and create part of the reality it describes. Performative-prediction research independently establishes the general action-dependent-data mechanism. Prevalence, frequency, decision harm, and baseline analyst error remain unmeasured.","source_ids":["S1","S3","S4"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Defra is an identifiable possible adopter/authorizer: its Futures team commissioned external work to build horizon-scanning capability. The Government Office for Science supplies an official workflow and commissioning audience. This establishes institutional capability and spending authority, but neither source expresses demand for a self-echo residual filter or reports this specific contamination problem.","source_ids":["S1","S2"]},"prior_art":{"proximity":"ADJACENT_PRIOR_ART","closest_analogues":[{"name":"Contribution analysis and process tracing","similarity":"Both begin with a hypothesized intervention pathway, identify expected observable implications, collect evidence, consider alternative explanations, and make a bounded contribution claim.","remaining_difference":"The proposal freezes a pre-action echo prediction, continuously represents observations as expected self-echo plus residual, preserves reconstructibility, and routes uncertain or protected evidence through explicit fallback; official evaluation guidance does not prescribe that residual-processing architecture.","source_ids":["S5"]},{"name":"Data-driven multi-touch attribution","similarity":"Both use timing, paths, exposures, counterfactual comparisons, and fractional or uncertain attribution to distinguish an intervention's contribution from other activity.","remaining_difference":"Google Analytics assigns conversion credit for advertising optimization; it does not qualify strategic evidence as independent, retain criticism and harm through protected bypasses, or reconstruct a complete horizon-scan evidence stream.","source_ids":["S7"]},{"name":"Performative-prediction frameworks","similarity":"Both explicitly model the fact that a decision or prediction changes the distribution subsequently observed and therefore separates learning from an assumed fixed environment.","remaining_difference":"The research framework concerns prediction-dependent data distributions and model stability, not source-level foresight provenance, pre-action manifests, human attribution review, raw-source auditing, or evidence-weighting residuals.","source_ids":["S4"]},{"name":"W3C PROV provenance graphs","similarity":"PROV-O can encode actions, sources, actors, derivation, responsibility, timestamps, plans, and transformation lineage required by the proposal.","remaining_difference":"PROV-O records provenance but does not estimate whether a later source was caused by an organizational action, compute residual evidence, or prescribe safety bypass and decompression rules.","source_ids":["S6"]}],"distinctive_claim_remaining":"Against ordinary source deduplication plus manual provenance/timing review, a frozen pre-action echo model with reconstructible residual cards will reduce false classification of self-generated activity as independent evidence and reduce reviewer time, without materially increasing false cancellation of independent, critical, dissenting, or protected signals, and with total manifest, model, audit, and fallback cost below the review cost avoided.","confidence":"MODERATE"},"implementation_evidence":{"support":"MODERATE","rationale":"The main components are technically conventional: structured scanning records, provenance graphs, intervention theories, path-aware attribution, blinded review, and counterfactual or contribution analysis. The organization that owns the scan can generally authorize a read-only shadow study. Feasibility is weakened by non-identifiability from overlapping actions, incomplete outbound ledgers, scarce counterfactuals, strategic manifest gaming, and the absence of validated labels for causal independence. Personal or sensitive information requires a documented lawful basis and potentially additional conditions. Raw-source retention, independent adjudication, protected-signal bypass, no live decision effect, and automatic fallback bound first-study safety, but those controls have not been tested.","source_ids":["S1","S5","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":3,"rationale":"Avoiding endogenous confirmation could materially improve strategic decisions, but no source measures prevalence, decision error, or realized harm from the exact problem.","source_ids":["S1","S3","S4"]},"stakeholder_pull":{"score":3,"rationale":"Defra has commissioned horizon-scanning capability work and GO-Science supports an institutional foresight workflow, but expressed demand for this specific control was not found.","source_ids":["S1","S2"]},"incremental_advantage":{"score":3,"rationale":"Pre-action freezing, reconstructible residuals, protected bypasses, and raw audits could improve on manual provenance and contribution analysis; comparative accuracy, time savings, and net cost are untested.","source_ids":["S5","S7"]},"distinctiveness_plausibility":{"score":3,"rationale":"The searched sources contain all major neighboring ideas but no exact foresight-specific combination. This is only bounded-search distinctiveness, not world novelty.","source_ids":["S4","S5","S6","S7"]},"technical_implementability":{"score":4,"rationale":"Existing metadata workflows, provenance standards, attribution models, and evaluation methods cover most components. Causal identifiability, not software construction, is the principal technical constraint.","source_ids":["S1","S5","S6","S7"]},"adoption_authority_feasibility":{"score":3,"rationale":"A foresight owner can authorize a retrospective shadow workflow, but access to action ledgers, communications, procurement records, scans, and possible personal data requires cross-functional authority and documented lawful bases.","source_ids":["S2","S8"]},"evidence_readiness":{"score":2,"rationale":"The study can be pre-registered, but decisive evidence requires proprietary action and scan records, blinded reviewers, independent adjudication, and measured workflow costs unavailable on the open web.","source_ids":["S1","S5"]},"safety_net_benefit":{"score":4,"rationale":"Full-source retention, independent audit, protected-signal bypass, and complete-review fallback directly address over-cancellation and strategic misuse, although their effectiveness remains untested.","source_ids":["S5","S6","S8"]},"scalability":{"score":2,"rationale":"Reusable provenance schemas are scalable, but pre-action manifests, human adjudication, overlapping interventions, and action-specific model calibration may grow faster than avoided review effort.","source_ids":["S1","S5","S6"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"One pre-registered retrospective shadow study covering one completed anticipatory action, one fixed observation window, two blinded review arms, an independent adjudicated sample, legal/privacy screening, and analysis of accuracy, reconstruction, protected-signal recall, time, and cost.","confidence":"LOW","assumptions":["Existing action, source, and brief records are accessible and need no new collection.","Approximately 100–500 source capsules and 4–8 reviewers are involved.","A lightweight prototype and manual adjudication are sufficient.","No direct 2026 labor-price source was found; this is a resource-equivalent estimate derived from the documented multi-person research, workshop, evaluation, and provenance workload."],"source_ids":["S1","S5","S6","S8"]},"initial_deployment_startup":{"band_2026_usd":"250K_TO_1M","scope":"Production design for one organization: action-manifest service, scan and provenance integration, model/version registry, residual review interface, audit sampling, access controls, fallback workflow, governance documentation, and validation.","confidence":"LOW","assumptions":["One existing horizon-scanning platform and several internal action systems require integration.","No custom foundation model or new surveillance collection is built.","Security, privacy, and records-management reviews are included.","No vendor quote or validated engineering estimate was available."],"source_ids":["S1","S6","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Six-to-twelve-month dual-run launch in one foresight unit, including training, action-owner onboarding, model stewardship, independent review, audits, fallback exercises, monitoring, and evaluation.","confidence":"LOW","assumptions":["Residual classifications remain advisory during dual running.","Multiple departments supply manifests and records.","Independent reviewers are organizationally separated from action owners.","The band includes substantial staff time, not only cash expenditure."],"source_ids":["S1","S2","S5","S8"]},"annual_recurring":{"band_2026_usd":"250K_TO_1M","scope":"Ongoing manifest preparation, model and taxonomy maintenance, analyst review, independent raw-source audits, provenance storage, privacy review, incident handling, resynchronization, and periodic comparative evaluation for one mature organizational program.","confidence":"LOW","assumptions":["The system covers several material anticipatory actions per year rather than every communication.","Human review and independent audit remain mandatory.","Costs fall sharply if used only episodically and rise above this band for enterprise-wide continuous coverage.","No externally validated operating-cost benchmark was found."],"source_ids":["S1","S5","S6","S8"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"UNCERTAIN","reason":"External evidence verifies that horizon scans are performative and that decisions can alter later observed data, but no opened source directly documents the proposed second-order echo being counted as independent foresight evidence or establishes its prevalence.","source_ids":["S1","S3","S4"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Defra's Futures team commissioned work to develop horizon-scanning capability, and GO-Science provides the official workflow used by government teams; this establishes a credible class and named example of adopter/authorizer, though not product-specific demand.","source_ids":["S1","S2"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The proposal can be compared with manual provenance/timing review on false independence, false cancellation, reconstruction, protected-signal recall, reviewer time, fallback, and total cost.","source_ids":["S5","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"A one-action, fixed-window, retrospective shadow study with frozen pre-action information, two review arms, retained raw sources, and predeclared halt thresholds is bounded and decision-isolated.","source_ids":["S1","S5","S6"]},"no_unresolved_safety_or_authority_stop":{"status":"UNCERTAIN","reason":"Read-only shadow use and full-source retention reduce immediate decision risk, but a specific partner must confirm authority, lawful basis, privacy notice obligations, sensitive-data conditions, reviewer independence, and access to cross-functional action records before work starts.","source_ids":["S8"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"Work packages and cost drivers are scoped from official workflows and standards, but no direct 2026 labor rates, vendor quotes, source volume, integration inventory, or audit sample size were externally verified.","source_ids":["S1","S5","S6","S8"]}},"next_evidence_step":"With one authorized foresight partner, pre-register a retrospective shadow study of one completed anticipatory action and a fixed post-action window. Reconstruct and freeze the action manifest and echo prediction using only records available before launch. Randomize matched source sets to (A) the ordinary deduplication plus manual provenance/timing workflow and (B) predicted-echo-plus-residual cards with identical access to full sources. Have a separate blinded panel adjudicate a random and risk-stratified sample for plausible organizational connection, independence, criticism, harm, and protected-signal status without claiming causal proof. Primary comparators are false-independent and false-cancellation rates, reviewer minutes, inter-rater agreement, full-vector reconstruction, independent/protected-signal recall, fallback frequency, and all-in labor. Before opening outcomes, set non-inferiority tolerances for reconstruction and protected-signal recall, a zero-tolerance halt for cancellation of a protected signal, and a minimum worthwhile improvement such as at least 20% lower reviewer time or false-independent classification without worse false cancellation. Falsify the intervention if it fails those thresholds, cannot beat simple timing/provenance tags or contribution analysis, cannot reproduce attribution categories, cannot reconstruct the evidence set, or costs at least as much as complete manual review.","blocking_evidence":["No direct evidence establishes how often self-generated second-order echoes contaminate horizon scans or materially change decisions.","No adopter has expressed demand for this exact intervention or committed records, reviewers, authority, or funding.","No labeled ground truth exists for whether distinct downstream sources are organizationally induced, independent, or jointly caused.","The incremental advantage over simple timing/provenance tags, contribution analysis, and full manual review has not been measured.","The completeness and honesty of outbound-action manifests and the severity of overlapping interventions are unknown.","Legal basis, transparency duties, sensitive-data conditions, retention rules, and access authority depend on the partner's records and jurisdiction.","Protected-signal recall, over-cancellation, reconstruction fidelity, reviewer agreement, fallback performance, and model-gaming resistance require field testing.","All four cost bands lack direct labor-rate, vendor-quote, data-volume, integration, and audit-frequency validation."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The eight-source bounded search found adjacent frameworks—performative foresight, performative prediction, contribution analysis, multi-touch attribution, and standardized provenance—but no exact implementation of a pre-action, reconstructible strategic-echo residual filter with independent raw audits and protected-signal fallback. This does not measure world novelty, patentability, freedom to operate, market size, or realized impact; patent and proprietary deployments were not searched comprehensively.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure a named foresight partner with authority and lawful access to one completed action ledger, the corresponding source corpus, briefs, and reviewer time records.","Pre-register the action boundary, fixed observation window, comparator workflows, attribution codebook, protected-signal classes, adjudication protocol, audit sample, acceptance thresholds, and halt rules.","Measure whether the problem occurs by estimating baseline false-independent classification and its effect on scenario interpretation or evidence weighting.","Demonstrate non-inferior complete reconstruction and independent/protected-signal recall, with no protected-signal cancellation and acceptable inter-rater reproducibility.","Show incremental improvement over simple provenance/timing tags and contribution analysis on false independence, false cancellation, reviewer time, and total resource cost.","Validate model behavior under incomplete manifests, overlapping actions, coincident external events, strategic manifest framing, version mismatch, and forced full-source fallback.","Obtain partner-specific privacy, records-management, security, reviewer-independence, and decision-authority approval.","Replace resource-equivalent cost estimates with measured labor, integration, audit, fallback, and recurring-maintenance data."],"reason":"Open sources establish the general performativity mechanism, credible horizon-scanning adopters, and implementable neighboring methods, but they cannot determine whether the exact contamination occurs in a candidate organization's proprietary scan records or whether the filter improves attribution safely and economically. Those claims require fieldwork, internal data, blinded human adjudication, and live workflow measurement, so further bounded web research cannot resolve the decisive uncertainty."},"proposal_index":3}