{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"bounded_rivalry_governance__criminology_forensic:P3:v0","cell_id":"bounded_rivalry_governance__criminology_forensic","search_queries":["site:nij.ojp.gov cold case investigation best practices guide multidisciplinary review","site:college.police.uk major crime investigation review cold case review hypotheses","criminal investigation tunnel vision confirmation bias competing hypotheses empirical study investigators","Analysis of Competing Hypotheses CIA official","site:college.police.uk APP review major investigation independent review hypotheses investigator mindset","site:library.college.police.uk \"Major Crime Investigation Manual\" 2021 review cold case","site:gov.uk criminal investigation reasonable lines of inquiry disclosure prosecution defense official guidance","linear sequential unmasking expanded forensic science bias management paper 2020","site:namus.nij.ojp.gov \"Cold Case Advisory\" multidisciplinary review process","site:nij.ojp.gov cold case review multidisciplinary team limited resources prioritization cases","site:college.police.uk \"cold case\" review major crime investigation manual","site:college.police.uk \"Investigative mindset\" hypotheses","\"Linear Sequential Unmasking–Expanded\" 2021 journal full text","doi Linear Sequential Unmasking Expanded Dror Kukucka Kassin Zapf 2021"],"sources":[{"source_id":"S1","title":"National Best Practices for Implementing and Sustaining a Cold Case Investigation Unit","publisher":"National Institute of Justice","url":"https://nij.ojp.gov/library/publications/national-best-practices-implementing-and-sustaining-cold-case-investigation","source_class":"OFFICIAL_GUIDANCE","publication_date":"2019-07-01","accessed_at":"2026-08-03","claims_supported":["Law-enforcement agencies are identifiable adopters of formal cold-case review mechanisms.","Cold-case work requires dedicated, bounded staffing and multidisciplinary expertise.","The guide recommends at least two full-time investigators, no more than five active cases per investigator, performance metrics beyond clearance rates, and multidisciplinary input.","The guide is recommended best practice, not a binding mandate."]},{"source_id":"S2","title":"NamUs Cold Case Advisory Process","publisher":"National Missing and Unidentified Persons System (NamUs), National Institute of Justice","url":"https://namus.nij.ojp.gov/media/image/1341","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"not stated on page","accessed_at":"2026-08-03","claims_supported":["An investigating agency can request a multidisciplinary cold-case review from an existing federal service.","The process compiles case materials through secure transfer and uses law-enforcement, medicolegal, DNA, and analytical specialists.","The review produces recommended forensic testing, investigative leads, and potential resources, followed by feedback and progress tracking.","This is an operational analogue for multidisciplinary review but not for rival hypothesis teams or competitive scoring."]},{"source_id":"S3","title":"Conducting Effective Investigations: Practice Evidence","publisher":"College of Policing","url":"https://assets.college.police.uk/s3fs-public/2023-08/Conducting-effective-investigations-practice-evidence-report.pdf?v=1693209421","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"2023-08","accessed_at":"2026-08-03","claims_supported":["The College engaged more than 800 officers and staff from 37 forces and wider law-enforcement agencies.","Practitioners expressed inconsistent and incomplete understandings of the investigative mindset.","Fewer than 10 percent of respondents explicitly mentioned impartiality, objectivity, or avoiding bias, and fewer than 20 percent mentioned gathering all available material.","Practitioners identified time, workload, supervision, training, and information access as barriers or supports relevant to implementation."]},{"source_id":"S4","title":"Tunnel Vision and Confirmation Bias Among Police Investigators and Laypeople in Hypothetical Criminal Contexts","publisher":"SAGE Open","url":"https://journals.sagepub.com/doi/10.1177/21582440221095022","source_class":"PRIMARY_RESEARCH","publication_date":"2022","accessed_at":"2026-08-03","claims_supported":["In a hypothetical-scenario experiment involving 40 Israeli police investigators and 40 laypeople, investigators expressed greater confidence in suspects' guilt than laypeople.","Investigators nevertheless responded to exonerating information, so the evidence does not support an invariant or absolute tunnel-vision effect.","The small, volunteer, jurisdiction-specific, hypothetical sample limits prevalence and operational generalization."]},{"source_id":"S5","title":"Psychology of Intelligence Analysis","publisher":"Central Intelligence Agency, Center for the Study of Intelligence","url":"https://www.cia.gov/resources/csi/static/Pyschology-of-Intelligence-Analysis.pdf","source_class":"AUTHORITATIVE_SECONDARY","publication_date":"1999","accessed_at":"2026-08-03","claims_supported":["Analysis of Competing Hypotheses already requires explicit reasonable alternatives, evidence for and against each, diagnosticity assessment, and identification of evidence that would change the judgment.","ACH creates an audit trail and emphasizes refutation rather than collecting support for a favorite explanation.","The author explicitly states that ACH does not guarantee a correct answer because judgments remain fallible and evidence incomplete.","ACH is major adjacent prior art to the proposed common record, competing hypotheses, contradictions, falsifiers, and post-result review."]},{"source_id":"S6","title":"Effects of Task Structure and Confirmation Bias in Alternative Hypotheses Evaluation","publisher":"Cognitive Research: Principles and Implications","url":"https://pubmed.ncbi.nlm.nih.gov/38866984/","source_class":"PRIMARY_RESEARCH","publication_date":"2024-06-13","accessed_at":"2026-08-03","claims_supported":["In Study 1 with 161 participants, hypotheses-in-rows presentation reduced confirmation bias, but the conventional ACH matrix and paragraph formats did not.","The ACH-style matrix did not improve sensitivity to evidence credibility.","In Study 2, most of 62 Dutch military analysts did not exhibit confirmation bias and were sensitive to credibility.","Benefits can depend on interface and task structure, creating direct counterevidence to assuming that formal competing-hypothesis structure is sufficient."]},{"source_id":"S7","title":"OSAC 2022-S-0030 Standard Methodology in Bloodstain Pattern Analysis, Version 2.1","publisher":"National Institute of Standards and Technology, Organization of Scientific Area Committees for Forensic Science","url":"https://www.nist.gov/system/files/documents/2024/09/17/OSAC%202022-S-0030%20Standard%20Methodology%20in%20Bloodstain%20Pattern%20Analysis%20Version%202.1.pdf","source_class":"STANDARD","publication_date":"2023-12-29","accessed_at":"2026-08-03","claims_supported":["A forensic proposed standard incorporates ordered information exposure, documentation of departures, defined source materials, and control of task-irrelevant information.","It requires hypothesis testing against independent data that can differentiate alternatives and documentation of support for and against proposed possibilities.","It demonstrates technical feasibility of read-only records, provenance, information-order controls, explicit alternatives, and auditable reasoning in a bounded forensic domain.","It is a proposed, discipline-specific standard and does not validate parallel investigative teams or competitive resource allocation."]},{"source_id":"S8","title":"Disclosure Manual: Chapter 5 — Reasonable Lines of Enquiry and Third Parties","publisher":"Crown Prosecution Service","url":"https://www.cps.gov.uk/prosecution-guidance/disclosure-manual-chapter-5-reasonable-lines-enquiry-and-third-parties","source_class":"OFFICIAL_GUIDANCE","publication_date":"2026-01-14","accessed_at":"2026-08-03","claims_supported":["Investigators must pursue reasonable lines of enquiry pointing toward or away from a suspect.","Enquiries involving private or third-party material must be necessary, proportionate, focused, justified, and documented.","Speculative searches lack a reasonable foundation, and privacy, fair-trial rights, less-intrusive alternatives, and disclosure obligations must be balanced.","A hypothesis-ranking panel cannot replace investigators, prosecutors, courts, or other legally designated decision-makers."]}],"problem_evidence":{"support":"MODERATE","rationale":"Cold-case resource constraints, the need to prioritize review and testing, uneven investigative-mindset practice, and susceptibility to tunnel vision are externally visible. However, no relied source measures how often incumbent teams control records or win resource disputes through evidence hoarding, presentation order, or attacks on alternatives. The strongest investigator experiment is small and hypothetical and also shows responsiveness to exonerating evidence.","source_ids":["S1","S3","S4"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Law-enforcement agencies, cold-case unit leaders, and NamUs are identifiable adopters or partners; official guidance and the operational NamUs process express demand for dedicated resources, multidisciplinary review, forensic-test recommendations, and performance monitoring. No source expresses demand for separated rival teams, a confidential leaderboard, equal hypothesis budgets, or a competitive validation gate specifically.","source_ids":["S1","S2","S3"]},"prior_art":{"proximity":"ADJACENT_PRIOR_ART","closest_analogues":[{"name":"Analysis of Competing Hypotheses","similarity":"Explicit alternatives are compared against common evidence, contradictions and diagnostic evidence are emphasized, change indicators are identified, and the analysis leaves an audit trail.","remaining_difference":"ACH ordinarily structures one collaborative analytic process; it does not establish independently staffed rival teams, equal resource caps, rights-adjusted scoring, limited validation prizes, appeals, or challenger windows.","source_ids":["S5","S6"]},{"name":"NamUs multidisciplinary cold-case review","similarity":"An agency submits compiled records for secure multidisciplinary analysis that yields testing recommendations, leads, resources, and follow-up.","remaining_difference":"The multidisciplinary team synthesizes one review rather than conducting a governed contest among separately advocated hypotheses.","source_ids":["S2"]},{"name":"NIJ cold-case unit best practices","similarity":"Dedicated capacity, bounded caseloads, multidisciplinary expertise, performance metrics, and formalized partnerships address scarce cold-case resources.","remaining_difference":"The guide prioritizes cases and organizes units, not rival hypotheses within one case through preregistration and scored validation slots.","source_ids":["S1"]},{"name":"OSAC ordered forensic interpretation with alternative hypotheses","similarity":"The proposed standard defines source materials, controls information order, documents assumptions, compares predictions with data, and requires consideration of alternatives.","remaining_difference":"It governs a single forensic discipline and analyst workflow, not major-case hypothesis teams or allocation of investigative resources.","source_ids":["S7"]}],"distinctive_claim_remaining":"Relative to a conventional case conference and ordinary ACH, a separated-team gate with identical cutoff records, prospectively registered discriminating predictions, equal analysis budgets, rights-adjusted scoring, and at most two validation slots will improve held-out prediction calibration and the information value of proposed next steps without increasing unsupported allegations, privacy burden, evidence consumption, hypothesis entrenchment, or panel inconsistency.","confidence":"MODERATE"},"implementation_evidence":{"support":"MODERATE","rationale":"Secure compilation of case records, multidisciplinary review, auditable source lists, information-order controls, alternative-hypothesis documentation, and rights-proportionate enquiry review all exist separately. A retrospective masked simulation is technically and legally plausible. No source demonstrates reliable scoring of investigative hypotheses, adequate panel agreement, safe separation of teams, prevention of strategic withholding, or superiority to collaborative review in real cold-case workflows.","source_ids":["S2","S5","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":4,"rationale":"If effective, better allocation of scarce analysis and testing could reduce premature closure, unsupported intrusion, duplicated work, and missed exculpatory evidence in consequential cases. Realized impact is unmeasured.","source_ids":["S1","S4","S8"]},"stakeholder_pull":{"score":3,"rationale":"Official bodies show clear pull for cold-case resources, multidisciplinary review, open-minded investigation, and testing recommendations, but not for competitive hypothesis governance itself.","source_ids":["S1","S2","S3"]},"incremental_advantage":{"score":2,"rationale":"The proposal adds governance and resource-allocation features to ACH and multidisciplinary review, but there is no evidence that adversarial separation and scoring outperform collaborative alternatives; task formatting can nullify expected debiasing benefits.","source_ids":["S5","S6"]},"distinctiveness_plausibility":{"score":3,"rationale":"No close match to the complete combination was found, although its analytic core substantially overlaps ACH, multidisciplinary review, and ordered forensic interpretation. World novelty remains unmeasured.","source_ids":["S1","S2","S5","S7"]},"technical_implementability":{"score":4,"rationale":"A no-case-impact simulation using copied records, controlled access, preregistration, logs, and held-out evidence is technically straightforward. Secure masking, record completeness, and leakage control require disciplined implementation.","source_ids":["S2","S7"]},"adoption_authority_feasibility":{"score":3,"rationale":"A review-unit authority can allocate internal analyst time and convene a simulation, but prosecutors, laboratories, disclosure officials, courts, and investigative authorities retain nondelegable powers. Jurisdiction-specific approvals remain necessary before operational use.","source_ids":["S1","S2","S8"]},"evidence_readiness":{"score":4,"rationale":"The claim is measurable retrospectively with masked time-split records, explicit comparators, held-out outcomes, and prespecified harms. Access to suitable records and qualified participants is proprietary and requires partnership.","source_ids":["S2","S5","S6","S7"]},"safety_net_benefit":{"score":4,"rationale":"The design could preserve alternative explanations, exculpatory material, privacy proportionality, and noncontestable legal review. These benefits depend on preventing rankings from being treated as guilt findings.","source_ids":["S4","S8"]},"scalability":{"score":3,"rationale":"Templates, audit logs, and standardized packets are reusable, but parallel staffing, secure duplication, independent panels, appeals, and record masking increase cost and limit throughput in resource-constrained units.","source_ids":["S1","S2","S3"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"Design and run a preregistered three-condition retrospective study on approximately 8–12 masked, time-split case packets, including packet preparation, participant backfill or compensation, secure access, independent scoring, statistical analysis, and safeguard review.","confidence":"LOW","assumptions":["No original evidence, live investigation, witness contact, or destructive test is used.","Approximately 60–120 analyst sessions are required across bounded-gate, conventional-conference, and ACH conditions.","The partner already has a secure environment and lawful authority to prepare deidentified records.","This is a resource-equivalent estimate, not a vendor quotation or bottom-up local budget."],"source_ids":["S1","S2","S3"]},"initial_deployment_startup":{"band_2026_usd":"250K_TO_1M","scope":"Create policy, eligibility and scoring instruments, a citation-addressable read-only evidence environment, role-based access and audit logging, training, legal/privacy/disclosure review, panel conflict procedures, and an independent evaluation protocol for one major-case unit.","confidence":"LOW","assumptions":["Existing case-management and secure-storage infrastructure can be extended rather than replaced.","The deployment remains non-operational until retrospective evidence and authority review are satisfactory.","Costs include staff time, training, security engineering, and legal review but exclude new laboratory equipment and forensic tests."],"source_ids":["S2","S7","S8"]},"operational_launch":{"band_2026_usd":"250K_TO_1M","scope":"Resource-equivalent cost for a tightly supervised first operational year or first cohort of up to five cases, including parallel-team time, an evidence custodian, panel review, quality assurance, appeals, monitoring, and independent post-stage evaluation.","confidence":"LOW","assumptions":["Operational use occurs only after empirical and legal approval.","Advancement allocates review capacity only; laboratory tests and coercive actions remain under ordinary authority and are budgeted separately.","The unit limits active caseload and uses existing investigators and specialists with backfilled time."],"source_ids":["S1","S2","S8"]},"annual_recurring":{"band_2026_usd":"1M_TO_5M","scope":"Sustain a single major-case review program with dedicated investigators or analysts, evidence administration, independent panels, secure-system operations, audits, training refreshers, appeals, and outcome evaluation.","confidence":"LOW","assumptions":["At least two full-time investigator-equivalents plus specialist, panel, administrative, security, and duplicated-team time are included.","Throughput is deliberately limited to protect record quality and review independence.","Laboratory testing, litigation, searches, and major travel are outside the estimate.","Local compensation, case complexity, and security requirements could move the total materially."],"source_ids":["S1","S2","S3"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"External official and primary sources support scarce cold-case capacity, inconsistent investigative-mindset practice, and a credible risk of tunnel vision, although the proposal's exact rivalry and evidence-hoarding mechanism has not been measured.","source_ids":["S1","S3","S4"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Investigating agencies and cold-case unit leaders are identifiable adopters, and NamUs already accepts agency requests for secure multidisciplinary cold-case reviews. This establishes credible organizational hosts, not expressed demand for the proposed contest.","source_ids":["S1","S2"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The proposal can be tested against both conventional case conference and ordinary ACH on held-out prediction calibration, discriminating test value, citation completeness, panel agreement, resource use, and rights burdens.","source_ids":["S5","S6","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"A masked, retrospective, time-split, no-case-impact comparison is bounded, reversible, preregistrable, and has explicit comparators and falsifiers.","source_ids":["S2","S5","S6","S7"]},"no_unresolved_safety_or_authority_stop":{"status":"YES","reason":"For the proposed next step only, masking, copied records, no contact or testing, suppressed rankings, and no operational consequence keep ordinary investigative, laboratory, prosecutorial, disclosure, and judicial authority intact. This gate does not authorize live deployment.","source_ids":["S2","S8"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"The four scopes are bounded and use official staffing and workflow evidence, but no source supplies a bottom-up 2026 budget for parallel hypothesis teams, secure packet preparation, or independent panels. Local salary, security, and case-complexity data are needed.","source_ids":["S1","S2","S3"]}},"next_evidence_step":"With one authorized cold-case or major-case review partner, preregister a three-condition retrospective crossover using 8–12 masked, time-split closed-case or synthetic packets: (A) the proposed bounded hypothesis gate, (B) a conventional multidisciplinary case conference, and (C) a noncompetitive ACH worksheet. Give all conditions the same cutoff record and analyst-hour budget; rotate qualified participants across conditions while preventing packet-recognition and later-outcome leakage. Before revealing later segments, capture hypotheses, citations, contradictions, calibrated probabilities, discriminating predictions, proposed next actions, expected information gain, privacy burden, evidence consumption, and third-party burden. Blind independent reviewers to condition and score held-out prediction calibration, ability to distinguish alternatives, citation completeness, unsupported allegations, proposal redundancy, panel reliability, time, and intrusion. Falsify the incremental claim if the gate does not outperform the better comparator on preregistered calibration and rights-adjusted information value, if confidence calibration worsens, if panel agreement falls below the preregistered reliability threshold, or if unsupported accusation, strategic withholding, participant entrenchment, privacy burden, or evidence-consumption proposals increase. Halt and void affected packets if masking fails, a participant recognizes a case, unequal records are detected, or later-outcome knowledge reaches analysts or scorers.","blocking_evidence":["No direct prevalence estimate shows how often cold-case hypothesis resource decisions are distorted by incumbent record control, selective presentation, evidence hoarding, or informal rivalry.","No field or retrospective experiment shows that separated rival teams and competitive scoring outperform conventional case conferences or ordinary ACH.","The reliability, construct validity, and susceptibility to gaming of the proposed rights-adjusted scoring rubric are unknown.","It is unknown whether team separation increases commitment, strategic withholding, cosmetically distinct hypotheses, or reluctance to share urgent exculpatory information.","Suitable masked time-split records, qualified participants, and lawful access are proprietary and require an agency partner.","Jurisdiction-specific authorization, disclosure, labor, privacy, records-retention, and defense-access implications require local review before operational use.","Local staffing, secure-system, panel, and case-preparation costs have not been measured.","No evidence establishes safe scalability beyond a small, retrospective, no-case-impact study."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The search assessed visible English-language official practices and research concerning cold-case units, multidisciplinary review, competing-hypothesis analysis, investigative bias, forensic information sequencing, alternative-hypothesis documentation, and rights-proportionate enquiries. It did not measure world novelty, patentability, freedom to operate, market size, realized impact, unpublished agency procedures, proprietary case-management systems, non-English practices, or every legal jurisdiction. Absence of an integrated match in these eight sources is not a novelty finding.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure an authorized agency or research partner with lawful access to closed, maskable case records and qualified investigators.","Preregister the three-condition retrospective crossover, primary outcome, harm measures, minimum panel-reliability threshold, sample-size logic, and exclusion rules.","Validate that time-split packets are complete enough to support fair comparison and quantify case-recognition and outcome-leakage risk.","Develop and test the scoring rubric for inter-panel reliability, construct validity, redundancy detection, and resistance to narrative-confidence effects.","Measure whether separation causes strategic withholding, hypothesis entrenchment, delayed sharing, unsupported allegations, or greater privacy and evidence-consumption burdens.","Obtain jurisdiction-specific written determinations covering research authority, disclosure, privacy, records retention, participant labor protections, and prohibition on operational use.","Produce a bottom-up local budget and staffing model before any deployment inquiry.","Advance only if the gate beats both conventional conference and ACH on held-out calibration and rights-adjusted information value without crossing any preregistered harm threshold."],"reason":"Bounded web research establishes a meaningful problem, credible organizational hosts, strong adjacent prior art, a distinct testable contrast, and a safe retrospective design. It cannot establish the proposal's incremental effect, panel reliability, behavioral side effects, secure workflow performance, or local authority in practice. Those questions require proprietary records, qualified participants, and live controlled testing; therefore the appropriate controller outcome is an empirical-research stop, not further bounded web research."},"proposal_index":3}