{"schema_version":1,"research_id":"eoa_inverse_innovation_exp06_external_evaluation_20260803","source_assessment_id":"bounded_rivalry_governance__criminology_forensic:P4:v0","cell_id":"bounded_rivalry_governance__criminology_forensic","search_queries":["site:ojp.gov police performance metrics crime reporting gaming CompStat official report","police recorded crime performance targets under recording official report","crime harm index policing funding performance primary research","police district performance community survey cross boundary displacement audit official","Chicago civilian oversight commission budget authority police district public safety official community commission ordinance","city police district performance funding prevention grants competitive official","DOJ police performance measures community trust data integrity official guidance","police performance framework public confidence recorded crime official 2026","systematic review police interventions crime displacement geographic primary research hot spots policing","Campbell systematic review hot spots policing displacement crime primary","crime reporting survey police administrative records underreporting official bureau justice statistics","police performance measurement unintended consequences gaming crime statistics primary study","site:oig.dc.gov 2026 MPD crime statistics systemic breakdown internal controls report","site:oig.dc.gov Metropolitan Police Department crime data integrity audit 2026","metrics management policing data manipulation reclassify serious crime study DOI CompStat 2021","\"The Canadian Police Performance Metrics Framework\" PDF","site:statcan.gc.ca \"Police Performance Metrics Framework\" indicators trust confidence","site:ojp.gov \"Implementing an Agency-Level Performance Measurement System\" PDF police"],"sources":[{"source_id":"S1","title":"Metrics Management and Bureaucratic Accountability: Evidence from Policing","publisher":"American Journal of Political Science; author manuscript hosted by the North Carolina Department of Justice","url":"https://ncdoj.gov/wp-content/uploads/2023/12/b323fb_bb54c8389f0144f98562896e2ddf8bd2.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"2021-10-04","accessed_at":"2026-08-03","claims_supported":["A national city-level analysis associates CompStat adoption with at least 3,500 additional minor arrests per city-year, increased classification of reported rapes as unfounded, and no reduction in serious crime.","Police metrics management creates incentives to prioritize rapidly measurable activity and manipulate administrative data.","CompStat is an established geographically decentralized performance-management practice, making it a close but narrower analogue."]},{"source_id":"S2","title":"How effectively do the police record crime? PEEL spotlight report","publisher":"His Majesty's Inspectorate of Constabulary and Fire & Rescue Services","url":"https://s3-eu-west-2.amazonaws.com/assets-hmicfrs.justiceinspectorates.gov.uk/uploads/how-effectively-do-police-record-crime-peel-spotlight-report.pdf","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2025-08","accessed_at":"2026-08-03","claims_supported":["Audits estimated that forces recorded 94.8% of crime in 2023–25, with force-level rates ranging from 86.4% to 98.1%.","After accounting for over-recording, approximately 285,000 crimes that should have been recorded were estimated to be missing in the year ending March 2025.","The inspectorate began crime-data-integrity inspections because of concern about performance targets affecting crime recording and recommends training, oversight, scrutiny, and continuing audits."]},{"source_id":"S3","title":"Crime","publisher":"U.S. Bureau of Justice Statistics","url":"https://bjs.ojp.gov/topics/crime","source_class":"OFFICIAL_ORGANIZATION_DATA","publication_date":"n.d.","accessed_at":"2026-08-03","claims_supported":["Police administrative data cover crimes reported to and recorded by police, whereas victim surveys also capture unreported crime.","Survey and administrative measures complement one another and can support triangulation, although national victim surveys do not automatically provide reliable police-district estimates."]},{"source_id":"S4","title":"The Canadian Police Performance Metrics Framework: Standardized indicators for police services in Canada","publisher":"Statistics Canada, Government of Canada","url":"https://publications.aws.tpsgc-pwgsc.cloud-nuage.canada.ca/site/eng/9.942989/publication.html","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2019-09-11","accessed_at":"2026-08-03","claims_supported":["Statistics Canada developed a standardized police-performance framework through literature review, police consultation, and examination of public performance reports.","The work examined the feasibility of uniform calls-for-service data and demonstrates that broader, standardized police-performance measurement is established adjacent practice.","The framework is a close analogue for combining crime, activity, resources, and public trust measures, but it is not a competitive continuation-funding league."]},{"source_id":"S5","title":"Implementing an Agency-Level Performance Measurement System: A Guide for Law Enforcement Executives","publisher":"Police Executive Research Forum; federally funded final report archived by the U.S. Office of Justice Programs","url":"https://www.ojp.gov/pdffiles1/nij/grants/214439.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"2006-04-01","accessed_at":"2026-08-03","claims_supported":["A tested police performance-measurement model combines community safety, perceptions of safety, trust and satisfaction, survey and nonsurvey measures, and accountability structures.","The Prince William County case demonstrates organizational implementation of a multi-measure system and reports that some recurring work was absorbed by existing staff.","The guide recommends triangulating measures, starting small, phasing implementation, obtaining senior and budget-staff support, involving communities, and formalizing governance.","Implementation is described as lengthy and iterative, supporting feasibility in principle but not the candidate's effectiveness."]},{"source_id":"S6","title":"Chicago Municipal Code §2-80-050: Community Commission for Public Safety and Accountability—Powers and duties","publisher":"City of Chicago code, hosted by American Legal Publishing","url":"https://codelibrary.amlegal.com/codes/chicago/latest/chicago_il/0-0-0-2693695","source_class":"GOVERNMENT_OR_REGULATOR","publication_date":"2024-12-16","accessed_at":"2026-08-03","claims_supported":["An identifiable municipal commission may access data, recommend Inspector General research or audits, review police budgets, recommend more effective use of public-safety resources, and recommend preventive, community-based, evidence-based violence solutions.","The commission may engage experts subject to legal approval and participate in policy development.","The commission recommends budget changes but does not itself possess unilateral authority to appropriate funds, establishing both a credible adopter and an authority limitation."]},{"source_id":"S7","title":"Towards an Index for Harm-Focused Policing","publisher":"Policing: A Journal of Policy and Practice, Oxford University Press","url":"https://academic.oup.com/policing/article-pdf/9/2/164/4255311/pau032.pdf","source_class":"PRIMARY_RESEARCH","publication_date":"2014-10-06","accessed_at":"2026-08-03","claims_supported":["Simple crime-count comparisons inadequately represent the multidimensional effects of policing.","A sentencing-based harm index was demonstrated using Philadelphia police-district data.","The paper explicitly considers police stops as possible community harms, closely anticipating the proposal's inclusion of enforcement burden but not its rivalry, audit, spillover-reserve, or portfolio-award design."]},{"source_id":"S8","title":"Police Performance Framework","publisher":"UK Home Office","url":"https://www.gov.uk/government/publications/police-performance-framework","source_class":"OFFICIAL_GUIDANCE","publication_date":"2026-01-26","accessed_at":"2026-08-03","claims_supported":["The Home Office expressly seeks performance analysis across the breadth of crime and policing activity rather than a single recorded-crime measure.","The framework is intended to improve accountability, crime reduction, public protection, trust, local and national improvement, and identification of promising practice.","This provides current official stakeholder pull for broader police-performance measurement, but not for competitive district funding."]}],"problem_evidence":{"support":"STRONG","rationale":"The exact local premise—that a particular city currently awards scarce prevention resources according to district recorded-crime standings—was not verified. The underlying problem is nevertheless visible and consequential: primary research links police metrics management to data manipulation and additional minor arrests without serious-crime reduction, regulatory audits find material variation and missing crime records, and official statistics distinguish recorded crime from victim-reported crime. Evidence for cross-boundary exporting as a frequent consequence is weaker; spatial policing research found in the search often reports diffusion rather than inevitable displacement.","source_ids":["S1","S2","S3","S7"]},"stakeholder_evidence":{"support":"MODERATE","rationale":"Current official frameworks express demand for broad, trustworthy, locally useful performance measurement, and Chicago supplies an identifiable civilian commission empowered to recommend audits, budget changes, preventive solutions, and expert analysis. No searched source expresses demand for this specific league, prize reserve, or district competition, and the commission cannot appropriate funds unilaterally.","source_ids":["S4","S5","S6","S8"]},"prior_art":{"proximity":"ADJACENT_PRIOR_ART","closest_analogues":[{"name":"CompStat and geographically decentralized metrics management","similarity":"Compares geographic police units using recurring quantitative performance data and managerial accountability.","remaining_difference":"The candidate adds preregistered prevention plans, independent reporting-integrity audits, survey measures, enforcement-burden penalties, boundary attribution, capped contest resources, provisional awards, and a protected noncompetitive service floor.","source_ids":["S1"]},{"name":"PERF agency-level performance-measurement system","similarity":"Uses multiple survey and administrative measures, community input, outcome expectations, quality assurance, and accountability structures.","remaining_difference":"It is an agency-management system rather than a scarce-prize contest and has no cross-boundary accounting, externality reserve, portfolio continuation award, or challenger cycle.","source_ids":["S5"]},{"name":"Canadian Police Performance Metrics Framework","similarity":"Standardizes a multidimensional view of police performance extending beyond crime rates to resources, activities, victimization, and public trust.","remaining_difference":"It is a reporting framework, not an audited district-community rivalry tied to marginal funding or delayed liability.","source_ids":["S4"]},{"name":"Crime-harm indexing and the UK Police Performance Framework","similarity":"Both replace raw crime counts with broader harm or performance concepts and recognize police activity, trust, and protection outcomes.","remaining_difference":"Neither source establishes the candidate's complete governance bundle or demonstrates that competitive continuation funding improves net regional harm.","source_ids":["S7","S8"]}],"distinctive_claim_remaining":"For municipalities already making district comparisons relevant to marginal prevention resources, a preregistered, independently audited shadow score that jointly accounts for verified serious harm, reporting integrity, resident reports, enforcement burdens, resource use, and adjacent-area outcomes will align the ordering of district-community prevention plans more closely with an independent net-harm criterion than either recorded-crime standings or a simple needs-based allocation formula, without changing core services or individual enforcement decisions.","confidence":"HIGH"},"implementation_evidence":{"support":"MODERATE","rationale":"Multi-measure police performance systems, administrative/survey triangulation, district harm indices, community participation, audits, and phased implementation are technically and organizationally precedented. Major candidate-specific gaps remain: sufficiently precise district-level resident measures may not exist; boundary attribution, resource-accounting comparability, delayed spillovers, and composite-weight validity are unresolved; a ranking could remain unstable or gameable; and Chicago's commission illustrates only partial authority. A retrospective, de-identified, nonpublic shadow analysis is safer and likely feasible with departmental, budget, privacy, labor, and legal cooperation, but live awards, withheld reserves, personnel consequences, or operational use require separate authority and due-process review.","source_ids":["S2","S3","S4","S5","S6","S7","S8"]},"scores":{"meaningful_impact":{"score":4,"rationale":"If successful, the design could reduce incentives for report suppression, reclassification, and low-value enforcement while improving allocation of marginal prevention support; those are consequential public-safety and rights outcomes.","source_ids":["S1","S2","S7"]},"stakeholder_pull":{"score":3,"rationale":"Governments and municipal oversight bodies visibly seek broader, trustworthy performance measurement, but no direct demand for the proposed funding league was found.","source_ids":["S6","S8"]},"incremental_advantage":{"score":3,"rationale":"The candidate integrates audit, reporting access, enforcement burden, geographic spillovers, and delayed liability into an existing rivalry; each component has analogues, while superiority over simpler multidimensional dashboards remains untested.","source_ids":["S4","S5","S7"]},"distinctiveness_plausibility":{"score":3,"rationale":"The contrastive bundle is plausibly distinct from CompStat, harm indices, and broad performance frameworks, but world novelty and exhaustive prior-art coverage were not measured.","source_ids":["S1","S4","S5","S7","S8"]},"technical_implementability":{"score":3,"rationale":"Existing systems establish feasibility of composite performance measurement and phased quality assurance, but district survey precision, boundary attribution, missing-data handling, reproducibility, and gaming resistance require proprietary-data testing.","source_ids":["S3","S4","S5","S7"]},"adoption_authority_feasibility":{"score":2,"rationale":"A credible commission can request audits, access information, recommend budgets and prevention policies, and hire experts, but appropriation, departmental data use, labor effects, privacy, and operational implementation require additional municipal actors and approvals.","source_ids":["S6"]},"evidence_readiness":{"score":2,"rationale":"The general problem and component practices are documented, but the candidate's incremental causal claim has no direct evaluation and cannot be resolved from open-web evidence alone.","source_ids":["S1","S2","S4","S5","S7"]},"safety_net_benefit":{"score":4,"rationale":"A nonpublic retrospective shadow phase, protected core-service floor, no individual enforcement use, independent audit, and predeclared stop conditions materially limit initial risk, though confidentiality and retaliatory misuse still require controls.","source_ids":["S5","S6"]},"scalability":{"score":2,"rationale":"Standardized measures can scale, but independent surveys, classification audits, cross-boundary models, appeals, and reserve administration create substantial recurring costs and coordination demands across districts.","source_ids":["S2","S4","S5"]}},"score_confidence":"MODERATE","costs":{"first_evidence":{"band_2026_usd":"50K_TO_250K","scope":"One retrospective, nonpublic shadow analysis for one completed period: metric preregistration, legal/privacy review, de-identified extracts, limited independent record audit, two comparators, sensitivity analysis, and a written stop/go report.","confidence":"LOW","assumptions":["Existing administrative extracts are usable without building a new records system.","A small independent team can audit a stratified sample rather than all incidents.","This excludes awards, operational changes, and a new district-representative resident survey."],"source_ids":["S3","S5"]},"initial_deployment_startup":{"band_2026_usd":"250K_TO_1M","scope":"Design and governance for a multi-district pilot: data agreements, survey design, boundary rules, secure analytic environment, rulebook, scorer training, conflict controls, appeals process, resource-accounting protocol, and community consultation.","confidence":"LOW","assumptions":["City systems can export standardized data.","Existing oversight, budget, and legal staff contribute material in-kind time.","No major software procurement or retrospective record remediation is required."],"source_ids":["S4","S5","S6"]},"operational_launch":{"band_2026_usd":"1M_TO_5M","scope":"One citywide or large multi-district operational year with independent surveys, record audits, secure scoring, community-partnership support, appeals, spillover review, and program administration; excludes the pre-existing marginal prevention prize pool and base public-safety budgets.","confidence":"LOW","assumptions":["Approximately 10–25 participating districts or equivalent units.","Independent survey and audit vendors are required.","The award reserve is withheld from already-budgeted continuation funds rather than newly appropriated program cost."],"source_ids":["S2","S4","S5"]},"annual_recurring":{"band_2026_usd":"1M_TO_5M","scope":"Annual surveys, data-quality and classification audits, scoring and sensitivity analysis, appeals, community engagement, delayed-spillover review, secure data operations, and post-contest evaluation; excludes base services and continuation awards.","confidence":"LOW","assumptions":["The city maintains recurring district-level survey precision and independent audit capacity.","Rules and systems require annual recalibration but not complete redevelopment.","Costs vary materially with district count, sample sizes, data quality, and union or legal obligations."],"source_ids":["S2","S4","S5"]}},"verified_pipeline_gates":{"externally_supported_problem":{"status":"YES","reason":"Research and official audits establish metric-induced manipulation risks, enforcement distortions, recording variation, and the inadequacy of treating recorded crime as the complete crime measure.","source_ids":["S1","S2","S3"]},"externally_credible_adopter_or_authorizer":{"status":"YES","reason":"Chicago's civilian public-safety commission is an identifiable potential adopter with audit, data-access, policy, expert-analysis, and budget-recommendation powers; City Council and executive agencies would still be needed for appropriation and implementation.","source_ids":["S6"]},"distinct_testable_incremental_claim":{"status":"YES","reason":"The candidate can be tested against recorded-crime and needs-based comparators using preregistered external validation, ranking-stability, reporting-integrity, enforcement-burden, and spillover criteria.","source_ids":["S1","S4","S5","S7"]},"bounded_next_evidence_step":{"status":"YES","reason":"A one-period, nonpublic shadow analysis with frozen data, predefined comparators, independent audit sampling, sensitivity analysis, and explicit stopping thresholds is bounded and consistent with official guidance to start small and phase implementation.","source_ids":["S5"]},"no_unresolved_safety_or_authority_stop":{"status":"UNCERTAIN","reason":"Shadow use is substantially bounded, but local approvals for sensitive records, survey linkage, geographic disclosure, labor implications, scorer access, and protection against retaliatory or operational use have not been verified. The cited commission cannot unilaterally appropriate funds.","source_ids":["S3","S6"]},"credible_cost_scope_and_range":{"status":"UNCERTAIN","reason":"Resource scopes are identifiable and prior implementation guidance describes staffing and phasing, but no jurisdiction, district count, data condition, audit sample, survey precision, or prize amount was supplied; therefore the bands are planning estimates rather than externally priced budgets.","source_ids":["S4","S5"]}},"next_evidence_step":"With one municipality and independent civilian/data-audit partners, preregister a retrospective shadow study using one completed 12-month period and no awards, publication, personnel use, or operational change. Freeze data and compare (A) the existing recorded-crime/managerial ordering, (B) a transparent population-and-need formula, and (C) the proposed net-harm composite. Validate all three against a separately constructed external criterion using independently audited serious-harm samples, report-access checks, resident survey measures, enforcement-contact burden, adjacent-area outcomes, and implementation fidelity. Double-score a blinded sample and run disclosed weight, missing-data, and boundary sensitivity analyses. Falsify further development if scorer agreement is below 0.90; more than 25% of top-half membership changes under modest preregistered assumptions; the proposed ordering does not outperform both comparators on the external criterion; district differences in auditable missingness or classification error exceed 5 percentage points without correction; person-level data cannot be governed safely; or any retaliation, record alteration, leaked ranking, or operational use occurs.","blocking_evidence":["No verified municipality was found where recorded-crime district standings currently determine a scarce marginal prevention prize, so local problem prevalence is unconfirmed.","No direct evidence shows that the proposed composite produces more valid or stable district orderings than recorded-crime or needs-based comparators.","Local district-level survey precision, classification-audit access, cross-boundary attribution, resource-accounting comparability, and delayed-spillover observability require proprietary data.","No live authority analysis confirms appropriation, contracting, records access, labor, privacy, appeal, reserve-withholding, and remediation powers for a specific jurisdiction.","Behavioral responses to the league—including new gaming, reduced cooperation, coercive substitution, or reporting suppression—require prospective observation or a live pilot.","No externally grounded local cost estimate exists for the required survey, secure data environment, independent audit capacity, appeals, or spillover review."],"research_disposition":"PARTNERED_RESEARCH_PROGRAM","world_novelty_boundary":"The search assessed visible open-web analogues in police metrics management, crime-data integrity, multidimensional performance frameworks, harm indices, and municipal oversight authority. It did not measure world novelty, patentability, freedom to operate, market size, or realized impact, and it did not exhaust municipal rules, unpublished programs, procurement records, proprietary systems, patents, or non-English prior art.","arm":"COMPLETE_PROPOSAL_PORTFOLIO","candidate_version":0,"controller_recommendation":{"action":"STOP_EMPIRICAL_RESEARCH_NEEDED","repairable":false,"material_progress_observed":true,"progress_targets":["Secure a municipality where district comparisons demonstrably influence a bounded marginal prevention resource.","Execute written data, privacy, audit-access, labor, legal-authority, and nonretaliation agreements for a shadow study.","Pre-register comparators, external validation criteria, weights, boundary rules, missing-data treatment, audit samples, and falsification thresholds.","Demonstrate reproducible scoring, acceptable district-level measurement precision, ranking stability, and superior external-criterion validity over both comparators.","Obtain independent confirmation that the shadow process creates no re-identification, retaliation, reporting suppression, record alteration, cooperation loss, or operational use.","Produce a jurisdiction-specific 2026-USD budget based on district count, survey samples, data remediation, audit workload, appeals, and secure-system requirements."],"reason":"Bounded web research substantially supports the general problem, identifies plausible adopters, and maps close prior art, but the remaining incremental claim depends on proprietary municipal data, local authority determinations, and live or shadow behavioral evidence. Those questions cannot be resolved by additional bounded web search."},"proposal_index":4}