{"closest_prior_art":[{"name":"Office of Evaluation Sciences evaluation methods suite","overlap":"Combines advance commitment to analytic choices, multiple-comparison adjustment, priority outcomes, disclosure of null findings, and guidance for retrospective observational evaluations.","remaining_difference":"It does not establish a court-rulemaking-specific claim registry that inventories every attempted specification, assigns the proposal's complete status taxonomy, and requires separated unused evidence before a selected finding becomes confirmatory.","source_ids":["SRC2"]},{"name":"Australian Centre for Evaluation preregistration and pre-analysis-plan guidance","overlap":"Addresses cherry-picking and specification searching by timestamping outcomes and analysis methods before data inspection, distinguishing exploratory from confirmatory analysis, and encouraging reporting of null results.","remaining_difference":"The guidance targets randomized trials and does not require an access-controlled ledger of all materially considered analyses, family-specific FWER/FDR treatment, or independent holdout confirmation tied to judicial rule adoption.","source_ids":["SRC4"]},{"name":"UK Government Evaluation Registry","overlap":"Requires registration of planned, live, and completed government evaluations; early upload of evaluation plans; recording of later changes; publication of findings by default; and handling of disclosure restrictions.","remaining_difference":"Registration occurs primarily at the evaluation level rather than as a complete claim-and-specification ledger, and it does not impose multiplicity-calibrated claim statuses or separated confirmation evidence.","source_ids":["SRC3"]},{"name":"HMCTS remote-hearing evaluation","overlap":"Demonstrates a real court-system evaluation spanning numerous jurisdictions, user groups, hearing stages, experiences, outcomes, and subgroup comparisons relevant to future remote-hearing policy.","remaining_difference":"The published report is an evaluation rather than an evidence-status charter; the retained source does not document a complete prospective claim family, multiplicity policy, all-analysis ledger, or reserved confirmation cohort.","source_ids":["SRC1"]}],"contrastive_claim_falsifier":"The contrastive claim would be falsified by an audit showing that completed remote-hearing pilot evaluations used for durable court-rule decisions already preregister complete outcome and specification families, retain every material null and alternative analysis, apply declared family-level or discovery-rate controls, label post hoc findings as exploratory, and require a separated evaluator to test selected claims on genuinely unused evidence before treating them as confirmed.","contrastive_claim_remaining":"Adjacent public-evaluation practices cover preregistration, multiple-comparison adjustment, evaluation registries, exploratory labeling, and null-result disclosure. What remains distinct and testable is their court-rulemaking-specific integration: the entire remote-hearing pilot analysis family determines each claim's status, every material analytic path is ledgered, and a selected claim cannot become confirmatory without separated testing on previously unused evidence.","experiment_id":"eoa_inverse_innovation_exp13_second_slot_policy60_20260806","gates":{"adequate_source_search":{"rationale":"Four search lanes covered the proposal directly, older terms such as cherry-picking and specification searching, government evaluation registries and guidance, court pilot practices, multiplicity, and holdout or replication combinations. Four opened direct sources from three government publishers were retained, including primary research and official guidance.","source_ids":["SRC1","SRC2","SRC3","SRC4"],"status":"PASS"},"bounded_next_test":{"rationale":"A read-only reconstruction of one completed pilot is bounded by one record set and yields measurable outputs: reconstructed-family coverage, omitted analyses, inter-reviewer classification agreement, review time, confidentiality incidents, and availability of unused evidence. It changes neither hearings nor rules.","source_ids":["SRC1","SRC2","SRC3","SRC4"],"status":"PASS"},"distinct_testable_claim":{"rationale":"Reviewers can test whether the charter improves reconstruction of the complete analysis family, produces reliable status classifications, prevents exploratory claims from being presented as confirmed, and identifies legally usable separated confirmation evidence. Existing guidance supplies individual components but not this court-rulemaking combination.","source_ids":["SRC1","SRC2","SRC3","SRC4"],"status":"PASS"},"no_obvious_safety_or_authority_stop":{"rationale":"No obvious stop applies to a retrospective paper exercise conducted under existing records and evaluation authority. The test must verify access permissions, confidentiality, sealing and disclosure controls, avoid active matters, and leave adoption decisions with the authorized judicial body; inability to meet those conditions triggers the stated halt.","source_ids":["SRC1","SRC3"],"status":"PASS"},"supported_problem":{"rationale":"The remote-hearing evaluation visibly spans many outcomes, jurisdictions, user groups and subgroup comparisons, while official evaluation guidance expressly identifies false-positive, selective-reporting, cherry-picking and specification-search risks from such analytic choice sets. The retained sources do not prove that a court rulemaking memorandum actually promoted a selected result, so support is partial rather than conclusive.","source_ids":["SRC1","SRC2","SRC4"],"status":"PASS"}},"prior_art_disposition":"ADJACENT_PRIOR_ART","problem_evidence":{"finding":"A large official remote-hearing evaluation reports numerous measures and subgroup comparisons across civil and other jurisdictions. General government-evaluation guidance recognizes that multiple outcomes, subgroups and analytic choices create false-positive and selective-reporting risks addressable through preregistration, multiplicity adjustment, exploratory labels and null-result reporting. Actual selective promotion in a permanent court-rule decision was not demonstrated by the retained sources.","source_ids":["SRC1","SRC2","SRC4"],"status":"PARTLY_SUPPORTED"},"research_id":"eoa_inverse_innovation_exp13_light_screen_20260806","schema_version":1,"screen_id":"E13P163","screen_survival":true,"search_lanes":{"component_combination":{"no_result_note":null,"queries":["policy evaluation preregistration analysis registry multiple hypothesis testing holdout replication official guidance","results registry analysis ledger specification searches selective reporting policy evaluation","site:uscourts.gov pilot evaluation court procedure research protocol analysis plan"],"source_ids":["SRC2","SRC3","SRC4"]},"direct_problem_and_intervention":{"no_result_note":null,"queries":["remote court hearing pilot evaluation report outcomes rulemaking civil hearings evaluation methodology","judicial pilot program evaluation preregistration multiple outcomes court procedure evidence","remote hearings evaluation report civil courts access fairness pilot"],"source_ids":["SRC1","SRC2"]},"products_practices_and_standards":{"no_result_note":null,"queries":["court program evaluation standards multiple comparisons preregistration","policy evaluation preregistration analysis registry multiple hypothesis testing holdout replication official guidance","site:ncsc.org court pilot evaluation framework remote hearings outcomes"],"source_ids":["SRC2","SRC3","SRC4"]},"synonyms_and_historical_terms":{"no_result_note":null,"queries":["results registry analysis ledger specification searches selective reporting policy evaluation","policy evaluation preregistration analysis registry multiple hypothesis testing holdout replication official guidance","court program evaluation standards multiple comparisons preregistration"],"source_ids":["SRC2","SRC3","SRC4"]}},"sources":[{"claims_supported":["Remote-hearing evaluations can span many jurisdictions, user groups, hearing stages, experiences and outcomes.","The report presents numerous subgroup comparisons and identifies access, support, fairness and procedural-justice considerations.","The source demonstrates the proposal's court-evaluation setting but does not itself establish selective reporting."],"publisher":"HM Courts & Tribunals Service / IFF Research","source_id":"SRC1","source_type":"PRIMARY_RESEARCH","title":"HMCTS Remote Hearing Evaluation: Research Report","url":"https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1040183/Evaluation_of_remote_hearings_v23.pdf"},{"claims_supported":["Testing multiple outcomes or intervention versions creates false-positive risk unless multiple comparisons are addressed.","Preregistration commits evaluators to design and analytic choices in advance and reduces result-tailoring and selective positive reporting.","Sharing unexpected and null results improves federal evaluation learning."],"publisher":"U.S. General Services Administration, Office of Evaluation Sciences","source_id":"SRC2","source_type":"OFFICIAL_GUIDANCE","title":"Evaluation Resources","url":"https://oes.gsa.gov/methods/"},{"claims_supported":["Government evaluation plans should be registered no later than the first data-collection round and updated when plans change.","Planned, live and completed evaluations and their findings are generally subject to registration and publication requirements.","Registration and publication remain subject to classification, freedom-of-information exemptions and institutional governance."],"publisher":"UK Cabinet Office and HM Treasury, Evaluation Task Force","source_id":"SRC3","source_type":"OFFICIAL_GUIDANCE","title":"Guidance on Using the Evaluation Registry","url":"https://www.gov.uk/guidance/guidance-on-using-the-evaluation-registry"},{"claims_supported":["Pre-analysis plans address cherry-picking, p-hacking and specification searching where outcomes or subgroups can be analyzed in many ways.","Exploratory analyses should be distinguished from confirmatory analyses.","Plans should be timestamped before data inspection, identify primary outcomes and methods, disclose justified deviations, and improve reporting of null results."],"publisher":"Australian Government Treasury, Australian Centre for Evaluation","source_id":"SRC4","source_type":"OFFICIAL_GUIDANCE","title":"Preregistration and Pre-analysis Plans for Randomised Controlled Trials","url":"https://evaluation.treasury.gov.au/toolkit/preregistration-and-pre-analysis-plans"}],"world_novelty_boundary":"This bounded four-source screen found adjacent public practices but no opened source implementing the full proposal for permanent remote civil-hearing rules. That result cannot establish world novelty, patentability, market size, expert acceptance, institutional feasibility, or realized value; unsearched jurisdictions, internal judicial protocols, unpublished evaluations and patent literature may contain closer matches."}