{"actors":["Judicial-council research staff who evaluate court-procedure pilots","Trial-court administrators who supply operational data","Judges and clerks who implement pilot procedures","Rulemaking committees that recommend permanent procedural rules","The judicial authority empowered to adopt or amend court rules","Litigants and counsel whose access, participation, and case processing may be affected","Independent evaluators assigned to the confirmation stage"],"affected_objective":"Calibrate the evidentiary status of findings used to decide whether a pilot procedure for remote civil hearings should become a permanent or statewide court rule.","arm":"ORDINARY_DIVERSE_P2","authority_safety":{"authorized_first_step":"Under existing evaluation and records authority, judicial-council research staff may conduct a read-only reconstruction of the analysis paths used in one completed remote-hearing pilot and apply a paper evidence-status charter without changing any proceeding or rule.","decision_authority":"Only the judicial body already empowered by constitution, statute, or court rules may adopt, amend, extend, or terminate the procedure; evaluators classify evidence but do not exercise rulemaking or adjudicative authority.","excluded_actions":["Changing hearing format, scheduling, venue, notice, appearance duties, or accommodation rights during the retrospective review","Withholding an urgent hearing or legally required in-person option to create confirmation evidence","Treating multiplicity-adjusted results as proof that the procedure is lawful, fair, or practically important","Using protected case information outside existing access, sealing, retention, and disclosure rules","Randomly assigning litigants or courts without separate legal, ethical, and institutional authorization","Extending, terminating, or publicly endorsing the pilot solely through the evidence-status charter"],"halt_rollback":"Halt if reconstruction exposes protected case information, interferes with an active matter, is used to predetermine rulemaking, or cannot distinguish analyses planned before results from those selected afterward; withdraw the paper classification, retain only records authorized under existing policy, and restore the ordinary evaluation process."},"baseline":"After a remote-hearing pilot, evaluators can examine numerous outcomes, litigant groups, court types, case categories, time windows, exclusion rules, and statistical models. A rulemaking memorandum may emphasize the most favorable association without showing the full set of analytic opportunities that produced it, allowing an exploratory pattern to appear like a planned evaluation result.","candidate_id":"multiple_testing_discipline__law_governance__ORDINARY_DIVERSE_P2","causal_chain":["A completed court-procedure pilot generates many plausible comparisons across outcomes, case types, courthouses, litigant characteristics, periods, and analytic specifications.","Repeated examination of the same pilot records creates multiple opportunities for a favorable result to arise from chance or analysis-specific variation.","If null findings and alternative specifications are omitted, the selected result appears to provide more singular support for permanent adoption than its search history warrants.","A rulemaking committee may consequently treat an exploratory benefit claim as decision-ready evidence when recommending a durable procedural rule.","A prospective evaluation charter defines the complete claim family, inventories attempted analyses, assigns outcome priorities, and applies a declared error-risk policy.","Findings receive explicit exploratory, adjusted-discovery, confirmed, or rejected status, while a frozen confirmation question is tested in a previously unused court cohort, later docket period, or independently administered replication.","The authorized rulemaker receives the complete discovery and confirmation record alongside legal, procedural, and practical considerations before deciding whether to adopt the rule."],"cell_id":"multiple_testing_discipline__law_governance","consequence":"A selectively favorable pilot finding can receive unwarranted evidentiary status and influence a durable court-procedure rule, potentially exposing litigants and courts to access, participation, fairness, or administrative consequences that were not supported by a confirmation burden matched to the evaluation's search space.","diversity_from_prior_proposals":"This opportunity concerns institutional evaluation and permanent judicial rulemaking for a court-procedure pilot. Its intervention prospectively partitions pilot evaluation from confirmation and governs the evidence package for a general procedural rule, rather than governing data-selected allegations or case-level enforcement escalation.","experiment_id":"eoa_inverse_innovation_exp13_second_slot_policy60_20260806","intervention":"Adopt a prospective evidence charter for evaluations of remote civil-hearing pilots. Before outcome analysis, the authorized evaluation lead records the primary and secondary outcomes, case categories, court groupings, litigant segments, time windows, exclusion rules, and model families that constitute the claim family. An access-controlled analysis ledger records every executed or materially considered specification and its status. The charter assigns a metric hierarchy and an error-risk policy: primary rulemaking claims receive family-level control, broad secondary screening may use discovery-rate control, and unadjusted searches remain explicitly exploratory. Any result selected after exploration is frozen as a confirmation question and cannot be described as confirmed until an evaluator separated from the discovery analysis tests it on a reserved court cohort, later docket period, or independently administered replication. The complete ledger, status labels, null results, confirmation outcome, and limitations accompany the public rulemaking record subject to lawful confidentiality restrictions.","mechanism_mapping":[{"counterfactual_removal":"Without a prospective record, outcome priorities and analysis choices can be reconstructed after results are known, obscuring which claims were planned and which emerged from search.","mechanism_slug":"preregistration","role":"Fixes the initial claim family, outcome hierarchy, analytic families, and confirmation routes before evaluators inspect pilot outcomes."},{"counterfactual_removal":"Without ordered outcomes, an unfavorable primary result can be displaced by whichever secondary measure, subgroup, or time window looks favorable.","mechanism_slug":"metric_hierarchy","role":"Separates primary rulemaking claims from secondary and exploratory outcomes and assigns each a corresponding evidentiary burden."},{"counterfactual_removal":"Without a durable ledger, null analyses and abandoned specifications can disappear from the memorandum considered by rulemakers.","mechanism_slug":"claim_registry","role":"Records planned and attempted analyses, specification changes, results, owners, status transitions, and confirmation obligations."},{"counterfactual_removal":"Without evidence unavailable during discovery, the same pilot records used to select a favorable claim can also appear to confirm it.","mechanism_slug":"holdout_validation","role":"Tests a frozen claim using a reserved court cohort or later docket period that was unavailable to the discovery analysts."}],"nearest_rivals":["Ordinary pilot evaluation summarizes implementation and outcomes but may not adjust the evidentiary status of a selected finding for the full analysis family.","Confounder control evaluates whether another factor explains a reported association; it does not address the many opportunities available to select that association.","Effect-size review asks whether a result is practically meaningful after it is credibly identified; it does not discipline selection across many outcomes and specifications.","Reproducibility procedures can rerun every analysis while still permitting one favorable result to be promoted without reference to the remaining searches.","Procedural due-process review evaluates the legality and fairness of remote hearings but does not determine whether empirical benefit claims survived a search-calibrated confirmation process."],"negative_tests":{"intervention_falsifier":"The charter fails in a shadow exercise if independent reviewers cannot reconstruct the full analysis family or reliably distinguish planned, exploratory, adjusted-discovery, and confirmed claims, or if committee materials continue to present exploratory results as confirmed despite the labels.","problem_falsifier":"The multiplicity problem is absent for the examined pilot if the rulemaking claim rests on one genuinely pre-specified outcome, population, period, and analysis path, with no material selection among alternative measures, subgroups, models, exclusions, or time windows.","risks":["Evaluators may define artificially narrow claim families that omit related analyses.","Outcome hierarchies may privilege administratively convenient measures over legally important burdens.","Strict family-level control may obscure useful exploratory leads that could support later study.","Reserved courts or later cohorts may differ for substantive reasons, making a failed confirmation difficult to interpret.","Discovery analysts may indirectly influence supposedly independent confirmation.","The ledger may disclose sealed records, protected characteristics, judicial deliberations, or security-sensitive operational information.","Rulemakers may mistake statistical confirmation for a complete assessment of legality, fairness, accessibility, or practical importance.","Confirmation requirements may delay a rule decision where existing law imposes a deadline."],"strongest_counterevidence":"The strongest counterevidence would be an audit showing that completed court-pilot evaluations already pre-specify complete analysis families, retain all material null and alternative analyses, apply a declared multiplicity policy, clearly separate exploratory claims, and use genuinely unused or independently collected evidence before selected findings support permanent rule adoption."},"next_evidence_step":"Choose one completed remote civil-hearing pilot under existing judicial evaluation authority. From retained protocols, code, meeting records, and draft memoranda, reconstruct every outcome, subgroup, court grouping, period, exclusion, and model materially examined. Have two authorized reviewers independently classify each reported claim as planned, exploratory, multiplicity-adjusted, confirmed, or indeterminate using a paper charter. Record omitted analyses, classification agreement, reconstruction time, confidentiality issues, and whether a lawful unused cohort or later period existed. Do not alter any case, pilot condition, or court rule.","observable_state":"For each pilot claim presented to rulemakers, an authorized reviewer can inspect a timestamped record of the claim family, outcome hierarchy, all executed and materially considered analyses, null and selected results, deviations from the evaluation plan, applicable error-risk rule, adjustment method, current status label, frozen confirmation question, separation and provenance of confirmation evidence, reviewer identities, and the authorized rulemaker's stated treatment of the evidence.","prior_art_status":"UNSEARCHED","problem":"A judicial rulemaking body evaluates a remote civil-hearing pilot by examining many outcomes, case categories, courthouses, litigant groups, time windows, exclusion rules, and model specifications. When the evaluation memorandum highlights a favorable pattern without the complete family of attempted analyses, an exploratory result selected from a broad search can be treated as confirmatory support for permanently changing court procedure.","proposal_index":2,"remaining_contrastive_claim":"The proposal adds value only when empirical support for a durable court-procedure rule is selected from multiple pilot outcomes or analytic paths. Its distinctive contribution is to make the entire evaluation family determine claim status and to require separated evidence before a selected finding becomes confirmatory; it does not decide the rule's legality, practical importance, distributive fairness, or ultimate merits.","revision_record":{"claim_changes":["Initial version; no novelty, prevalence, demand, or effect-size claim is made."],"conceptual_changes":["Initial version maps multiple-testing discipline to empirical evaluation preceding permanent judicial rulemaking."],"evidence_changes":["Prior art remains unsearched; the first evidence step is a bounded reconstruction of one completed pilot evaluation."],"operational_changes":["Initial operation is limited to a read-only reconstruction and paper evidence-status exercise with no change to cases, pilot conditions, or rules."],"parent_version":null,"progress_targets_addressed":["Independent law-and-governance problem centered on judicial rulemaking","Complete claim-family and analysis-history visibility","Prospective separation of pilot discovery from confirmation","Authority-preserving and non-interventional first step","Distinct problem and intervention falsifiers"]},"schema_version":1,"structural_mapping":[{"archetype_element":"Define the claim family","domain_realization":"Before analysis, specify the remote-hearing outcomes, case categories, court groupings, litigant segments, periods, exclusion rules, and model families that jointly create discovery opportunities."},{"archetype_element":"Inventory attempted looks","domain_realization":"Record every executed or materially considered outcome comparison, subgroup analysis, time window, exclusion, transformation, and model specification, including null and abandoned paths."},{"archetype_element":"Set the error-risk policy","domain_realization":"Assign strict family-level control to claims proposed as primary support for a durable rule, discovery-rate treatment to organized secondary screening, and exploratory status to unrestricted searches."},{"archetype_element":"Apply a multiplicity-aware rule","domain_realization":"Evaluate primary and secondary findings under their declared family-level or discovery-rate procedure and withhold confirmatory standing from unadjusted post hoc selections."},{"archetype_element":"Label claim status","domain_realization":"Mark each pilot claim as planned-unconfirmed, exploratory, adjusted-discovery, confirmed, rejected, or indeterminate in the rulemaking evidence record."},{"archetype_element":"Confirm before costly action","domain_realization":"Freeze a selected claim and test it through a separated evaluator using a reserved court cohort, later docket period, or independently administered replication before calling it confirmed."},{"archetype_element":"Preserve the discovery record","domain_realization":"Retain the evaluation plan, analysis ledger, null results, deviations, status changes, confirmation materials, reviewer access history, and rulemaker's treatment of the evidence under applicable judicial records rules."}],"title":"Prospective Evidence Charter for Permanent Court-Procedure Pilot Adoption","version":0}