Precedent Consistency Audit¶
Test or assessment — instantiates Precedent-Guided Decision Governance
Samples recurrent decisions to test whether similar cases were treated alike, citations were complete, exceptions were equally available, and status stayed fresh.
A precedent regime can look healthy at every individual decision and still be failing in aggregate — materially similar cases quietly diverging, adverse cases going uncited, exceptions leaking to insiders, stale rules lingering. The Precedent Consistency Audit is the periodic test that checks whether the system actually does what it claims, after the fact and at the population level. Its defining property is retrospective sampling: it pulls a set of already-decided cases and grades them against the regime's own promises — were like cases treated alike, was the adverse-search coverage real, were exceptions equally accessible, were validity statuses current. It produces findings and rates, not decisions; it does not touch any live case and does not resolve any conflict. Where a citation graph flags anomalies continuously as they arise, the audit deliberately steps back and asks a sampled population, "did the promise hold here?"
Example¶
A city housing authority grants and denies exceptions to occupancy rules under a precedent regime — hardship extensions, accommodation waivers, transfer approvals. Everything appears orderly case by case, but the authority runs a quarterly consistency audit to check the aggregate. The auditor draws a stratified sample of the quarter's exception decisions and grades each on several axes: were two applicants with materially similar hardships granted the same relief; did the decision file show an adverse-case search or only supportive citations; were the more generous exceptions reaching all applicant groups or clustering among those with caseworker relationships; and did any decision rest on a precedent whose validity status had since gone stale.
The audit surfaces a pattern no single file revealed: hardship extensions were granted at markedly different rates across two intake offices deciding indistinguishable cases, and one office almost never documented an adverse search. The audit decides nothing about those applicants — it cannot reopen them. What it delivers is a graded finding that triggers retraining at one office and a tightened search requirement, closing a drift the case-by-case process could not see.
How it works¶
The assessment's distinctive work is sampling and grading a population, not deciding a case:
- Draw a representative sample. Recurrent decisions are sampled — often stratified by decision type, office, or outcome — so the audit measures the system rather than a cherry-picked slice.
- Grade against the regime's promises. Each sampled case is scored on similarity-treatment consistency, citation and adverse-search completeness, exception accessibility, and status freshness — the things the regime claims to guarantee.
- Test exception equality explicitly. The audit checks whether favorable exceptions and nonpublic guidance reached similarly situated parties or leaked only to insiders — the secret-precedent failure.
- Report rates and patterns, not verdicts. Output is a set of measured gaps and drift signals routed to whoever owns remediation, not an action on any live matter.
Tuning parameters¶
- Sampling density and stratification — how many cases, and along which strata. Denser, well-stratified samples detect finer drift but cost more auditor time.
- Grading rubric strictness — how tight the "treated alike" and "adequate search" standards are. Strict rubrics catch subtle inequality but raise false-positive findings on defensible variation.
- Audit cadence — how often the audit runs. Frequent audits catch drift early but burden the system; rare ones let inequality compound.
- Scope of axes — how many promises each pass tests (consistency only, or also citation, access, and freshness). Broader scope is comprehensive but dilutes depth per axis.
- Independence of the auditor — how separated the auditor is from the decisions under review, trading candor against convenience.
When it helps, and when it misleads¶
Its strength is that it catches the failures that are invisible one case at a time — surface-similarity drift, selective citation, and unequal exception access only show up as patterns across many decisions, and the audit is the only mechanism that looks at the population. It operationalizes the formal-justice demand that like cases be treated alike by actually measuring whether they were, rather than trusting that they were.[n1]
Its failure mode is that an audit measures conformity to a rubric, and a rubric can be gamed or simply wrong: decision makers who know the audit's axes can produce files that pass the checks — a boilerplate adverse-search note, a citation added for form — while the underlying inconsistency persists. A poorly-specified "similar case" standard will either flag legitimate distinctions as drift or wave real inequality through. And because the audit is retrospective and sample-based, it can only find drift after it has happened and only in the cases it happened to draw. The guarding discipline is to rotate and conceal the audit's exact axes, pair rate findings with spot-read case review, and treat a passing audit as evidence of health rather than proof of it.
How it implements the components¶
This assessment fills the retrospective-verification slice of the archetype:
consistency_and_drift_monitor— it is the periodic drift check: sampled measurement of whether materially similar cases actually received like treatment.precedent_retrieval_and_adverse_search— it audits the coverage of prior searches, grading whether decisions documented adverse and limiting cases or only supportive ones.nonprecedential_decision_boundary— it tests exception equality, checking whether nonpublic guidance and favorable exceptions reached similarly situated parties rather than leaking to insiders.
It measures; it neither decides nor corrects. It does not choose a treatment (precedent_treatment_decision, held by Follow–Distinguish–Overrule Memo) or resolve the conflicts it may surface (precedent_conflict_resolution_rule, held by Precedent Conflict Panel). Unlike the continuous, node-level alerting of Precedent Citation Graph, it works by periodic population sampling rather than live propagation.
Related¶
- Instantiates: Precedent-Guided Decision Governance — the audit is the feedback loop that tells the regime whether its promises are actually kept in aggregate.
- Consumes: Precedent Citation Graph supplies the citation and status data the audit samples over.
- Sibling mechanisms: Adverse-Precedent Search Protocol · Follow–Distinguish–Overrule Memo · Material-Fact Comparison Matrix · Precedent Authority and Validity Table · Precedent Case Brief · Precedent Change Notice and Transition Plan · Precedent Citation Graph · Precedent Conflict Panel
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Precedent Consistency Audit operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it samples recurrent decisions to test whether similar cases were treated alike, citations were complete, exceptions were equally available, and status stayed fresh.
Independent corroboration: The frozen evidence defines Precedent Consistency Audit as 'Samples recurrent decisions to test whether similar cases were treated alike, citations were complete, exceptions were equally available, and status stayed fresh', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Law & Governance
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Testing whether like cases receive like treatment is rooted in legal equality and precedent governance.
Related originating lineages:
- Accounting & Auditing — Auditing contributes risk-based sampling, evidence trails, and systematic control testing.
Review resolution: Both blind reviewers agree that law governance is the primary origin. Reconciliation resolves domain reach disagreement, encyclopedia synthesis disagreement. Formative alternate lineages are retained as accounting_auditing; later breadth of use is recorded separately as domain_reach=multi_domain, while origin_mode=cross_disciplinary_synthesis describes the relationship among origin lineages.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] The principle that like cases should be treated alike — the formal core of justice traceable to Aristotle — is a demand about aggregate consistency, not any single case. The audit is what turns that demand from an assumption into a measured rate. ↩