Count Impact Assessment¶
Test / assessment — instantiates Entity Individuation Criteria Design
Estimates how a proposed individuation rule changes entity counts, denominators, eligibility, and exposure before the rule is adopted.
Every individuation rule is also, silently, a counting rule — change what makes something one entity and you change how many there are, which changes every rate, quota, and eligibility test computed on top of them. Count Impact Assessment is the before-and-after study that makes that silent effect loud: given a proposed rule (a new same-as threshold, a redrawn unity boundary, a different persistence stance), it re-derives the inventory of countable individuals under the new rule and the old one side by side, then traces the deltas into the metrics that consume those counts. Its defining move is that it never asks whether the rule is correct — that judgment belongs to the criteria authors. It asks only what the rule does to the numbers, so a change that looks purely definitional cannot slip through without its arithmetic consequences on the table.
Example¶
A national tuberculosis surveillance program is deciding whether two positive lab results from the same patient within 14 days should count as one case or two. Clinically it feels like a bookkeeping detail. Count Impact Assessment treats it as an intervention. Analysts re-run the past three years of records under both rules: under "merge within 14 days" the annual case count drops by roughly 8%, and — because the denominator of the treatment-completion rate shrinks while the numerator barely moves — the reported completion rate rises by about two points. That rate is the one districts are funded against.
The assessment does not recommend either rule. It produces a table: total cases, per-district cases (the drop is concentrated in three high-volume clinics that re-test aggressively), the affected rates, and the eligibility line for a supplemental-funding tier keyed to case volume — two districts cross it under the new rule. That table is what turns a quiet definitional edit into a governed decision, because now everyone can see that "one case or two" quietly moves money.
How it works¶
- Freeze a comparison inventory. Take a representative slice of real records and materialize the countable-individual register twice — once under the incumbent rule, once under the proposal — from the same underlying data, so every difference is attributable to the rule alone.
- Diff the registers. Report not just the net count change but its distribution: which populations, sites, or strata absorb the merges and splits, since aggregate stability often hides large offsetting local swings.
- Propagate into consumers. Push both inventories through the metrics, rates, quotas, and eligibility tests that read them, and flag every threshold the change causes something to cross.
- Surface reflexivity risk. Note where the count feeds a target or payment, so authors can see whether the rule creates an incentive to game the new boundary.
Tuning parameters¶
- Sample scope — a single site's records versus the full historical corpus. Wider scope catches distributional surprises but costs compute and time; narrow scope risks missing the stratum where the rule bites.
- Consequence depth — stop at raw counts, or chase deltas all the way into dollars, rights, and privacy exposure. Deeper tracing is more decision-relevant but multiplies assumptions.
- Baseline choice — compare against today's rule, against a naïve no-rule count, or against several candidate proposals at once. The baseline frames how large the impact looks.
- Materiality threshold — how big a delta must be before it is reported rather than footnoted. Set low near funding lines, higher for descriptive metrics.
When it helps, and when it misleads¶
Its strength is converting "it's just a definition" into an auditable set of numbers, and catching the one failure this archetype dreads most — a count whose basis quietly shifts and takes eligibility or funding with it. It is at its best precisely when the individuation change is small and technical, because those are the ones that ship unreviewed.
Its failure mode is that it can be weaponized backward: run several candidate rules, then pick the one whose count lands where you wanted it. A count that becomes a target stops measuring and starts being gamed — Goodhart's law in miniature.[n1] The assessment also inherits the representativeness of its sample; a slice that under-covers the aggressive-retest clinics will understate the impact. The guarding discipline is to fix the sample and the candidate rules before looking at any output, and to report the full distribution of deltas rather than the headline net, so a convenient aggregate cannot hide the local swings that actually matter.
How it implements the components¶
countable_inventory_register— materializes it twice, as the paired before/after inventories that are the assessment's raw material; the register here is a comparison artifact, not a system of record.downstream_consequence_audit— its core deliverable: the traced effect of the count change on denominators, rates, eligibility, and exposure.
It does not author the unity_criterion or identity_criterion whose change it measures — those are set in the Entity Definition Workshop and checked in the Identity and Unity Test Checklist; this assessment only computes their arithmetic downstream effect.
Related¶
- Instantiates: Entity Individuation Criteria Design — supplies the consequence evidence a rule change must clear before adoption.
- Consumes: Master Entity Registry — the source of the real records the paired inventories are built from.
- Sibling mechanisms: Entity Definition Workshop · Identity and Unity Test Checklist · Individuation Criteria Charter · Entity Resolution Policy · Split/Merge Decision Tree · Master Entity Registry · Edge-Case Adjudication Panel · Versioned Identity Rulebook
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: The mechanism deliberately materializes the same real inventory under incumbent and proposed individuation rules and propagates both through downstream metrics and thresholds, generating evidence of the change's impact before adoption.
Nearest alternative: Assessment, Review & Assurance — It produces an impact finding, but the evidence is generated through an active counterfactual replay rather than inspection of an unchanged inventory.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Public Administration & Policy
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Policy impact assessment cohered prospective comparison of how a rule change alters eligibility, affected populations, denominators, and resource exposure.
Related originating lineages:
- Law & Governance — Legal classification rules supply the boundary changes whose adoption alters who or what counts.
- Statistics & Experimental Design — Measurement theory supplies sensitivity analysis for changes in operational definitions, units, and derived rates.
Review resolution: The protocol synthesizes public-policy impact assessment with legal equality review and statistical disparity measurement; both listed alternates are provenance-bearing.
Attribution caveat: The specific focus on individuation counts is an encyclopedia synthesis of regulatory-impact and measurement-sensitivity practice.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
Notes¶
[n1] Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." Here the risk is that once a case-count rule is known to drive funding, sites adjust behavior (or authors adjust the rule) to move the count rather than to reflect reality. ↩