Canonical Identity Resolution Pass¶
Transformation procedure — instantiates Inclusive Membership Union Design
Turns each source collection's own identifiers into one canonical member key, so the same real-world entity is recognized as the same member wherever it appears.
Before you can union several collections, you have to agree on what counts as "the same member." Canonical Identity Resolution Pass is the step that decides, for every incoming record, which real-world entity it refers to and stamps it with a single canonical key — the classic record-linkage problem. Source A calls the entity R. Chen, source B calls it Robert Chen Jr., source C carries only an internal number; the pass judges that these are one entity and assigns them one shared identifier. Its defining move is that it manufactures the identity key — it produces the crosswalk that says "these differently-labelled records are one member" — but it stops there. It does not yet remove the duplicates or emit a combined set; it only makes them recognizable as duplicates. Everything downstream depends on this key being right.
Example¶
A regional hospital network is combining patient records from three clinics that never shared a system. Clinic A stores Robert Chen, clinic B Bob Chen with a date of birth, clinic C only a medical-record number. Left alone, a union would treat one patient as three. The resolution pass runs in stages: first a deterministic match on strong keys (national health ID, exact DOB plus surname), then a probabilistic pass on weaker signals (fuzzy name similarity, overlapping address) for the records the strong keys missed. A type gate sits in front of every comparison — a record tagged provider is never allowed to match a record tagged patient, even when the names are identical, so Dr. Chen the physician is never fused with Robert Chen the patient.
The output is not a cleaned list but a crosswalk: clinicA:8831, clinicB:P-204, and clinicC:MRN-77 all now carry canonical key PT-0009142. Ambiguous pairs that score in the middle band are routed to a human reviewer rather than auto-merged. That crosswalk is what makes the later union honest — the same person is now one member no matter which clinic contributed the record.
How it works¶
- Block, then compare. Partition records into candidate blocks (same DOB year, same postal prefix) so the pass compares plausible pairs instead of every record against every other.
- Deterministic first, probabilistic second. Exact matches on strong identifiers resolve cheaply and confidently; only the residue goes to fuzzy scoring.
- Gate on type. Two records may be compared only if their declared entity types are compatible; a mismatch blocks the link outright, before any name similarity is even scored.
- Score into three bands. Auto-merge above an upper threshold, no-merge below a lower one, and route the middle band to review — the pass never silently guesses on the hard cases.
- Emit a crosswalk, not a collapsed set. The result is a mapping from source identifiers to canonical keys; consuming the crosswalk to actually fold records together is a separate step.
Tuning parameters¶
- Match threshold — how high the score must be to auto-merge. Raise it and you avoid false merges but leave more true duplicates unlinked; lower it and you catch more but risk fusing two distinct entities.
- Blocking granularity — how coarse the candidate blocks are. Coarser blocks catch more true pairs but explode the comparison count; finer blocks are fast but miss matches that straddle a block boundary.
- Deterministic / probabilistic weighting — how much of the work strong keys do before fuzzy scoring takes over. More determinism is safer where clean identifiers exist; more probabilistic reach is needed for messy, human-entered data.
- Review-band width — how wide the "send to a human" middle zone is. Wider bands cost reviewer time but catch the dangerous ambiguous merges the automation would get wrong.
- Transitivity handling — whether A↔B and B↔C automatically imply A↔C. Allowing transitive chains resolves fragmented entities but can snowball a single bad link into a giant false cluster.
When it helps, and when it misleads¶
Its strength is that it gives the whole union a trustworthy notion of sameness: get the key right and duplicate collapse, provenance, and overlap counts all become well-defined; get it wrong and every downstream number is quietly corrupted. The formal backbone here is the Fellegi–Sunter model of probabilistic record linkage, which frames matching as a likelihood decision with explicit error rates rather than a yes/no guess.[1]
Its failure mode is the two-sided error that model names. Over-merging fuses two different entities into one member — in a patient index that can attach one person's allergies to another's chart, which is why the review band exists. Under-merging leaves the same entity split across several keys, silently inflating the union. The classic misuse is matching across incompatible types because a field name happens to align — a product_id treated as a customer_id because both are called id. The discipline that guards against it is to keep the type gate strict, hold the middle band for human eyes, and retain the source identifiers so any bad merge can be traced and reversed rather than baked in.
How it implements the components¶
canonical_member_identity_key— its primary output: the crosswalk that assigns one canonical key per real-world entity across mismatched source identifiers.type_compatibility_gate— enforced as a precondition on every comparison, so records of incompatible entity types are never linked no matter how similar their labels.
This pass assigns keys; it does not fold the matched records into a single emitted member — collapsing duplicates that share a key (duplicate_collapse_policy) is Deduplicating Union Pass, the nearest twin. It also does not run the union membership test itself (inclusive_membership_rule), which belongs to Inclusive-OR Membership Test.
Related¶
- Instantiates: Inclusive Membership Union Design — supplies the canonical identity key the whole union is built on.
- Consumes: Source Collection Ingestion Workflow delivers the registered, normalized source records this pass links.
- Sibling mechanisms: Deduplicating Union Pass · Inclusive-OR Membership Test · Overlap and Coverage Dashboard · Provenance Tagging Protocol · Type Compatibility Checklist · Union Delta Review · Union Specification Sheet
Editorial Notes¶
Form Classification¶
Form family: Intervention, Treatment & Transformation
Rationale: The pass blocks and compares source records, resolves matches, and rewrites disparate identifiers to one canonical member key, so its operative form is a data transformation.
Nearest alternative: Analysis, Modeling & Optimization — Similarity analysis supports matching, but the defining result is changed identity state across the collections.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Data Science & Analytics
Origin pattern: Convergent development
Present-day reach: Multi-domain
Rationale: Data integration established entity resolution that maps source identifiers to one canonical real-world entity key.
Related originating lineages:
- Library & Information Science — Authority-control practice independently reconciles variant names to governed identities.
- Statistics & Experimental Design — Probabilistic record linkage supplies error-bounded match and non-match thresholds.
Review resolution: Data science is primary through entity resolution and probabilistic record linkage. Library authority control and statistical linkage developed materially independent identity-matching traditions, so convergent origin and multi-domain reach are warranted.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] The Fellegi–Sunter model (1969) formalizes record linkage as a decision between "match" and "non-match" based on the likelihood ratio of agreement patterns, with tunable thresholds that fix expected false-match and false-non-match rates — the reason the pass sorts pairs into auto-merge, review, and no-merge bands rather than a single cut. withdrawn registry ↩