Entity-Grouped Split¶
Validation-design protocol — instantiates Leakage-Resistant Validation Design
Partitions train and test by the underlying entity — patient, speaker, site, household, lineage — so no single entity has rows on both sides of the boundary.
When one real-world entity produces many rows, a random split scatters that entity's rows across train and test — and the model can then score well simply by recognising the entity rather than learning the pattern. The Entity-Grouped Split removes that shortcut by changing the atomic unit of partitioning: instead of splitting rows, it splits groups, keeping every row belonging to one entity wholly on a single side. Its defining choice is the grouping key — the answer to "what is the thing that must not appear on both sides?" Get that key right and the test set finally measures generalisation to new entities, which is almost always the question deployment will actually ask.
Example¶
A team building a voice-authentication model has thousands of utterances but only a few hundred speakers, with many clips per speaker. Split by clip at random, and most speakers have some clips in train and some in test; the model quietly learns to identify the speaker's microphone and vocal timbre, and reports near-perfect accuracy. The Entity-Grouped Split sets the grouping key to speaker: every speaker's clips go entirely to one side, so the test set contains only voices the model has never heard. Accuracy drops sharply — and honestly — revealing that much of the earlier score was identity recognition, not the intended task. The same protocol re-appears wherever rows cluster: images per patient, transactions per household, readings per sensor, sequences per molecular scaffold.
How it works¶
Its one distinguishing decision is the unit; everything else follows from it:
- Choose the group key — the entity whose presence on both sides would constitute leakage (speaker, patient, site, household, lineage).
- Assign whole groups, not rows, to folds — grouped k-fold or a grouped holdout — so a group is never divided.
- Where several nuisance factors exist, nest the grouping (e.g. grouped and temporal), prioritising the leakage that most resembles the deployment gap.
- Accept that folds become uneven in size — a large entity moves as one block — and rebalance on entities rather than rows.
Tuning parameters¶
- Grouping-key granularity — patient vs. patient-visit, device vs. session. Coarser groups block more leakage but reduce the number of independent units and can waste data.
- Nesting — grouped only, or grouped combined with a temporal or site split. Each added axis closes one leak pathway at the cost of shrinking the usable test set.
- Fold count — a single grouped holdout vs. grouped k-fold. More folds give steadier estimates unless groups are so few that each fold turns tiny.
- Balance policy — how hard to enforce class or covariate balance across sides when whole entities must move together; a dominant entity can skew a fold.
- Fallback for missing keys — when an explicit entity key is absent, whether to derive groups from a near-duplicate cluster instead (handing the question to a scan).
When it helps, and when it misleads¶
Its strength is that it neutralises identity leakage — the model recognising the entity rather than the phenomenon — and reframes the evaluation around the "new entity" generalisation that production usually demands.
It misleads exactly when the grouping key fails to capture the true shared factor. Split by image-id when the real nuisance is the scanner, or by user-id when one person holds two accounts, and leakage survives a split that looks rigorous on paper. Over-coarse grouping has the opposite failure: it discards so many independent units that the estimate becomes high-variance and pessimistic. The classic misuse is declaring an entity-grouped split while implementing it on a key that doesn't actually carry the leak. The discipline is to name the shared nuisance first and choose the key to match it — standard grouped cross-validation practice[1] — and to confirm the key with a duplicate scan when identity is uncertain.
How it implements the components¶
split_unit_definition— it is the definition of the split unit: it declares the entity, not the row, as the atomic thing a partition may not divide.holdout_integrity_boundary— by keeping every row of an entity on one side, it preserves the train/test boundary against identity bleed, so the holdout stays genuinely unseen.
It defines and enforces the grouping but does not detect the near-duplicates that reveal a mis-chosen key — that is Duplicate and Near-Duplicate Scan — and it addresses entity leakage, not temporal look-ahead, which is handled by Time-Based Holdout.
Related¶
- Instantiates: Leakage-Resistant Validation Design — the split defines the unit at which the holdout boundary is drawn.
- Sibling mechanisms: Duplicate and Near-Duplicate Scan · Feature Availability Audit · As-Of Join Rule · Benchmark Deduplication Scan · Fresh Holdout Retest · Time-Based Holdout · Nested Cross-Validation · Preprocessing Fit-on-Training-Only · Label Proxy Screen · Leakage Ablation Test · Holdout Access Log
Editorial Notes¶
Form Classification¶
Form family: Protocol, Workflow & Routine
Rationale: Entity-Grouped Split operates as a repeatable ordered procedure or handoff sequence that coordinates action because it partitions train and test by the underlying entity — patient, speaker, site, household, lineage — so no single entity has rows on both sides of the boundary.
Independent corroboration: The frozen evidence defines Entity-Grouped Split as 'Partitions train and test by the underlying entity — patient, speaker, site, household, lineage — so no single entity has rows on both sides of the boundary', so its operative form is Protocol, Workflow & Routine.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Data Science & Analytics
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Specialized
Rationale: Machine-learning validation established group-aware train-test partitioning so repeated observations from one entity cannot leak across the evaluation boundary.
Related originating lineages:
- Statistics & Experimental Design — Independent experimental units and clustered observations supplied the statistical foundation.
Review resolution: Scikit-learn defines GroupKFold specifically as non-overlapping group folds; the exact artifact is machine-learning validation built on statistical independence, so data science is primary.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
Notes¶
An entity-grouped split assumes the entity key is known and trustworthy. Its worst blind spot is the hidden duplicate — two records that are secretly the same entity under different keys — which a grouped split will happily place on opposite sides. That is why it pairs naturally with a near-duplicate scan: the scan discovers the clusters that the grouping key should have merged.
References¶
[1] Roberts, D. R., et al. "Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure". Ecography 40(8), 913–929 (2017). Recommends blocked cross-validation matched to temporal, spatial, hierarchical, or phylogenetic dependence in the data. registry ↩