Label Proxy Screen¶
Diagnostic screen — instantiates Leakage-Resistant Validation Design
Scans every candidate feature for the tell-tale signature of a target proxy — a column that is suspiciously predictive because it is really a downstream trace of the outcome — and files the suspects for confirmation.
Some features are predictive not because they cause the outcome but because they are produced by it: a status flag flipped when a case closes, a code entered only after the event, an aggregate refreshed once the result is known. The Label Proxy Screen is the cheap, standing scan that hunts for exactly this signature — target leakage[n1] — before any model is trusted. It walks the candidate feature set asking, of each column, "is this implausibly aligned with the label, and could its value only have come from the outcome?", and files every suspect into a leakage-pathway inventory. Its defining move is that it screens features by their relationship to the target, not the pipeline by its plumbing: it is a front-line detector that produces a list of things to distrust, not a verdict on how much any of them matters.
Example¶
A hospital builds a thirty-day readmission model. The Label Proxy Screen runs across the feature table and flags three columns. discharge_disposition = "expired" correlates almost perfectly with "not readmitted" — a proxy for the outcome by another name. post_discharge_care_coordinator_assigned is populated only after the readmission decision is effectively made, so its presence is a downstream trace. A rolled-up count_of_followup_visits is computed over a window that extends past the prediction point. None of these is obviously a label, but each carries the target's fingerprint: near-perfect association plus a value that materializes only once the outcome exists. The screen doesn't decide the model is broken; it hands the modeling team a short, ranked inventory of pathways to confirm — the input the Leakage Ablation Test then removes and measures.
How it works¶
What distinguishes it from ordinary feature selection is that it is looking for the shape of a proxy, not for weak predictors to drop:
- Flag features whose univariate association with the target is implausibly high for a genuine predictor — the "too good to be a cause" signature.
- Cross-check each suspect against when its value is populated relative to the decision point; a field that only exists post-outcome is a proxy regardless of how it correlates.
- Look for near-duplicate encodings of the label — status strings, resolution codes, post-event aggregates — that restate the outcome in feature clothing.
- Emit a ranked pathway inventory of suspects with the reason each was flagged, so confirmation can be targeted rather than exhaustive.
Tuning parameters¶
- Suspicion threshold — how strong an association counts as "too predictive." Lower it and the screen catches subtle proxies but floods the inventory with false alarms; raise it and clean-looking leaks slip through.
- Post-outcome test — how strictly a feature must prove it is populated before the decision point. Strict timing checks catch traces that correlation alone misses, but need reliable availability metadata to run.
- Scope of scan — raw columns only, or engineered and aggregated features too. Wider scope catches leaks born in feature engineering but is heavier to run each build.
- Automation cadence — a one-off audit or a check wired into every feature-set change. Continuous screening keeps new proxies from creeping in but must stay fast enough not to be disabled.
When it helps, and when it misleads¶
Its strength is coverage at low cost: it sweeps the whole feature set cheaply and surfaces the single most common and most embarrassing leak — a feature that is secretly the answer — long before a model reaches production. It is the wide net that decides what is worth the expensive confirmation step.
Its central failure mode is that it trades precision for recall: a purely statistical screen flags legitimately strong predictors as suspects and, worse, misses proxies whose association is only moderate, so passing the screen is not a clean bill of health. The classic misuse is to treat a flag as a conviction and silently delete every suspect — which can amputate genuine signal to make a leak worry disappear. The discipline that guards against this is to treat the screen's output as an inventory of hypotheses to confirm by removal and measurement, never as a final judgment, and to pair every statistical flag with a mechanistic reason the feature could be a downstream trace.
How it implements the components¶
Label Proxy Screen realizes the detection-and-enumeration side of the archetype — finding candidate leaks, not measuring or sealing them:
leakage_pathway_inventory— its output artifact: the ranked list of suspect features, each tagged with why it looks like a target proxy.contamination_probe_suite— it is the cheap, always-on front-line probe in the suite, run on every feature set to catch proxies before modeling.
It does not measure how much any suspect actually inflates performance — that is Leakage Ablation Test's leakage_severity_rating — and it does not establish when each field becomes knowable; that timeline comes from Feature Availability Audit.
Related¶
- Instantiates: Leakage-Resistant Validation Design — the screen is the first-pass detector that decides which pathways are worth confirming.
- Consumes: Feature Availability Audit supplies when each field is populated, which the screen uses to separate post-outcome traces from honest predictors.
- Sibling mechanisms: Leakage Ablation Test · Duplicate and Near-Duplicate Scan · Feature Availability Audit · As-Of Join Rule · Benchmark Deduplication Scan · Entity-Grouped Split · Nested Cross-Validation · Preprocessing Fit-on-Training-Only · Time-Based Holdout · Fresh Holdout Retest · Holdout Access Log
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Label Proxy Screen operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it scans every candidate feature for the tell-tale signature of a target proxy — a column that is suspiciously predictive because it is really a downstream trace of the outcome — and files the suspects for confirmation
Independent corroboration: The frozen evidence defines Label Proxy Screen as 'Scans every candidate feature for the tell-tale signature of a target proxy — a column that is suspiciously predictive because it is really a downstream trace of the outcome — and files the suspects for confirmation', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Data Science & Analytics
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Production machine-learning practice developed feature audits for target leakage and post-outcome proxies.
Related originating lineages:
- Statistics & Experimental Design — Study-design traditions supplied temporal ordering and contamination checks for predictors.
- Ethics of Technology & AI Governance — Algorithmic fairness practice materially shaped screening for forbidden or harmful proxies.
Review resolution: Both independent reviews place the primary lineage in data_science. The queued differences (domain_reach_disagreement, encyclopedia_synthesis_disagreement) concern secondary metadata rather than primary provenance. The final retains statistics_experimental_design, tech_ethics_ai_governance only where a reviewer supplied a formative-lineage rationale; downstream application by itself is not treated as origin. origin_mode=cross_disciplinary_synthesis records the relationship among origin traditions, while domain_reach=multi_domain records application breadth separately. encyclopedia_synthesis=true reflects whether either reviewer identified a corpus-specific synthesis, and confidence=high preserves the more cautious evidence assessment.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Target leakage — a feature carries information about the label that would not exist at prediction time, because the feature is itself produced by, or downstream of, the outcome. It is the canonical failure this screen looks for; the data-mining literature on leakage (Kaufman, Rosset & Perlich) formulates and catalogues it. ↩