Balanced-Panel Completeness Check¶
Test / assessment — instantiates Time Series Cross-Section Analysis
Assesses whether units have observations across required periods and where missingness threatens comparison.
A balanced-panel completeness check audits the unit-by-period grid for holes before any comparison is trusted. It tallies which units are observed in which required periods, flags late entrants, drop-outs, and interior gaps, and — the part that distinguishes it from a mere data-quality tally — asks whether the missingness is benign or selective: whether the units that vanish differ systematically from the units that remain. Its output is not an estimate but a verdict on the panel itself: which comparisons the data can legitimately support, over which window, and where attrition would quietly bias whatever design runs on top. It is the gatekeeper that stops an elegant model from being fitted on a silently self-selected sample.
Example¶
A state education agency has six years of reading scores for 500 schools and wants to study whether scores are trending up. Before any analysis, an analyst builds the presence matrix — schools by years, a mark in each cell that has data. It reveals structure a raw average would ignore: 380 schools have all six years; 70 entered late (newly opened); 50 have interior gaps from skipped testing years; and 30 exited before year six. The decisive finding is why the 30 exited — they were disproportionately the lowest-scoring schools, several of them closed for poor performance. Simply dropping incomplete schools to get a tidy "balanced" panel would delete the weakest performers and make the surviving trend look like improvement that never happened. The check flags this as attrition correlated with the outcome, recommends reporting the trend both with and without the leavers, and blocks the naive listwise deletion that would have manufactured good news.
How it works¶
- Build the presence matrix. Mark every unit-period cell as observed or missing; summarize with a balance ratio (fully observed units over total).
- Classify the missingness pattern. Separate late entry, interior gaps, and attrition, since each threatens comparison differently.
- Test whether leavers differ from stayers. Compare units that drop out against those that remain on observable characteristics and on the outcome itself; systematic differences signal missingness that is not at random.
- Recommend a handling rule. Restrict the window, keep an unbalanced panel with appropriate methods, or impute — but never silently delete.
Tuning parameters¶
- Required-period set — which periods a unit must have to count as complete. A strict set shrinks the usable panel; a lax set admits units too sparse to compare.
- Balance threshold — the minimum share of complete units before the panel is judged usable; too high discards data, too low tolerates bias.
- Attrition-difference tolerance — how large a leaver-versus-stayer gap is allowed before the missingness is treated as non-random.
- Missingness rule — listwise deletion, imputation, or window restriction; each trades completeness against introduced bias.
When it helps, and when it misleads¶
Its strength is catching selection before it contaminates an estimate — it is the discipline that separates "the units we can see" from "the units there are," and it is the cheapest insurance against a confident conclusion drawn from survivors. The failure mode is subtle: the check can pass mechanically and still mislead. A perfectly balanced subsample can itself be selected — if the units that survived to be complete are the robust ones, restricting to them is textbook survivorship bias, a special case of data that is missing not at random.[n1] And a grid can be complete cell-by-cell while the measurements inside those cells have drifted, so the units are present but no longer comparable. The classic misuse is reporting the balance ratio as a clean bill of health. The guarding discipline is to always compare leavers to stayers rather than merely counting cells, and to treat completeness as necessary but never sufficient for comparability.
How it implements the components¶
panel_validity_diagnostics— this check is the missingness, attrition, and comparability diagnostic the archetype calls for.unit_time_observation_matrix— it operates on the presence structure of the unit-by-period matrix, reading which cells exist.time_index_and_window— the "required periods" and observation window define what complete even means for a given question.
It draws no contrast and fits no effect: it does not implement within_unit_change_contrast or between_unit_difference_contrast — those belong to the estimators it certifies, such as difference_in_differences_design and fixed_effects_panel_model. This check only guarantees the grid those designs stand on; it never interprets it.
Related¶
- Instantiates: Time Series Cross-Section Analysis — it supplies the archetype's demand that missingness and attrition be reported before attribution.
- Sibling mechanisms: fixed_effects_panel_model · difference_in_differences_design · event_study_panel_plot · unit_time_dashboard · lagged_panel_regression · peer_trajectory_benchmarking
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Assesses whether units have observations across required periods and where missingness threatens comparison, making its operative form a bounded evaluation of existing evidence or work that produces a finding or disposition.
Independent corroboration: The frozen evidence defines Balanced-Panel Completeness Check as 'Assesses whether units have observations across required periods and where missingness threatens comparison', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Longitudinal and panel-data methodology audits the unit-by-period grid and selective attrition before estimation.
Related originating lineages:
- Data Science & Analytics — Data-quality tooling operationalizes completeness checks over panel tables.
- Economics & Finance — Econometrics developed extensive balanced-versus-unbalanced panel practice.
Review resolution: Statistics is the agreed primary lineage through longitudinal missingness and attrition checks. Econometrics is a major panel-data tradition and data engineering operationalizes the unit-by-period completeness audit.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Missing-not-at-random (MNAR) attrition occurs when the probability that a unit drops out depends on its own (unobserved) outcome. Survivorship bias is its most common form: restricting to units that remained complete over-represents the robust or successful cases, so a trend computed on survivors flatters reality. This is why comparing leavers to stayers matters more than the raw count of complete cells. ↩