Process-Based Missingness Audit¶
Procedure — instantiates Missingness-Aware Estimator Selection
Uses field knowledge, collection logs, device records, administrative rules, or interview protocols to infer why data became absent.
The Process-Based Missingness Audit is the investigative procedure that reconstructs why values are absent by going to the data-generating process itself — collection logs, device event records, administrative rules, fieldworker notes, interview protocols — rather than inferring anything from the numbers that survived. Its defining stance is that the mechanism is a fact about how the data were produced, not a statistical property to be estimated: you learn that a field is blank because a form only asks it of applicants over a threshold, or that readings vanished because a device powers down below a temperature, and that knowledge names the mechanism directly. This is what separates the audit from every purely statistical mechanism here — it supplies the causal story that the estimators and diagnostics then take as input, and it does so from evidence outside the dataset.
Example¶
A benefits agency has administrative records where "monthly earnings" is missing for many claimants, and an analyst is tempted to impute it. The audit instead goes to the process. Interviews with caseworkers and a read of the intake rules reveal three distinct reasons a claimant's earnings can be blank: the field is not collected for claimants under a certain benefit category (a structural non-applicability, not missing data at all); it is left blank by caseworkers during high-caseload periods (a workload-driven, roughly ignorable gap); and it is withheld by claimants with irregular cash income who fear disqualification (an outcome-driven, non-ignorable gap). The audit documents each pathway with its evidence, registers which analysis path each licenses — drop the non-applicable rows entirely, treat the workload gaps as MAR conditional on office and date, and flag the withholding gaps for sensitivity analysis — and records the explanations it rejected along the way. The downstream estimator selection now rests on a defended account of the process, not a guess read off the residuals.
How it works¶
- Trace the collection pathway. Walk the actual process — forms, systems, device behavior, protocols — that produces or suppresses each value.
- Gather process evidence. Pull collection logs, paradata, admin rules, and practitioner knowledge that explain absence.[n1]
- Name the mechanism per source. Attribute a candidate mechanism (structural, ignorable, or non-ignorable) to each distinct cause of absence.
- Register the licensed paths. Record which analysis each mechanism permits, the rationale, and the rejected alternatives, as a standing ledger.
Tuning parameters¶
- Evidence depth — from a quick protocol read to full fieldworker interviews and log forensics; deeper audits are more credible but costly.
- Cause granularity — one blanket story versus a separate mechanism per pathway; finer attribution is more accurate but harder to act on.
- Structural-vs-missing boundary — how strictly "not applicable," censored, and truly missing are separated, which determines what even enters the missing-data analysis.
- Documentation formality — a note versus a registered, reviewable ledger of causes, paths, and rejections.
When it helps, and when it misleads¶
Its strength is that it is the only source of information about the mechanism that does not come from the (surviving) data: statistical tests can never confirm dependence on unobserved values, but a collection log or an administrative rule sometimes can settle it outright, and it routinely separates true missingness from non-applicability that should never have been imputed.
Its failure mode is that process accounts can be partial or self-serving — practitioners rationalize, logs are incomplete, and a plausible-sounding story can harden into an unexamined assumption. The classic misuse is to accept a single convenient narrative ("respondents just forgot") and treat it as license for a simple method without corroboration. The guarding discipline is to demand documented evidence for each attributed cause, to register the rejected explanations alongside the accepted ones, and to hand genuinely uncertain pathways to sensitivity analysis rather than resolving them by assertion.
How it implements the components¶
missingness_mechanism_assumption_frame— the audit supplies the mechanism assumption from process evidence, attributing an MCAR/MAR/MNAR (or structural) label to each cause of absence on documented grounds rather than statistical inference.analysis_path_registry— it produces the standing ledger of which analysis paths each mechanism licenses, the assumption rationale, and the alternatives rejected, making the selection auditable.
It does not quantify which cells are absent or publish the pattern artifact — missingness_pattern_inventory and diagnostic_trace_and_reporting_record are Missingness Indicator Matrix's; the audit explains why data are missing, whereas the matrix only records which values are missing and how their absence is patterned.
Related¶
- Instantiates: Missingness-Aware Estimator Selection — the process-evidence route to naming the mechanism.
- Sibling mechanisms: Doubly Robust Missingness Adjustment · Full-Information Maximum Likelihood Path · Inverse-Probability Weighting Model · MCAR Diagnostic Test and Balance Review · Missingness Indicator Matrix · Multiple Imputation Workflow · Pattern-Mixture Sensitivity Model · Selection-Model Sensitivity Analysis · Tipping-Point Analysis
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Process-Based Missingness Audit operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it uses field knowledge, collection logs, device records, administrative rules, or interview protocols to infer why data became absent.
Independent corroboration: The frozen evidence defines Process-Based Missingness Audit as 'Uses field knowledge, collection logs, device records, administrative rules, or interview protocols to infer why data became absent', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Diagnosing the data-generating process behind absence is central to statistical missing-data analysis.
Related originating lineages:
- Data Science & Analytics — The data_science tradition materially shaped Process-Based Missingness Audit through its own practice of production data pipelines, predictive modeling, and machine-learning validation.
- Ethnography & Qualitative Methods — The ethnography_qualitative_methods tradition materially shaped Process-Based Missingness Audit through its own practice of field-grounded inquiry into situated practices and participant experience.
Review resolution: Both blind reviewers agree that statistics experimental design is the primary origin. Explicit reconciliation resolves reported ambiguity. Formative alternate lineages are retained as data_science, ethnography_qualitative_methods; later breadth of use is recorded separately as domain_reach=multi_domain, while origin_mode=cross_disciplinary_synthesis describes the relationship among origin lineages.
Attribution caveat: The exact encyclopedia label appears to synthesize established practices; the primary domain identifies the strongest formative lineage, while the alternates record material ingredients rather than downstream uses.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
Notes¶
[n1] Paradata — data about the data-collection process itself (call attempts, timestamps, interviewer IDs, device logs, edit trails). Because paradata describe how observation happened, they are a primary source for inferring why values are missing, and often become the auxiliary variables that MAR-based estimators rely on. ↩