Randomization Integrity Audit¶
Procedure — instantiates Baseline Covariate Balance Verification
A forensic check that the assignment actually recorded in the data matches the intended randomization — right allocation ratio, right sequence, no overrides or broken linkage.
A Randomization Integrity Audit interrogates the assignment mechanism itself rather than the covariates it produced. Every other mechanism in this archetype reads the realized allocation as given and asks whether the groups look comparable; this one asks the prior question — is the allocation on record the one the randomization procedure was supposed to generate? It reconstructs what the intended process should have produced (from the seed, the block structure, the enrollment sequence) and compares it against the assignment actually stored: the observed allocation ratio, the order of enrollment, any manual overrides, and the linkage between assignment records and baseline data. Covariate imbalance is a symptom the other mechanisms detect; this audit hunts the cause — a broken assignment service, a subverted allocation, a mislinked join key.
Example¶
A political-science field experiment tests a door-to-door get-out-the-vote canvass across 40 precincts, randomized to treatment or control and executed through several regional field offices. The aggregate balance table looks fine, so nobody suspects trouble. An independent auditor, brought in as routine, replays the intended allocation and finds that one field office's recorded assignments diverge sharply from the randomization list: its treatment/control split is 70/30 against an intended 50/50, several precincts have enrollment timestamps out of the sequence the algorithm would have generated, and there are override flags on exactly the high-turnout precincts. The pattern points to a coordinator who reassigned "easy" precincts to treatment after seeing the list — a broken concealment. The audit routes the finding to investigation, and the compromised office's data is flagged and excluded before any effect is estimated.
How it works¶
- Reconstruct the intended allocation from the randomization seed, block scheme, and enrollment log, independent of outcomes.
- Compare recorded vs. intended assignment row by row, and test the observed allocation ratio against what the procedure should have produced.
- Inspect the operational trail — enrollment order and timestamps, manual override flags, and the keys linking assignment to baseline records — for evidence of bypass, misconfiguration, or broken joins.
- Route findings to investigation, targeted exclusion, redesign, or an explicit limitation, done by a reviewer outside the analysis team so the audit is not marking its own homework.
Tuning parameters¶
- Audit depth — allocation-ratio spot-check vs. full log replay reconstructing every assignment. Deeper audits catch subtle subversion but cost time and access.
- Independence level — internal reviewer vs. fully external auditor. More independence raises credibility and cost.
- Trigger — routine on every study vs. suspicion-triggered when imbalance or anomalies surface. Routine audits catch silent failures; triggered audits conserve effort.
- Allocation-ratio alarm threshold — how far the observed split may drift from intended before it flags. Tight thresholds catch small tampering but fire on normal chance drift.
When it helps, and when it misleads¶
Its strength is that it catches the one failure the covariate tables structurally cannot see: the allocation was never what it claimed to be. A misconfigured assignment service, a coordinator overriding the list, or a broken linkage key produces bad science that looks perfectly balanced on the surface, and only an audit of the mechanism surfaces it.
Its failure mode is confusing chance with subversion. A realized allocation will differ from the intended split by ordinary sampling variation[1], and treating every such drift as evidence of tampering is a false alarm that wastes goodwill and can itself invite improper rerandomization. The mirror error is false reassurance: a clean allocation ratio does not prove that concealment held or that no override occurred — it only fails to find one. The guarding discipline is to distinguish chance imbalance from mechanism failure using the intended distribution as the yardstick, and to reserve the audit's heavy machinery for anomalies the process itself, not the covariates, made suspicious.
How it implements the components¶
assignment_map_integrity_link— its core function: verify that the recorded assignment matches the intended randomization map and links cleanly to the baseline records.imbalance_response_pathway— a confirmed assignment failure is routed to investigation, targeted exclusion, redesign, or an explicit limitation of claims.independent_balance_reviewer— the audit is conducted by someone outside the analysis team so its findings carry weight.
It computes no covariate balance metric — that is Standardized Mean Difference Table — and it does not check balance within strata or clusters, which is Stratified Balance Check.
Related¶
- Instantiates: Baseline Covariate Balance Verification — it verifies the assignment process behind the realized allocation, the failure the covariate checks cannot see.
- Sibling mechanisms: Standardized Mean Difference Table · Baseline Characteristics Table · Covariate Balance Plot · Stratified Balance Check · Automated A/B Balance Dashboard · Prespecified Adjusted Estimation Plan · Balance Exception Report
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Randomization Integrity Audit operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it a forensic check that the assignment actually recorded in the data matches the intended randomization — right allocation ratio, right sequence, no overrides or broken linkage.
Independent corroboration: The frozen evidence defines Randomization Integrity Audit as 'A forensic check that the assignment actually recorded in the data matches the intended randomization — right allocation ratio, right sequence, no overrides or broken linkage', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Specialized
Rationale: Checking realized treatment allocation against the planned random sequence belongs to experimental-design integrity practice.
Related originating lineages:
- Accounting & Auditing — Reconciliation, override tracing, and forensic evidence review supplied the audit form.
- Medicine & Healthcare — Clinical trial operations materially shaped allocation concealment and override checks.
Review resolution: Both blind reviewers agree on statistics_experimental_design as the primary origin. Explicit reconciliation resolves alternate_origin_disagreement. The merged alternate lineages retain only domains the reviewers identified as materially formative; domain_reach=specialized records later applicability separately from origin breadth.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
Chance imbalance and mechanism failure demand opposite responses: chance imbalance is handled analytically (adjust, stratify, caveat), while a broken mechanism means the affected data cannot be trusted at all and must be excluded or the study rerun. Keeping this audit distinct from the covariate-metric mechanisms is what keeps those two responses from being confused.
References¶
[1] Senn, S. “Testing for Baseline Balance in Clinical Trials.” Statistics in Medicine 13, no. 17 (1994): 1715–1726. Explains why baseline differences can arise under proper randomization. registry ↩