Source Variance Audit Matrix¶
Audit matrix — instantiates Shared-Source Variance Isolation
Lays the output dimensions against every shared source in a grid so that each place a common source touches more than one dimension is written down before any correlation is trusted.
A Source Variance Audit Matrix is the first, purely bookkeeping move of the archetype: a grid whose rows are the output dimensions you intend to read as separate evidence and whose columns are every source — rater, instrument, batch, session, pipeline, time window — that can touch more than one of them at once, with each cell marking whether that source is shared. Its single defining idea is that leakage cannot be reasoned about until shared exposure is written down as data; the matrix makes provenance visible without yet estimating, modeling, or removing anything. It converts "these four metrics agree" into "these four metrics all pass through one logging pipeline, and only one of them has an independent collection path."
Example¶
A payments startup's growth dashboard shows four supposedly independent health metrics — activation rate, 7-day retention, transactions per user, and referral rate — all trending up for a quarter. Before anyone celebrates, an analyst builds a Source Variance Audit Matrix. Rows: the four metrics. Columns: the shared sources she can name — the client-side event SDK, the nightly ETL job, the definition of "active user," the experiment-assignment service, and the reporting time zone. Filling cells, a pattern jumps out: all four metrics are computed downstream of the same event SDK, which shipped a new version the week the trend began; three of the four also share the "active user" definition that was quietly widened. Only referral rate has an independent server-side source. The output is not a verdict but a map: one column — the SDK — lights up across every row, flagging the correlation as suspect and telling the team exactly which pipeline to interrogate first.
How it works¶
Rows come straight from the claim frame: the dimensions whose independence a decision rests on. Columns are the shared-source inventory. Each cell records presence — and optionally strength or direction — of shared exposure, often color-graded. Then the grid is read two ways: down each column, where a source touching many rows is a leakage suspect, and across each row, where a dimension whose only filled cells are shared columns has no independent evidence path left. Each nonempty cell can be annotated with the plausible route by which the source reaches that dimension, so the completed grid doubles as a coarse pathway map. The mechanism stops at documentation; it hands the lit-up columns to the design and modeling siblings rather than adjudicating anything itself.
Tuning parameters¶
- Cell resolution — binary "shared / not," a graded exposure strength, or a signed direction. Finer cells surface subtle leakage but invite false precision and take longer to fill.
- Source granularity — one lumped "pipeline" column versus decomposed SDK / ETL / definition columns. Splitting isolates the true culprit but multiplies bookkeeping.
- Dimension scope — audit only the dimensions a claim rests on, or the whole metric set. Wider scope catches surprises; narrow scope keeps the grid legible.
- Evidence bar for a filled cell — mark on suspicion versus only on confirmed shared provenance. A low bar over-flags; a high bar misses undocumented sources.
- Refresh trigger — one-shot versus re-audit on every pipeline, instrument, or definition change.
When it helps, and when it misleads¶
Its strength is that it defeats source blindness — the failure of inspecting only outputs and never asking how they were produced. A filled matrix makes dashboard double-counting obvious at a glance and routes effort to the right column instead of a general fog of suspicion.[n1] Its central failure mode is that the grid is only as complete as the inventory behind its columns: an unlisted source is simply an empty region, so a tidy, mostly-blank matrix can radiate false reassurance exactly where the worst leakage hides. The classic misuse is treating that clean grid as proof of independence rather than as a record of what was checked. It also marks exposure, not effect — a shared column is a hypothesis, not a correction. The guarding discipline is to keep the matrix a living register with an explicit "sources not yet ruled out" note, and to hand every lit column onward to a design or modeling mechanism rather than declaring leakage from the grid alone.
How it implements the components¶
dimension_claim_frame— the rows are exactly the dimensions the frame declares should stand as separate evidence; the audit only exists because that independence claim is being made.shared_source_inventory— the columns are the inventory, forced out of a vague worry into an enumerated, checkable list.source_dimension_pathway_map— each filled cell records a plausible route from a source into a dimension, so the finished grid is a coarse pathway map.
It records exposure but never acts on it: it does not rotate or separate sources (source_separation_design) — that's [Batch, Rater, or Instrument Counterbalancing Protocol] — nor estimate or remove the shared component (common_variance_adjustment_rule, precision_weight_update) — that's [Common Factor or Random-Effect Model].
Related¶
- Instantiates: Shared-Source Variance Isolation — it supplies the provenance map the rest of the workflow depends on.
- Sibling mechanisms: Multitrait-Multimethod Matrix · Common Factor or Random-Effect Model · Residual Correlation Diagnostic · Negative-Control Outcome Probe · Batch, Rater, or Instrument Counterbalancing Protocol · Variance Partitioning Report · Leakage Sensitivity Grid
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Source Variance Audit Matrix operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it lays the output dimensions against every shared source in a grid so that each place a common source touches more than one dimension is written down before any correlation is trusted.
Independent corroboration: The frozen evidence defines Source Variance Audit Matrix as 'Lays the output dimensions against every shared source in a grid so that each place a common source touches more than one dimension is written down before any correlation is trusted', so its operative form is Assessment, Review & Assurance.
Nearest alternative: Representation, Specification & Plan — Source Variance Audit Matrix includes features of a static representation, map, specification, schema, or prospective plan that externalizes information, but its defining operation is a bounded evaluation of existing evidence or work that produces a finding or disposition.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Universal
Rationale: Marking shared inputs across output dimensions before interpreting correlation is a statistical common-source and dependence audit. GAO reliability guidance requires tracing source, transformations, and error risk; statistics supplies covariance interpretation.
Related originating lineages:
- Accounting & Auditing — A matrix makes dependency claims traceable for review.
- Data Science & Analytics — Feature lineage identifies outputs that inherit the same upstream variation.
- Economics & Finance — Economics, finance, and mechanism-design practice supplies a parallel or contributing lineage for the mechanism's defining operation: lays the output dimensions against every shared source in a grid so that each place a common source touches more than one dimension is written down before any correlation is trusted.
- Information Theory — information_theory contributes information theory and signal representation to this mechanism's defining operation—Lays the output dimensions against every shared source in a grid so that each place a common source touches more than one dimension is written down before any correlation is trusted—without displacing the selected primary historical lineage.
- Systems Thinking & Cybernetics — Common drivers create correlated downstream behavior without direct coupling.
Review resolution: The blind reviewers disagree on primary lineage (statistics_experimental_design versus accounting_auditing). Authoritative or primary research supports statistics_experimental_design as the best historical origin: Marking shared inputs across output dimensions before interpreting correlation is a statistical common-source and dependence audit. GAO reliability guidance requires tracing source, transformations, and error risk; statistics supplies covariance interpretation. The cited GAO, Case Study Evaluations: Triangulation and Evidence Matrices; GAO, Assessing Data Reliability directly supports the mechanism's defining operation. All independently supported contributing domains are retained without an arbitrary cap. origin_mode=cross_disciplinary_synthesis records lineage, while domain_reach=universal records later applicability separately from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
Notes¶
The matrix is deliberately pre-quantitative and the cheapest mechanism in the family: it can be built in an afternoon from documentation alone, before a single row of data is pulled. That is its point — it often kills a false multidimensional story before an expensive model is ever fit.
[n1] Common-method variance — the portion of observed correlation between measures that comes from their sharing a measurement method or source rather than from the constructs themselves. Naming and recording the shared method is the standard first defense, which is what this matrix operationalizes. ↩