Skip to content

Filter Independence Check

Diagnostic — instantiates Structural Filter Intersection Audit

Tests whether a system's parallel filters are actually independent, or whether shared owners, inputs, or criteria make several of them pass and fail together.

The Filter Independence Check targets a specific illusion: a stack that looks like many independent gates may really be a few. Filters that share an owner, a data source, a training set, or a criterion tend to reject the same items, so the "five layers of review" collapse into two effective judgments — and the surviving set is far narrower and more uniform than the count implies. The check's whole product is a dependency graph of the filters: which ones are coupled, by what, and how strongly their outcomes move together. It does not judge any filter's quality; it maps the wiring between them.

Example

A platform advertises "five independent layers" screening user posts. The Independence Check traces each layer's inputs and finds that three of the automated classifiers were fine-tuned on the same labeled dataset, and the human reviewers are shown the classifier scores before they decide — so their judgment anchors to the machines'. Running a sample of posts through all five and watching which reject the same items confirms it: a post that trips one layer trips nearly all of them. The five "independent" layers are really about two independent judgments, and a single labeling bias in the shared dataset propagates through the entire stack. The output is a graph — nodes are filters, edges are shared-data / shared-owner / feeds-into links — annotated with how tightly each pair co-rejects.

How it works

Instead of evaluating each filter on its merits, the check looks only at the relationships between filters. It records structural links — common owner, common input data, one filter consuming another's output — and measures behavioral coupling by running a shared sample through every filter and computing how often they reject the same items. High co-rejection plus a traceable common cause means the filters are dependent, and the effective number of filters is the number of uncorrelated judgments, not the nominal count.

Tuning parameters

  • Dependency-type scope — count only structural links (shared owner/data), or also behavioral correlation (they happen to reject the same items). Behavioral catches more but confounds cause with coincidence.
  • Correlation threshold — how much co-rejection counts as "not independent." Strict flags subtle coupling; loose catches only the blatant.
  • Serial vs. shared-cause weighting — how heavily to weight "filter B sees filter A's output" (anchoring) against "A and C draw on the same source." Different remedies follow from each.
  • Co-rejection sample size — how many items to push through all filters to observe joint behavior; larger samples stabilize the correlation estimate.

When it helps, and when it misleads

Its strength is puncturing the "defense in depth" story and reporting the number that actually matters — the effective count of independent filters, which governs how uniform the surviving set will be. Its failure mode is that it diagnoses coupling but not whether coupling is bad: sometimes filters should share a standard, and the check read alone can argue for spurious diversity. The classic misuse is running it backwards — picking a loose threshold to certify a stack as "diverse enough." The discipline that keeps it honest is to pair it with the intervention that actually reduces coupling and re-run afterward, rather than treating a single diagnosis as a verdict.[1]

How it implements the components

  • filter_dependency_graph — the whole output: filters as nodes; shared-owner, shared-input, and feeds-into edges; each annotated with observed outcome-correlation.

It does NOT enumerate the filters in the first place (parallel_filter_inventory — Filter Stack Map), enact the fix (independence_and_diversity_controls — Filter Rotation or External Challenge), or model what survives (surviving_intersection_model — Intersection Matrix). It supplies the coupling picture those act on.

Notes

Independence here is a property of the filters, not the producers. Whether the people upstream self-censor in correlated ways — all anticipating the same disapproval — is a different coupling, and it belongs to the Producer Pressure Survey. Confusing the two leads to "fixing" filter diversity while a shared producer-side chilling effect keeps the output uniform anyway.

References

[1] The Condorcet Jury Theorem — that a majority of many imperfect but independent judges trends toward the right answer — holds only while the judgments stay independent. Once they share a common bias, adding more judges (or filters) stops adding safety. It is the formal reason a stack's effective count, not its nominal count, is what matters. ↩