Source Diet Audit¶
Test or assessment — instantiates Epistemic Boundary Permeability Design
Reviews what sources, channels, authorities, and evidence classes actually enter the environment over a defined period.
A community can argue for hours about whether it is open-minded and never look at the one thing that would settle it: what it actually consumed. Source Diet Audit is a retrospective, sampled census of intake — over a fixed window it catalogues every input a bounded environment took in, coded by source, channel, authority, and evidence class, and counts the concentrations. Its defining move is that it measures the diet rather than judging any single meal: it does not rate whether a given item was fair or true, and it does not fix anything. It produces a factual portrait of the information a group has been eating, so that "we hear all sides" can be checked against the tally instead of asserted.
Example¶
A metro newsroom suspects its climate desk leans on the same few voices. Over an eight-week window an editor pulls every attributed quote and citation in the section and tags each one: source type (wire agency, named scientist, advocacy group, industry press office, reader), institutional affiliation, and whether it carried primary data or secondary commentary. The tally is unflattering — roughly 70% of attributions trace back to three press-release pipelines, affected residents and primary datasets almost never appear, and two think tanks supply about a third of everything labelled "expert."
The output is not a verdict on any story. It is a census: the diet is narrow, PR-fed, and heavy on secondary commentary. That portrait is what tells the desk which channels are missing and worth widening — and it can be re-run next quarter to see whether anything actually shifted.
How it works¶
- Fix the boundary and the window. Name the environment being audited and the closed period sampled — a section, a team, a feed, one quarter.
- Sample or census the intake. Pull the items that actually entered; either every one or a defensible sample.
- Code each item. Tag by source, authority, channel, and evidence class against a scheme set before looking.
- Tally concentration and absence. Compute top-source share, primary-versus-secondary ratio, and an explicit list of evidence classes that never appeared.
The whole method is descriptive and time-boxed. It is a snapshot of a past window, replicable on a cadence so trends become visible — not a live instrument and not an intervention.
Tuning parameters¶
- Window length — a short window is cheap but noisy; a long one smooths but blurs recent change. Match it to how fast the environment turns over.
- Sample vs. census — a full census is authoritative but costly; sampling is faster but can be gamed by an unlucky or cherry-picked draw.
- Coding granularity — coarse buckets are quick and comparable; fine ones surface hidden monoculture but invite mis-coding and false precision.
- Concentration threshold — the top-source share (or Herfindahl-style measure) at which you flag "monoculture." Set it before the count to avoid motivated goalposts.
- Re-audit cadence — one-shot diagnosis, or a standing quarterly re-run that turns the audit into a trend line.
When it helps, and when it misleads¶
Its strength is that it drags an invisible pattern into daylight: source monoculture is nearly impossible to notice from inside the daily flow, and a tally makes it undeniable. It gives every downstream intervention — rotation, bridging, ranking changes — a factual baseline to aim at.
Its central weakness is that an audit measures only what it counts, so it inherits every flaw in its coding scheme and window. Mis-drawn categories or a convenient window can flatter or damn a diet at will, and the count says nothing about the quality of what entered. The classic misuse is treating a diverse-looking tally as proof of healthy reasoning when the nominally varied sources all share the same prior — a diverse masthead over a single worldview is a selection bias[n1] the count won't catch. The discipline that guards against this is to fix the coding scheme and window before looking, carry an explicit "what never appeared" list alongside the "what did," and treat the audit as a baseline to act on, not a grade to celebrate.
How it implements the components¶
Source Diet Audit fills the diagnostic-baseline side of the archetype — the components that describe the intake as it is:
belief_environment_boundary— the first step names the exact unit and window under audit, making the object of diagnosis concrete.input_filter_map— the coded tally is the map of what enters and what is kept out, rendered from real intake rather than from rules on paper.reinforcement_gradient_model— concentration and repetition counts show which sources and claims get amplified through sheer recurrence.
It stops at the snapshot. The always-on rendered instrument — filter_transparency_surface and its live boundary_health_metric — belongs to Filter Transparency Dashboard, the audit's nearest twin: the audit samples a closed past window and issues a report, while the dashboard renders the filter continuously.
Related¶
- Instantiates: Epistemic Boundary Permeability Design — the audit supplies the intake baseline the rest of the design depends on.
- Sibling mechanisms: Filter Transparency Dashboard · Cross-Cutting Source Rotation · Recommendation Diversity Constraint · Rival Hypothesis Red Team · Bridge Panel or Boundary-Spanner Session · Steelman Counterposition Brief · Caricature Detection Review · Counterevidence Precommitment Question · Belief-Update After-Action Review
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Source Diet Audit operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it reviews what sources, channels, authorities, and evidence classes actually enter the environment over a defined period.
Independent corroboration: The frozen evidence defines Source Diet Audit as 'Reviews what sources, channels, authorities, and evidence classes actually enter the environment over a defined period', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Communication & Media Studies
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Universal
Rationale: Reviewing the distribution of channels, authorities, and evidence classes entering an information environment is a media-exposure and framing audit. Entman grounds selection and salience, while ICD 203 requires attention to the quality and breadth of the source base.
Related originating lineages:
- Cultural Studies — Authority and canon shape which voices enter an environment.
- History & Historiography — history_historiography contributes source criticism, provenance, documentary context, and corroboration to this mechanism's defining operation—Reviews what sources, channels, authorities, and evidence classes actually enter the environment over a defined period—without displacing the selected primary historical lineage.
- Library & Information Science — Collection diversity and source coverage can be systematically inventoried.
- Organizational & Management Science — Organizational design, management, and operational governance supplies a parallel or contributing lineage for the mechanism's defining operation: reviews what sources, channels, authorities, and evidence classes actually enter the environment over a defined period.
- Security Studies & Intelligence Analysis — security_intelligence contributes security engineering, threat analysis, and intelligence practice to this mechanism's defining operation—Reviews what sources, channels, authorities, and evidence classes actually enter the environment over a defined period—without displacing the selected primary historical lineage.
- Sociology & Anthropology — sociology_anthropology contributes sociology and anthropological study of institutions and social relations to this mechanism's defining operation—Reviews what sources, channels, authorities, and evidence classes actually enter the environment over a defined period—without displacing the selected primary historical lineage.
Review resolution: The blind reviewers disagree on primary lineage (communication_media_studies versus history_historiography). Authoritative or primary research supports communication_media_studies as the best historical origin: Reviewing the distribution of channels, authorities, and evidence classes entering an information environment is a media-exposure and framing audit. Entman grounds selection and salience, while ICD 203 requires attention to the quality and breadth of the source base. The cited Entman, Framing: Selection and Salience; ODNI, Intelligence Community Directive 203: All-Source Analytic Standards directly supports the mechanism's defining operation. All independently supported contributing domains are retained without an arbitrary cap. origin_mode=cross_disciplinary_synthesis records lineage, while domain_reach=universal records later applicability separately from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- Entman, Framing: Selection and Salience
- ODNI, Intelligence Community Directive 203: All-Source Analytic Standards
Notes¶
[n1] Selection bias — the systematic distortion that arises when the cases that enter a sample are not representative of the population they are taken to represent. A source diet that looks varied can still be selection-biased if every source was drawn from the same corner of the space, which is why the audit tracks concentration and absence, not just count. ↩