Bias Audit¶
Review protocol — instantiates Bias-Specific Decision Audit
Runs a consequential decision through one structured pass — classify its type, map the few distortion pathways that actually threaten it, deploy only the matching checks, and either revise the process or record the bias risk left standing.
A Bias Audit is the reasoning pass a consequential decision is put through before it closes: name what kind of decision it is, sketch the handful of distortion pathways that plausibly threaten this decision, deploy only the checks that match them, and end by either changing the process or writing down the bias risk that remains. Its defining idea is orchestration, not checking. The audit does not itself mask a name, gather independent numbers, or run a differential — it decides which of those are worth running here, sequences them before commitment, and converts their findings into a revision or a conscious acceptance. Strip out the classification and the map and it collapses into a generic debiasing ritual; keep them and it becomes a targeted, proportionate review that a later reader can see was aimed at real risks rather than at bias in the abstract.
Example¶
A venture fund's investment committee is about to lead a Series B in a company two partners have already spent months courting. Before the vote, the chair runs a bias audit. First the classification: this is a high-conviction, follow-the-thesis investment, irreversible once the wire goes out, made by people with sunk diligence hours and reputational skin in the deal. That type points at three pathways, not twenty — confirmation around the favored thesis, escalation from the diligence already spent, and optimism baked into the growth model. The audit maps those three, then deploys a matching check for each: a disconfirming-evidence pass owned by a partner who was not on the deal, a fresh base-rate look at the company's segment, and a written "what would make us walk" line. It does not bolt on a generic fraud checklist or a governance review — those pathways aren't live here.
The vote still happens, but the memo now carries a one-paragraph residual-risk note: the team accepts the remaining key-person concentration as un-auditable at this stage. The outcome is not a cleaner feeling; it is a decision whose known blind spots were checked and whose un-checkable ones are named.
How it works¶
- Classify, then bound. Name the recurring decision form and its stakes, reversibility, participants, and incentives — because the type is what tells the audit which checks are even relevant.
- Map, don't enumerate. Produce a small set of plausible distortion pathways for this decision, not the wall-poster of all known biases. The map is the audit's load-bearing artifact; if it can't say which pathway is plausible and why, the audit is too generic.
- Deploy matched checks. For each mapped pathway, select the one sibling mechanism that addresses it — a masking procedure, an independent estimate, a forced differential — and run it before the decision hardens.
- Revise or accept. Convert findings into a concrete change (new criterion, reopened alternative, added evidence, changed sequence) or, where further review isn't proportionate, an explicit residual-risk acceptance. An audit that can change nothing is theater.
Tuning parameters¶
- Audit depth — from a two-minute self-run to a formal independent review with sign-off. Deeper audits catch subtler distortion but cost time and can overstandardize decisions that need local judgment; match depth to stakes and reversibility.
- Map cardinality — how many pathways the map may hold. A tight map (two or three) stays actionable; a loose one drifts back toward the all-bias checklist it was meant to replace.
- Revise-vs-accept threshold — how strong a finding must be before it forces a process change rather than a noted risk. Set it low and every audit reopens the decision; set it high and findings die as documentation.
- Auditor independence — whether the person running the audit has a stake in the outcome. Independence sharpens the map but costs a reviewer who understands the decision's substance.
- Trigger scope — which decisions get audited at all. Audit everything and people route around it; audit only the costly-and-irreversible and you keep the review usable.
When it helps, and when it misleads¶
Its strength is fit: because every check is tied to a mapped pathway, the audit spends scrutiny where distortion is actually likely and leaves low-risk decisions alone. It also makes the un-fixable visible — residual risk is named rather than silently carried.
Its failure modes are structural. The worst is family-anchor overreach — the audit swelling to absorb every debiasing concern until it is once again "be less biased," the exact generic advice it exists to replace. Close behind are findings without authority (the audit surfaces a problem but no one can act on it) and the bias blind spot[n1], where auditors reliably detect distortion in others' reasoning while missing their own, so a self-run audit quietly flatters the very decision it reviews. The discipline that keeps it honest is to hold the map small and specific, give the audit real authority to change or stop the decision, and — for high-stakes calls — put the audit in hands that don't own the outcome.
How it implements the components¶
decision_type— the audit opens by classifying the recurring decision form, which is what makes the rest of the review selective rather than universal.bias_vulnerability_map— its core artifact: the small, argued set of distortion pathways plausible for this decision.process_revision— a finding is converted into a concrete change to criteria, evidence flow, sequence, roles, or cadence.residual_risk_acceptance— where further review isn't proportionate, the audit names the bias risk left standing rather than pretending it was removed.
It does not perform the concrete checks themselves (targeted_bias_check — those are Blind or Masked Review, Independent Estimation, and Diagnostic Debiasing Check), nor capture the running record of what was checked (evidence_and_process_trace — that is Structured Review Form).
Related¶
- Instantiates: Bias-Specific Decision Audit — the Bias Audit is the archetype's orchestrating spine, turning decision type into a matched, proportionate review.
- Consumes: the concrete checks it deploys — Blind or Masked Review, Independent Estimation, Diagnostic Debiasing Check — and, for the record it leaves behind, Structured Review Form.
- Sibling mechanisms: Decision Checklist · Structured Review Form · Hiring Review Rubric · Blind or Masked Review · Independent Estimation · Diagnostic Debiasing Check · Decision Log · Reference-Class Forecasting
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Runs a consequential decision through one structured pass — classify its type, map the few distortion pathways that actually threaten it, deploy only the matching checks, and either revise the process or record the bias risk left standing, making its operative form a bounded evaluation of existing evidence or work that produces a finding or disposition.
Independent corroboration: The frozen evidence defines Bias Audit as 'Runs a consequential decision through one structured pass — classify its type, map the few distortion pathways that actually threaten it, deploy only the matching checks, and either revise the process or record the bias risk left standing', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Psychology
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Diagnosing decision-specific distortion pathways and matching debiasing checks draws primarily on judgment-and-decision research.
Related originating lineages:
- Behavioral Economics — Choice and incentive research identifies predictable context-driven deviations from judgment norms.
- Organizational & Management Science — Decision-process governance turns diagnosed distortions into workflow and accountability changes.
- Statistics & Experimental Design — Blinding, controlled comparison, and sampling checks address specific inferential distortions.
Review resolution: Psychology is the agreed primary lineage through empirically studied decision biases. Organizational management supplies process redesign, statistics supplies matching diagnostic checks, and behavioral economics supplies incentive distortions; reach is multi-domain rather than universal.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
The archetype's plurality invariant lives or dies here: the Bias Audit selects narrower patterns — a dissent channel, a premortem, a selection-bias correction — as checks, but must not absorb them when one of them is the primary intervention. If suppressed dissent or planning optimism is the whole problem, the matching archetype should lead and the audit should step back to a supporting role.
[n1] The bias blind spot — Emily Pronin, Daniel Lin, and Lee Ross's finding that people rate themselves as less susceptible to cognitive biases than their peers. It is why an audit run by the same people who made the decision tends to under-detect the distortions that matter, and why high-stakes audits are handed to an independent reviewer. ↩