Fairness Audit by Stratum¶
Metric / dashboard — instantiates Stratified Treatment
Checks whether differential treatment is producing intended fit without unacceptable disparate harm, exclusion, or hidden under-service.
Fairness Audit by Stratum is the recurring measurement instrument that watches a stratified system to see whether its deliberate differences in treatment are still fit rather than harm — whether any stratum is being quietly under-served, excluded, or pushed below the floor. Its defining feature is that it never sets treatment; it only measures. It reads the outputs of whatever mechanisms differentiate care and tests them against an explicit standard of what equivalent outcomes should look like. Because the archetype treats unlike cases differently on purpose, this audit is the counter-check that keeps "differently" from sliding into "unfairly."
Example¶
A bank runs a tiered credit-decisioning system: applicants are sorted into risk bands that receive different interest rates and credit limits. On paper it is calibrated and profitable. The fairness audit slices the outcomes a second way. Every month it breaks approval rate, price, realized default, complaint rate, and servicing outcomes down both by risk band and by legally protected group, and compares each slice against two benchmarks: an outcome-equivalence standard (applicants of equal realized repayment should receive equal terms) and a floor (no group's rate of approval-with-support falls below a defined baseline).
The dashboard flags that borderline applicants clustered in a few postal codes are being priced as though higher-risk than their observed defaults justify — a local miscalibration producing disparate harm that the aggregate profit number hid completely. The audit does not repair the pricing; it raises the alarm and routes it to the policy owners. Its job is to make the subgroup failure visible before it becomes a pattern. The figures are illustrative; the move — re-slicing outputs against a pre-set standard — is the mechanism.
How it works¶
- Pre-register the standard. Decide what equivalent outcomes should look like before seeing the results, so the benchmark can't be chosen to flatter the system.
- Slice two ways at once. Cut outcomes by stratum and by protected attribute; harm often hides in the interaction, not either margin.
- Compare to standard and floor. Test each slice against the equivalence benchmark and against the minimum every stratum is guaranteed.
- Alarm and escalate. Threshold breaches raise a flag to the owners of the treatment policy — the audit hands off, it does not fix.
Tuning parameters¶
- Outcome metrics — auditing many outcomes catches more harm but multiplies false alarms and analyst load; a narrow set is cheap but blind spots grow.
- Equivalence-standard strictness — a tight standard catches subtle disparity but flags noise; a loose one is quiet but launders real harm.
- Slice granularity — fine slices localize harm but shrink sample sizes into unreliability; coarse slices are stable but hide pockets.
- Alarm threshold — a low bar surfaces problems early at the cost of alarm fatigue; a high bar is calm but slow.
- Audit cadence — frequent audits catch drift fast but cost standing effort; occasional audits are lean but let harm accumulate between runs.
When it helps, and when it misleads¶
Its strength is exactly the archetype's blind spot: subgroup failure hidden by aggregate success. An average can look healthy while one stratum quietly rots, and this is the instrument that pulls that stratum out of the average and shows it plainly, measured against a floor no one is allowed to fall below.
Its deep limitation is that you cannot satisfy every fairness criterion at once — several reasonable statistical definitions of fairness are mutually incompatible whenever base rates differ, so an audit necessarily commits to some criteria and is silent on others. That opens the classic misuse: fairness theater, in which the audit reports only the metric the system happens to pass and calls the differentiation vindicated. A concrete guard is a bright-line screen like the four-fifths rule — treat any group's selection rate below roughly four-fifths of the top group's as presumptive adverse impact worth investigating — used as a tripwire, not a hall pass.[1] The discipline that keeps it honest is to fix the standard in advance and report the metrics it fails alongside the ones it passes.
How it implements the components¶
fairness_policy— it encodes the legitimate reasons a difference is allowed and the distinctions that are prohibited, then holds the system to them.monitoring_feedback— it is the recurring slice-compare-alarm loop that asks whether the strata still work and whether the assignment rule has drifted.outcome_equivalence_standard— it defines and tests the explicit benchmark of what equivalent outcomes across strata should be, so "fair" is a measured claim, not a feeling.minimum_service_floor— it checks that no stratum has fallen below the defensible baseline, catching abandonment of low-intensity strata.
It measures; it does not treat. It does not set the treatment_policy or the resource_or_threshold_differentiation whose outputs it audits — those come from Segmented Customer Treatment Rules and Stratum-Specific Threshold Schedule; nor does it own the assignment_rule that placed the cases — that is Risk Stratification Protocol.
Related¶
- Instantiates: Stratified Treatment — Fairness Audit by Stratum is the monitoring instrument that keeps deliberate differentiation from becoming opaque discrimination or silent under-service.
- Consumes: Segmented Customer Treatment Rules and Stratum-Specific Threshold Schedule — it reads the treatment and threshold outputs these produce and tests them against its standard.
- Sibling mechanisms: Case Management Tiers · Clinical Risk Banding · Risk Stratification Protocol · Tiered Service Catalog · Vulnerability-Based Support Workflow
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: The audit evaluates sliced outcome evidence against preregistered equivalence standards and protected floors and produces disparate-harm or under-service findings.
Nearest alternative: Monitoring, Sensing & Alerting — Threshold breaches can recur, but the audit is a bounded evaluative pass that hands findings to policy owners rather than continuous observation.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Ethics of Technology & AI Governance
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Auditing differential algorithmic or system outcomes across protected strata is canonical fairness and accountability practice.
Related originating lineages:
- Law & Governance — Disparate-impact and equal-protection doctrine materially shape unacceptable-harm and exclusion criteria.
- Statistics & Experimental Design — Stratified estimation and disparity testing materially supply the quantitative comparison. Stratified estimation and uncertainty assessment materially shape subgroup comparisons.
Review resolution: Both reviewers agree that tech_ethics_ai_governance is primary. I retain statistics_experimental_design, law_governance only as formative origin lineages; cross_disciplinary_synthesis is appropriate because the final form materially combines the agreed primary with the retained formative lineages. Reach is multi_domain because the structure transfers across several fields but is not a near-universal human pattern, an applicability judgment kept separate from provenance. Encyclopedia synthesis is true because the exact generalized artifact is an encyclopedia-authored combination or refinement. No unresolved historical ambiguity remains after reconciling the secondary fields.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] U.S. Equal Employment Opportunity Commission, U.S. Civil Service Commission, U.S. Department of Labor, and U.S. Department of Justice. Uniform Guidelines on Employee Selection Procedures. 29 CFR Part 1607 (1978). Uses the four-fifths ratio as a practical screening rule for adverse impact while requiring consideration of other relevant evidence. registry ↩