Gender Shades

Buolamwini, J., & Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. Proceedings of Machine Learning Research, 81, 77-91.

Type

Conference paper

Intellectual base

Primary research

Year

2018

Link

https://proceedings.mlr.press/v81/buolamwini18a.html

Cited from

economics_finance, ethnography_qualitative_methods, philosophy, psychology

Cited by

5 citations across 5 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Distributional Effects
  • The concealment invariant is decisive and quantifiable: a model with 95% overall accuracy can post 99% on a majority group comprising most of the data and 60% on a small minority group, and the headline number conceals the gap entirely because the majority's weight dominates the average.

    This sourceDemonstrates how high overall classifier accuracy conceals large error gaps on under-represented subgroups.

• Epistemic Justice
  • In AI and machine-learning fairness

    This sourceAudit of three commercial gender-classification systems showing error rates up to 34.7% for darker-skinned women vs. 0.8% for lighter-skinned men, demonstrating that algorithmic systems systematically under-perform for underrepresented groups.

• Negative Case Analysis
  • When the discordant cases cluster — say the model fails systematically on dark-skinned faces or on a rare dialect — that clustering is boundary inference: it localizes where the account is mis-drawn, and distinguishes a mechanism problem (biased training data) from a mere scope problem (a few odd inputs).

    This sourceSlice analysis showing classifier error clusters on darker-skinned female faces (up to 34.7% vs 0.8% for lighter-skinned males) — clustered discordant cases localizing a mechanism (unbalanced training data) problem rather than a scope problem.

• Social Identity Theory
  • Buolamwini and Gebru's (2018) Gender Shades audit makes the further structural point that algorithmic categorization can itself encode and amplify group-defining errors, treating intersectional sub-populations as out-of-distribution and feeding back differential performance into identity-salient downstream uses.

    This sourceAudit of commercial gender-classification systems demonstrating that algorithmic categorization can encode and amplify intersectional group-defining errors, treating sub-populations as out-of-distribution and producing identity-salient downstream feedback.

Mechanisms

• Example, Counterexample, and Decision-Task Test
  • Its classic trap is testing only typical cases and reporting a high pass rate, so a systematic failure on a rare protected exception is averaged away — surface fluency confirmed, the dangerous edge never probed.

    This sourceShows that aggregate accuracy can conceal systematic failure for a less represented protected subgroup.

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Registry ID ref:3592f8677afd · see in the full table