Gender Shades

Buolamwini, J., & Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. Proceedings of Machine Learning Research, 81, 77-91.

Type

Conference paper

Intellectual base

Primary research

Year

2018

Volume

81

Pages

77-91

Link

https://proceedings.mlr.press/v81/buolamwini18a.html

Cited from

economics_finance, ethnography_qualitative_methods, philosophy, psychology

Cited by

6 citations across 6 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Distributional Effects
  • The concealment invariant is decisive and quantifiable: a model with 95% overall accuracy can post 99% on a majority group comprising most of the data and 60% on a small minority group, and the headline number conceals the gap entirely because the majority's weight dominates the average.

    This sourceDemonstrates how high overall classifier accuracy conceals large error gaps on under-represented subgroups.

• Epistemic Justice
  • In AI and machine-learning fairness

    This sourceAudit of three commercial gender-classification systems showing error rates up to 34.7% for darker-skinned women vs. 0.8% for lighter-skinned men, demonstrating that algorithmic systems systematically under-perform for underrepresented groups.

• Negative Case Analysis
  • When the discordant cases cluster — say the model fails systematically on dark-skinned faces or on a rare dialect — that clustering is boundary inference: it localizes where the account is mis-drawn, and distinguishes a mechanism problem (biased training data) from a mere scope problem (a few odd inputs).

    This sourceSlice analysis showing classifier error clusters on darker-skinned female faces (up to 34.7% vs 0.8% for lighter-skinned males) — clustered discordant cases localizing a mechanism (unbalanced training data) problem rather than a scope problem.

• Social Identity Theory
  • Buolamwini and Gebru's (2018) Gender Shades audit makes the further structural point that algorithmic categorization can itself encode and amplify group-defining errors, treating intersectional sub-populations as out-of-distribution and feeding back differential performance into identity-salient downstream uses.

    This sourceAudit of commercial gender-classification systems demonstrating that algorithmic categorization can encode and amplify intersectional group-defining errors, treating sub-populations as out-of-distribution and producing identity-salient downstream feedback.

Domain-specific

• Distributional Blind Spot
  • The Gender Shades study (Joy Buolamwini and Timnit Gebru, 2018) is the defining demonstration

    This sourceThe Gender Shades benchmark, balanced by gender and skin type, on which three commercial gender-classification systems misclassified darker-skinned females at up to 34.7% while the maximum error for lighter-skinned males was 0.8%.

    Supported in partVerified against a saved copy of the source

    “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification”

    Read by an automated reader; not a human review. How support was checked

Mechanisms

• Example, Counterexample, and Decision-Task Test
  • Its classic trap is testing only typical cases and reporting a high pass rate, so a systematic failure on a rare protected exception is averaged away — surface fluency confirmed, the dangerous edge never probed.

    This sourceShows that aggregate accuracy can conceal systematic failure for a less represented protected subgroup.

Verification

Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.

Does it back the claim? Read against the text for 1 of 6 citations: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.

Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.

See how references were verified.

Registry ID ref:3592f8677afd · see in the full table