Skip to content

Classification Impact Review

A review — instantiates Social Reality Construction Audit

Traces what a category does once it is assigned — the access, standing, resources, scrutiny, and accountability that ride on the label — and how those effects land differently on differently placed people.

A Classification Impact Review starts from an existing category — often one handed over by a Category Audit — and follows its downstream consequences: who gets in or out, who moves up or down in standing, what resources and burdens attach to the label, and how the same category helps some people while trapping others. Its defining lens is consequences, not definition mechanics: it asks what the label does and to whom, disaggregating the effect across differently positioned people rather than reporting an average.

Example

A child-welfare agency assigns a "family risk score," and a score above a cutoff makes a family "high-risk," routing it to intensive supervision. The review traces the effects. High-risk families get more home visits — support, but also more surveillance and more occasions for a removal-triggering observation. The label follows the family across agencies. Caseworkers read the score before ever meeting the family, coloring interpretation. And being scored high can raise future scores: more contact means more recorded incidents. The review also disaggregates — for one family the label unlocks services; for another it becomes a self-perpetuating mark. This is Hacking's looping effect: a classification changes the people classified, who then come to fit it more closely.[1]

How it works

The distinguishing discipline is to follow the label's consequences and disaggregate by who is affected:

  • Take the category as given — the definition and assignment are upstream; the review starts after the label lands.
  • Enumerate the decisions the label gates — access, resources, scrutiny, accountability, interpretation.
  • Map boundary and status effects — who is admitted, excluded, elevated, or marked.
  • Disaggregate by position — trace how differently placed people experience the same label, and check for looping over time.

It does not audit how the category is defined or assigned; those are the upstream questions.

Tuning parameters

  • Effect horizon — immediate gating versus long-run looping, where the label reshapes the future data recorded about a person. A longer horizon catches self-perpetuation but is harder to evidence.
  • Disaggregation depth — treat all labelees alike versus slice by position or group. Finer slicing surfaces differential harm; coarse slicing hides it.
  • Effect types in scope — access only, or standing, interpretation, and accountability too. A wider net finds the subtle burdens.
  • Counterfactual — whether to compare labeled cases against otherwise-similar unlabeled ones to isolate the label's own effect.

When it helps, and when it misleads

Its strength is separating the person from the label's consequences and exposing looping effects a static audit misses, turning "this category is harmful, or helpful, and to whom" into something concrete. Its failure mode is reviewing effects only for the average case and missing the minority the label traps; its classic misuse is running the review to defend a category ("look at the services it unlocks") while ignoring the surveillance or stigma it imposes. The discipline that guards against both is to disaggregate by position and include a labeled-versus-unlabeled counterfactual.

How it implements the components

Classification Impact Review realizes the consequences side of the archetype — what the label produces, not how it is defined:

  • boundary_and_status_effect_map — its core: who the label admits or excludes and how it moves people's standing.
  • affected_party_meaning_map — traces, analytically, how differently positioned people experience, benefit from, or are trapped by the same label.

It does not define or assign the category (constructed_category_or_practice → Category Audit) or propose other ways to construe the situation (alternative_frame_set → Alternative Frame Workshop).

Notes

Its affected_party_meaning_map is inferred analytically from how the label operates. Stakeholder Meaning Elicitation fills the same component by gathering meanings firsthand from participants, and Culture Survey by measuring them at scale — the three are complementary, not redundant.

References

[1] Ian Hacking's looping effect of human kinds — classifications of people interact with the people classified, who change in response, so the category and its members shift each other over time. Named because that feedback is exactly what a static category audit cannot see. ↩