'Everyone Wants to Do the Model Work, Not the Data Work'¶
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021). 'Everyone Wants to Do the Model Work, Not the Data Work': Data Cascades in High-Stakes AI. Proceedings of CHI 2021.
Cited by¶
2 citations across 2 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Elicitation Channel Contribution
- Data labelling: the rubric is the channel — the same image under different rubrics yields systematically different labels, and a model trained on the labels learns channel plus signal jointly.
This sourceShows that annotation rubrics systematically shape labels, so a model trained on them learns channel-plus-signal jointly.
- Data labelling: the rubric is the channel — the same image under different rubrics yields systematically different labels, and a model trained on the labels learns channel plus signal jointly.
- Evidence-Fidelity Decay
- in code review written hours after reading a diff, in medical chart documentation written at end of shift, in diplomatic cables and meeting minutes drafted hours after a meeting, and in machine-learning data labelling under time pressure, where labels produced after batch fatigue blend observation of the artefact with reconstruction from the labeller's developing schema.
This sourceDocuments how under-valued, time-pressured data/labeling work produces compounding downstream 'data cascades,' so label sets pass surface checks while encoding annotators' developing schemas rather than the artifact.
- in code review written hours after reading a diff, in medical chart documentation written at end of shift, in diplomatic cables and meeting minutes drafted hours after a meeting, and in machine-learning data labelling under time pressure, where labels produced after batch fatigue blend observation of the artefact with reconstruction from the labeller's developing schema.
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:f9b926cbf4ce · see in the full table