Observation Bias & Annotation Reliability¶

← Back to Domain-Specific Families

Abstractions about retrospective notes, salience and recency effects, perceived scrutiny, familiarity, and agreement among independent annotators.

6 abstractions in this family — domain-specific abstractions that sit near one another in structural-signature space (k-means over structural-signature embeddings). Each is shown with its short description.

Fieldnote Backfill — The validity threat by which a delayed field write-up silently fuses direct observation, inference, and gap-filling reconstruction into one undifferentiated register, collapsing the attribution that tells the strongest evidence from the weakest.
Frequency Illusion — Explain the felt surge in a thing's frequency right after you first notice it as a change in the perceiver, not the world — selective attention now logs real occurrences while confirmation-biased memory counts hits and discards misses.
Inter-Annotator Agreement — Measure whether independent raters applying the same coding scheme to the same items converge, using a statistic that subtracts the agreement expected by chance from the marginal label distribution.
Recency Effect — The empirical finding that the last items in a temporal sequence are recalled or weighted more heavily than middle items, driven by a short-term-store component and a temporal-distinctiveness component that a filled delay can separate.
Spotlight Effect — Explain the felt certainty that everyone noticed as an estimation error: the actor anchors on how salient a self-feature is from the inside and under-corrects toward observers' far lower base-rate attention, so predicted notice runs about double the actual.
Well-Travelled Road Effect — People underestimate the duration of familiar routes and overestimate equally long unfamiliar ones, because retrospective duration is reconstructed from the count of distinct, attention-demanding event traces memory retained — few for automatic familiar travel, many for novel decision-dense travel.