A Coefficient of Agreement for Nominal Scales

Cohen, J. (1960). A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement, 20(1), 37-46.

Type

Journal article

Intellectual base

Primary research

Year

1960

Volume

20

Issue

1

Pages

37-46

DOI

10.1177/001316446002000104

DOI registrant

SAGE Publications

Link

https://doi.org/10.1177/001316446002000104

Cited by

7 citations across 7 artifacts.

Each citation links to the sentence it supports in the citing article.

Domain-specific

• Cohen's Kappa
  • The original named coefficient was introduced by Cohen in 1960; the formula used here is checked in an independently inspectable research paper, not attributed to uninspected pages of Cohen's original article.

Mechanisms

• Classification Audit
  • The right metric here is inter-rater reliability — the degree to which independent judges applying the same schema reach the same label — because low reliability is the sharpest evidence that the fault is in the schema and not the judges.

    This sourceIntroduces chance-corrected agreement between independent nominal-category judges as an inter-rater reliability measure.

• Competency Signoff
  • Inter-rater reliability is the honest measure of how much that judgment can be trusted.

    This sourceMeasures chance-corrected agreement between independent nominal-category judges.

• Convergence–Divergence Rubric
  • A good rubric behaves like an inter-rater instrument — two people applying it to the same streams should land on the same level.
• Intersubjective Replication Check
  • Reliability is made explicit through an inter-rater or inter-lab statistic rather than assumed.

    This sourceIntroduces kappa as a statistic for the reproducibility, or reliability, of judgments by two raters on nominal categories.

• Measurement System Analysis
  • The apt anchor is inter-rater reliability, formalized by statistics like Cohen's kappa that measure agreement beyond chance.

    This sourceIntroduces Cohen’s kappa as an inter-rater reliability statistic that measures nominal agreement beyond chance.

• Sparse Tagging Taxonomy
  • Its failure mode is inconsistency between taggers: the same item gets different tags from different people, so the code means different things depending on who applied it — the inter-rater reliability problem that dogs all human coding.

    This sourceFrames differing category assignments to the same items as an inter-rater agreement problem and provides chance-corrected kappa.

Verification

Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.

Does it back the claim? Not recorded. Neither this nor any other of the 7 citations of this work carries a recorded support check.

Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.

See how references were verified.

Links previously used in the corpus

Before the registry existed this work was also linked 1 other way.

• https://journals.sagepub.com/doi/10.1177/001316446002000104 ×1

Registry ID ref:8fbbed32ed6c · see in the full table