Item analysis¶

A psychometric evaluation and selection process that examines candidate questions for difficulty, discrimination, redundancy, model fit, fairness and construct coverage before assembling or revising a test.

Version: v2 · 2026-09-19 · History
Domain-specific #: 5127
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomains: Psychometrics, Test Item Evaluation → Experimental Design & Statistics

Core Idea¶

Item analysis is the combined statistical and substantive examination of individual test items to decide whether they function appropriately and collectively measure the intended construct. A representative sample answers an item pool; difficulty, item-total association, discrimination, distractors, reliability contribution, dimensionality, model fit or differential functioning expose weak items; expert review protects content coverage and meaning. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.

How would you explain it like I'm…

Checking Every Quiz Question

When grown-ups make a quiz, they check each question one by one to see if it is a good question. They look at how kids answered it, and they also ask experts whether it really asks about the right thing. Bad questions get fixed or thrown out.

Question-by-Question Test Checkup

Item analysis means checking every single question on a test to see if it is doing its job. First, a group of people like the ones the test is meant for answers the questions. Then the test makers use numbers, such as how many people got each question right and whether strong test-takers did better on it. Experts also read the questions to make sure they cover the right topics and make sense. Weak questions are revised or dropped, and the checking can repeat.

Item-Level Test Diagnosis

Item analysis is the question-by-question examination of a test during its construction, combining statistics with expert judgment. A representative sample answers a pool of items, and analysts compute each item's difficulty, how strongly it relates to the total score, how well it separates strong from weak test-takers, and how its distractors behave. They also check its effect on reliability and whether it fits the intended model or behaves differently for different groups. Expert review then protects content coverage and meaning. It differs from validating the whole test: the focus is on diagnosing and selecting individual items against a stated construct and population.

Item analysis combines statistical and substantive review of individual test items to decide whether each functions appropriately and whether together they measure the intended construct. A representative sample responds to an item pool. Analysts examine item difficulty, item–total association, discrimination, distractor performance, contribution to reliability, dimensionality, fit to a measurement model, and differential item functioning across groups. Expert review checks content coverage and meaning against a construct blueprint. Evidence is judged at the item level relative to a declared measurement model and target population, and decisions integrate statistics with judgment rather than relying on numbers alone. The process is iterative and distinct from whole-test validation.

Scope of Application¶

Item analysis belongs to psychometrics and is useful where the analyst can specify a target construct and population, candidate items, response data, scoring keys, a classical or item-response model, item statistics, content judgments and retain-revise-remove decisions, then evaluate evidence is evaluated at the item level against a declared measurement model, target population and construct blueprint, and decisions integrate statistics with substantive judgment. The scope is broad within that domain but bounded by the need for evidence is evaluated at the item level against a declared measurement model, target population and construct blueprint, and decisions integrate statistics with substantive judgment. Statistical thresholds are population- and purpose-dependent; item analysis does not justify high-stakes use without broader validity, reliability, accessibility and fairness evidence.

Clarity¶

The abstraction clarifies a crowded vocabulary by making evidence is evaluated at the item level against a declared measurement model, target population and construct blueprint, and decisions integrate statistics with substantive judgment the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test. A bare label is insufficient because the name Item analysis can be used for a formal identity, an implementation, or a neighboring result unless carrier and convention are stated.

Manages Complexity¶

Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Item analysis. Item analysis compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.

Abstract Reasoning¶

Identify the carrier. State what the elements, states, objects, or observations are: a target construct and population, candidate items, response data, scoring keys, a classical or item-response model, item statistics, content judgments and retain-revise-remove decisions. Reject examples whose alleged carrier belongs to a different problem.
Lock the constitutive rule. Express evidence is evaluated at the item level against a declared measurement model, target population and construct blueprint, and decisions integrate statistics with substantive judgment independently of one notation or implementation.

Knowledge Transfer¶

Knowledge transfers strongly among subfields of psychometrics because they reuse a target construct and population, candidate items, response data, scoring keys, a classical or item-response model, item statistics, content judgments and retain-revise-remove decisions, A representative sample answers an item pool; difficulty, item-total association, discrimination, distractors, reliability contribution, dimensionality, model fit or differential functioning expose weak items; expert review protects content coverage and meaning., and type the carrier, state every parameter and convention in the definition, test that evidence is evaluated at the item level against a declared measurement model, target population and construct blueprint, and decisions integrate statistics with substantive judgment, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.

Relationships to Other Abstractions¶

Current abstraction Item analysis Domain-specific

Parents (1) — outgoing typed relations

Item analysis is a kind of Selection Prime

Item analysis selects and revises questions using empirical and content criteria.

Hierarchy path (1) — routes to 1 parentless root

Item analysis → is a kind of Selection

Neighborhood in Abstraction Space¶

Item analysis sits in a crowded region of the domain-specific corpus (10^th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Psychometric Measurement & Judgment Biases (29 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25