How references were verified

← Back to references

Two different questions

A citation can fail in two unrelated ways: the source may not exist, or it may exist and not say what the sentence claims it says. Those are checked separately here, they are at different grains, and the second is the one that matters.

Existence is a property of the work. One identifier, one answer, and it is cheap to check at scale.

Support is a property of the citation, not of the work. The same book can be cited soundly in one article and wrongly in another, so support is recorded against each individual citation and is never inferred for a work as a whole. It is expensive: it means reading the source against the sentence.

Where that stands today

• 11,805 of 34,800 works (34%) have a DOI or ISBN that resolves to a registered record.
• 180 works carry an identifier that does not resolve. Their pages say so and do not link it.
• 22,815 works (66%) have no resolved identifier. 20,644 carry no identifier to check; the other 2,171 carry one that has simply not been checked yet. Their pages say which.
• 2,068 works were re-examined in a second, independent pass that read the citation against the article text.
• 19,050 of 50,020 citations carry a published grade from a check of the cited work against the specific sentence they sit on (support checks, below); a further 909 were checked but could not be verified.

That figure understates the checking done. It counts only checks whose result is published against the individual citation. Citations checked before there was a field to record the result in cannot be told apart, in the registry, from citations nobody has looked at, so they are not counted.

Support-checking is the expensive pass and it is early. When the pre-existing corpus was first support-checked rather than merely existence-checked, roughly 8% of the citations examined turned out not to back the claim they sat on — wrong paper, unfilled placeholder, or an annotation copied from an unrelated entry. That rate is why the work continues, and why this page states coverage instead of implying completeness.

Verification is deliberately split into separate passes — find the claims that need a citation, supply one, then confirm it — because combining them degrades every one of the three judgements. A separate layer of reviewers re-checked a sample from scratch, without seeing who produced the citation.

Every figure on this page is counted from the registry at build time, not written by hand.

What this does not establish

A support-checked citation means a real source supports that sentence. It does not mean the source is correct, that the literature agrees with it, or that the surrounding argument is sound. Where authoritative sources genuinely disagree, the claim was kept and the disagreement noted rather than being quietly resolved in one direction.

The registry, and what it exposes

These 34,800 works were deduplicated from 50,020 citations. Deduplication is not cosmetic: before it, the same book could appear a dozen different ways, under different author renderings, pointing at five different URLs. The registry makes a correction propagate everywhere at once.

It also makes the gaps countable, which is the point:

• 1,766 works (5%) have no authoritative link yet. Many are pre-internet or out-of-print sources; the rest are simply unfinished work.
• 1,242 works (4%) could not be classified by publication type, because their citation is too thin to classify from.
• Editions are counted separately. A 1988 book and its 2013 revised edition are two works, because they are.

What each publication type means, with how many works carry it, is at the foot of the References page.

Support checks

An automated reader checked citations from the prime abstractions, domain-specific abstractions and mechanisms, each with the sentence it sits on and whatever of the cited work could be obtained, and asked one question: does the work state what the sentence attributes to it? 19,050 of its answers are published — read against the full text for 8,819, against a saved copy of the source page for 876, against the publisher's abstract for 6,955 — and shown under the citation on the article page and on the work's page here, in three grades:

• Supported (4,340) — the work states the claim.
• Supported in part (14,679) — the work states some of what the sentence asserts. Most sentences make more than one assertion and a single source rarely carries all of them; the disclosure says which part this source does not cover. It is not a finding against the sentence.
• Borderline support (31) — two runs of the same check disagreed on this citation. The reader's own stated confidence turned out not to predict these disagreements, so it is not shown; the disagreement itself is.

A further 909 citations were checked but could not be verified: the source could not be read closely enough to judge the sentence. They carry no badge, and the References page counts them on a line of their own.

Every disclosure names what it was verified against. A verdict from the publisher's abstract is a weaker check than one from the full text, and the page says which it was rather than letting one word cover both.

Rechecked against the whole text. Published grades whose source is held in full are being read again against the whole of that text, not an excerpt of it. Where the second reading would withdraw a published grade, or raise a grade to Supported or lower it from Supported, a further reader decides between the two readings without being told which reader wrote which, and the page shows that decision.

Not yet verified. Citations the reader could not read — no text obtainable, or a catalogue page instead of the work — carry no verdict and no badge, and so does everything not yet reached; that is most of the corpus, and it is the honest default. Checking is slow — a source read against a sentence, one citation at a time — and carrying it across the whole encyclopedia will take a long stretch of calendar time. Negative verdicts are held back until a second, human-directed pass separates the wrong document is cited from the right document does not say this: an automated reader working from an abstract confuses the two, and either is a correction to the article rather than a label for the reader. A verdict is also withdrawn when the sentence under the marker is no longer the sentence the reader saw.

What it establishes. That the cited work says what the sentence says it says — nothing more. Not that the work is right, not that the field agrees with it, and not that a human has read it.

Contested claims

Claims where authoritative sources legitimately disagree are retained by design. They are not yet individually flagged on the page where they appear. That work is planned and not done, and this page will say so until it is.