Low Agreement Among Reviewers Evaluating the Same NIH Grant Applications¶
Pier, E. L., Brauer, M., Filut, A., Kaatz, A., Raclaw, J., Nathan, M. J., Ford, C. E., et al. (2018). Low Agreement Among Reviewers Evaluating the Same NIH Grant Applications. Proceedings of the National Academy of Sciences, 115(12), 2952-2957.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Outlier Leverage
- Position limits from finance transfer into A/B-test design as per-user contribution caps and into reviewer-influence design in peer-review and hiring panels, where no single reviewer's score should determine the outcome.
This sourceLow inter-reviewer agreement and reviewer-leniency effects motivate aggregating across multiple independent reviewers so no single reviewer's score determines a funding outcome — the panel analogue of bounding any one observer's influence.
- Position limits from finance transfer into A/B-test design as per-user contribution caps and into reviewer-influence design in peer-review and hiring panels, where no single reviewer's score should determine the outcome.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Not recorded. The single citation of this work carries no recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:2d091dbd6376 · see in the full table