Low Agreement Among Reviewers Evaluating the Same NIH Grant Applications¶
Pier, E. L., Brauer, M., Filut, A., Kaatz, A., Raclaw, J., Nathan, M. J., Ford, C. E., et al. (2018). Low Agreement Among Reviewers Evaluating the Same NIH Grant Applications. Proceedings of the National Academy of Sciences, 115(12), 2952-2957.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Outlier Leverage
- Position limits from finance transfer into A/B-test design as per-user contribution caps and into reviewer-influence design in peer-review and hiring panels, where no single reviewer's score should determine the outcome.
This sourceLow inter-reviewer agreement and reviewer-leniency effects motivate aggregating across multiple independent reviewers so no single reviewer's score determines a funding outcome — the panel analogue of bounding any one observer's influence.
- Position limits from finance transfer into A/B-test design as per-user contribution caps and into reviewer-influence design in peer-review and hiring panels, where no single reviewer's score should determine the outcome.
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:2d091dbd6376 · see in the full table