Do ImageNet Classifiers Generalize to ImageNet?¶
Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019). Do ImageNet Classifiers Generalize to ImageNet?. Proceedings of the 36th International Conference on Machine Learning (ICML).
Cited by¶
2 citations across 2 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Researcher Degrees of Freedom
- In machine-learning evaluation, tuning on the test set, model selection across many architectures, choice of benchmark suite, and prompt selection each reproduce the structure: many silent comparisons summarised as one.
This sourceBuilds fresh test sets to probe adaptive overfitting from repeated test-set reuse in ML benchmark evaluation.
- In machine-learning evaluation, tuning on the test set, model selection across many architectures, choice of benchmark suite, and prompt selection each reproduce the structure: many silent comparisons summarised as one.
Mechanisms¶
- Baseline Comparison Table
- The table only measures on the tasks inside it, so a method that wins here can still fail out of distribution — that blind spot belongs to the Challenge Case Red Team and the out-of-distribution monitor, not here.
This sourceFinds substantial accuracy drops when strong benchmark classifiers are evaluated on newly collected data.
- The table only measures on the tasks inside it, so a method that wins here can still fail out of distribution — that blind spot belongs to the Challenge Case Red Team and the out-of-distribution monitor, not here.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. Neither this nor any other of the 2 citations of this work carries a recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 1 other way.
Registry ID ref:f28862f37b1a · see in the full table