Producing Wrong Data Without Doing Anything Obviously Wrong!¶
Mytkowicz, T., Diwan, A., Hauswirth, M., & Sweeney, P. F. (2009). Producing Wrong Data Without Doing Anything Obviously Wrong!. Proceedings of ASPLOS ’09, 265-276.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Mechanisms¶
- Algorithm Benchmarking
- A benchmark is honest only about the sizes it ran and the inputs it used; extrapolating a smooth line past the last rung can hide a cliff sitting just beyond it, and a microbenchmark can produce a beautiful, meaningless number
This sourceShows that seemingly innocuous experimental-setup choices can make systems performance measurements support incorrect conclusions.
- A benchmark is honest only about the sizes it ran and the inputs it used; extrapolating a smooth line past the last rung can hide a cliff sitting just beyond it, and a microbenchmark can produce a beautiful, meaningless number
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:4e16874378db · see in the full table