Hidden Technical Debt in Machine Learning Systems¶
Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., et al. (2015). Hidden Technical Debt in Machine Learning Systems. Advances in Neural Information Processing Systems 28 (NeurIPS 2015), 2503-2511.
Cited by¶
3 citations across 3 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Data Drift
- In machine learning and data science, the origin substrate, deployed models silently degrade as customer mix, prices, behaviour, fraud tactics, or sensor characteristics evolve; the entire field of model operations exists largely to detect and respond to drift.
This sourceEstablishes that deployed models silently degrade as the world evolves (changes in the external world), motivating the model-operations discipline of drift detection and refresh.
- In machine learning and data science, the origin substrate, deployed models silently degrade as customer mix, prices, behaviour, fraud tactics, or sensor characteristics evolve; the entire field of model operations exists largely to detect and respond to drift.
- Training Serving Skew
- A single worked instance shows the substance: a churn model whose precision drops in serving while still scoring well on its original holdout turns out to have two distinct skews — a new feature absent at training and a feature pipeline that normalises differently in production — neither a model failure, each with a different remedy.
This sourceNames training/serving skew from divergent training and serving feature pipelines as a maintenance hazard.
- A single worked instance shows the substance: a churn model whose precision drops in serving while still scoring well on its original holdout turns out to have two distinct skews — a new feature absent at training and a feature pipeline that normalises differently in production — neither a model failure, each with a different remedy.
- Versioning
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 1 other way.
Registry ID ref:a2e9f199296a · see in the full table