Hidden technical debt in machine learning systems.¶
Sculley, D., Holt, Golovin, Davydov, Phillips, Ebner, Chaudhary, et al. (2015). Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems.
Cited by¶
4 citations across 4 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Data Drift
- In machine learning and data science, the origin substrate, deployed models silently degrade as customer mix, prices, behaviour, fraud tactics, or sensor characteristics evolve; the entire field of model operations exists largely to detect and respond to drift.
This sourceEstablishes that deployed models silently degrade as the world evolves (changes in the external world), motivating the model-operations discipline of drift detection and refresh.
- In machine learning and data science, the origin substrate, deployed models silently degrade as customer mix, prices, behaviour, fraud tactics, or sensor characteristics evolve; the entire field of model operations exists largely to detect and respond to drift.
- Training Serving Skew
- A single worked instance shows the substance: a churn model whose precision drops in serving while still scoring well on its original holdout turns out to have two distinct skews — a new feature absent at training and a feature pipeline that normalises differently in production — neither a model failure, each with a different remedy.
This sourceNames training/serving skew from divergent training and serving feature pipelines as a maintenance hazard.
- A single worked instance shows the substance: a churn model whose precision drops in serving while still scoring well on its original holdout turns out to have two distinct skews — a new feature absent at training and a feature pipeline that normalises differently in production — neither a model failure, each with a different remedy.
- Versioning
Mechanisms¶
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. Neither this nor any other of the 4 citations of this work carries a recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 2 other ways.
- https://papers.nips.cc/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html ×1
- https://proceedings.neurips.cc/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html ×1
Registry ID ref:a2e9f199296a · see in the full table