Skip to content

Stale-While-Revalidate Cache

Caching protocol — instantiates Synchronized Release Dampening

On cache expiry, serves the slightly-stale value instantly to everyone while a single background refresh runs — so a synchronized expiry never becomes a stampede on the origin.

When a cached value expires, a naive cache blocks its readers and sends them to the origin — and if that value is hot, every reader converges on the origin in the same instant. Stale-While-Revalidate Cache refuses to make anyone wait. On expiry it keeps serving the stale value for a bounded grace window and fires exactly one asynchronous revalidation in the background; when the refresh lands, it flips to fresh. Its defining move is to remove the expiry timestamp as a herd trigger by decoupling the read path from the refresh path — because no reader ever waits on the origin, the many readers never pile onto it at once. Staleness is the currency it spends: it accepts a few seconds of "slightly old" in exchange for the origin never seeing the synchronized wave. The behavior is standardized as the stale-while-revalidate directive of RFC 5861.[1]

Example

An online store caches each product's rendered price-and-availability fragment with a 30-second freshness lifetime. During a flash sale one hot product's fragment is read roughly 5,000 times a second. The moment it expires, a naive cache would send that entire wave at the pricing service at once — a stampede on the exact service already under the most load.

With stale-while-revalidate configured (a fresh lifetime of 30 seconds, then a stale-serving grace window of, say, several minutes), the expiry plays out very differently. The first reader after expiry is handed the 30-second-old price immediately and, in the background, kicks off one revalidation. The other ~5,000 reads that second are also served the stale price, touching the origin not at all. A fraction of a second later the refresh returns and the cache is fresh again. Across the whole expiry event the pricing service saw one request; every shopper saw a price at most a few seconds stale. The synchronized expiry produced no herd, because not a single read ever blocked on the origin.

How it works

  • Two lifetimes per entry. A fresh lifetime and, past it, a longer stale-serving grace window. Inside the grace window the value is served without blocking.
  • Serve-stale-and-trigger. The first read after expiry returns the stale value and starts one background revalidation; that refresh is deduplicated so only a single one runs, and reads arriving during it keep getting stale.
  • Keep the read path off the origin. Only the async refresh contacts the origin; readers are never blocked and never queued behind it.
  • Fall back at the ceiling. Past the grace window — or on explicit demand — revert to a blocking fetch. That ceiling is the hard limit on how stale you are willing to go.

Tuning parameters

  • Fresh lifetime vs. stale-grace window — the fresh TTL sets how current the data is; the grace window sets how long you'll serve stale to dodge a stampede. A longer grace window absorbs bigger herds but risks handing out older values.
  • Revalidation trigger — lazy (on first miss) vs. proactive (refresh shortly before expiry so the value is rarely stale at all). Proactive smooths further but spends origin calls you might not have needed.
  • Staleness ceiling — the hard age past which stale is refused and a blocking fetch is forced; the guard against serving dangerously old data.
  • Refresh coalescing scope — per-node vs. one cluster-wide revalidation (via a shared lock or lease). Cluster-wide suppresses more origin load but adds coordination.

When it helps, and when it misleads

Its strength is that it turns a cliff — synchronized expiry into simultaneous origin hits — into a non-event, and no reader ever pays latency for it. It is the natural defense for hot, read-heavy keys where a few seconds of staleness is harmless, and it is most effective exactly when load is highest, because it caps origin traffic at one refresh regardless of read volume.

Its cost is that it trades correctness for smoothness: everything served in the grace window is, by definition, stale, which makes it wrong for data where staleness is unsafe — a bank balance, a just-revoked credential, inventory that has actually hit zero. A too-generous grace window paired with an origin that stays down will keep serving very old data while quietly masking that the refresh is failing — a silent staleness leak. The misuse is stretching the grace window to hide an origin that simply can't keep up, using the mechanism as a bandage over a capacity problem instead of as a desynchronizer. And it offers nothing for a herd of distinct, uncacheable requests, where readers want different things and there is no shared value to serve. The discipline is a hard staleness ceiling, an alarm on revalidation failures so stale never hides an outage, and caching only what is safe to serve slightly old.

How it implements the components

  • shared_release_signal — the expiry timestamp is the synchronized cue that would otherwise release every reader toward the origin at once; the mechanism exists to neutralize precisely this signal, decoupling reads from it.
  • coalescing_rule — exactly one revalidation runs per expiry while all concurrent readers share the stale value, so the many-readers-one-refresh fan-in is the coalescing rule in action.

It does NOT block callers to share a single *fresh in-flight result — that is Single-Flight Request Coalescing, which dedupes concurrent misses by making waiters wait; this mechanism dedupes a temporal expiry by serving stale and never blocking. It does not gate or rate-limit admissions (Token-Bucket Admission Gate) and does not cap concurrency at the origin (Semaphore-Limited Release); it simply routes almost all reads away from the origin entirely.*

Notes

Stale-while-revalidate and single-flight coalescing are complementary, not competing. This mechanism handles the warm expiry — there is an old value to serve while one refresh runs — but on a cold start there is no stale value to hand out, and the very first fill of a hot key can still stampede. That first-fill gap is exactly what a single-flight coalescer covers, by blocking the concurrent misses onto one leader. A hot key often wants both: single-flight for the cold fill, stale-while-revalidate for every expiry after.

References

[1] RFC 5861 (HTTP Cache-Control Extensions for Stale Content) defines the stale-while-revalidate response directive, which permits a cache to serve a stale response while it revalidates in the background, and the companion stale-if-error directive for serving stale content when the origin is unreachable. ↩