Skip to content

Cache or Read Replica

Software layer — instantiates Request–Response Capability Provisioning

Serves repeated or read-only requests from a synchronized copy placed close to demand, absorbing load that would otherwise hit — and overwhelm — the authoritative source.

A popular shared capability is often overwhelmed not by unique work but by the same reads asked over and over. The Cache or Read Replica answers those from a copy: a cache holds recent results keyed by the request, and a read replica holds a synchronized duplicate of the authoritative data, so a large fraction of requests are satisfied without ever touching the origin. Its defining move is trading freshness for load and latency — the copy may be slightly behind the source, and in exchange the origin is shielded from the bulk of read traffic while clients get answers faster and nearer. It adds no new answers and changes no contract; it multiplies the reach of the answers that already exist and puts a wall between read demand and the authoritative core.

Example

A retailer's browse-and-checkout traffic spikes fiftyfold during a Black-Friday drop. The product-catalog database — the authoritative origin, also handling inventory writes — would melt if every "show me this product" hit it directly. A read replica plus an edge cache serve product pages, prices, and descriptions from copies; roughly 95% of reads never reach the origin, which is left free to process the writes that actually matter: orders and stock decrements. A price changed at 9:00 propagates to the replicas within a few seconds, so a shopper might briefly see a slightly stale figure — but the site stays up through the surge. The origin survives because a copy absorbed the read storm, and the only cost is a few seconds of staleness on a product blurb.

How it works

  • Place a copy between clients and origin. A cache keyed by a request fingerprint, or a replica kept in sync with the source, sits in the request path.
  • Serve reads from the copy. A hit returns immediately without touching the origin; only misses — and all writes — fall through to it.
  • Keep the copy fresh enough. Via TTL expiry, invalidation on change, or streaming replication, trading staleness against how much origin load you deflect.
  • Isolate reads from writes. Read demand lands on the copy, not the authoritative core, so a read surge cannot starve the writes that must stay correct.

Tuning parameters

  • Freshness (TTL / replication lag) — how stale the copy may get. Longer deflects more load but serves older data; shorter is fresher but leaks more traffic through to the origin.
  • Invalidation strategy — expire-on-time versus invalidate-on-write. Time-based is simple but serves knowingly stale data; write-driven is fresh but complex and easy to get subtly wrong.
  • What's cacheable — only idempotent, non-personalized reads, or more. Widening coverage lifts the hit rate but risks serving one user another's data, or caching something that should never have been.
  • Placement — near the origin (a shared replica) versus near clients (an edge cache). Edge cuts latency most but multiplies copies that must be kept coherent.
  • Consistency guarantee — eventual versus read-your-writes. Stronger guarantees reassure clients but shrink the load you are able to deflect.

When it helps, and when it misleads

Its strength is that it absorbs the repetitive read load that would topple a shared origin, cuts latency by serving from nearby, and isolates read surges from the authoritative write path — often the cheapest way to raise a capability's effective capacity many times over.

Its failure mode is that staleness bites precisely where correctness matters: a shopper checks out against a cached old price, or a cached permission survives a revocation for a few dangerous seconds. Worse, a cache whose entries all expire at once can unleash a stampede that hits the origin harder than no cache at all.[1] The classic misuse is caching what must not be stale or must not be shared — authorization decisions, personalized or sensitive data, monetary balances — so the copy silently serves wrong or leaked answers under the appearance of a fast one. The discipline that keeps it safe is to cache only what tolerates being briefly wrong, make the staleness bound explicit rather than incidental, and stagger or lock expiry so a coordinated miss can't dogpile the origin.

How it implements the components

The replica realizes the offload and isolation side of the archetype — it does not change what the capability computes:

  • cache_or_replica_layer — it is this layer: the synchronized copy that serves reads in the origin's place.
  • isolation_boundary — it walls read demand off from the authoritative core, so a read surge cannot starve or topple the write path.

It does not add compute capacity to the origin itself (that is Autoscaling Worker Pool), define the request key it looks up (the API or RPC Endpoint's request contract, reused as a fingerprint by Idempotent API), or promise an availability level (Service-Level Agreement); the replica only offloads and shields.

Notes

A cache is a correctness bet, not just a performance tweak. It silently changes the consistency contract that clients experience, so the choice of what to cache is really a decision about what is allowed to be briefly wrong — which is why it belongs to whoever owns the data's semantics, not only to whoever owns the latency budget.

References

[1] Cache stampede (also thundering herd or dogpile) — when a popular cached entry expires and many concurrent requests all miss at once, they hammer the origin simultaneously, often harder than if no cache existed. Staggered expiry, request coalescing, or a short lock on recomputation are the standard guards. ↩