Cache hierarchy¶

A layered memory architecture that places progressively larger and slower caches between processors and backing memory to exploit locality at several scales.

Version: v2 · 2026-09-19 · History
Domain-specific #: 3571
Domain group: Applied Sciences & Engineering
Origin domain: Computer Science & Software Engineering
Subdomain: Computer Architecture → Computer Science & Software Engineering

Core Idea¶

A cache hierarchy coordinates levels such as L1, L2 and L3 through placement, replacement, write and coherence policies; a miss at one level triggers lookup or fill from a lower level. Recently or nearby accessed blocks remain in small fast stores, while less immediate data resides in larger slower levels, reducing average access latency when temporal and spatial locality hold. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.

How would you explain it like I'm…

Pocket, Box, and Closet

You keep your favorite toys in your pocket, more toys in a box on your desk, and everything else in the big closet. When you want a toy, you check your pocket first, then the box, then the closet. The toys you just played with stay close by. A computer keeps its data the same way, in a few layers of storage from tiny-and-fast to big-and-slow.

Fast-to-Slow Memory Layers

A computer's processor is much faster than its main memory, so it keeps copies of data in smaller, faster storage spaces called caches. These come in levels: L1 is tiny and super fast, L2 is bigger and a bit slower, and L3 is bigger and slower still. When the processor needs something, it checks L1 first; if it's not there (a "miss"), it looks in the next level down, and brings the data back up. This works because programs tend to reuse data they used recently and data stored near it. Rules decide where data goes, what gets thrown out when a cache is full, and how changes get written back.

Multilevel Cache Organization

A cache hierarchy is a set of cache levels, such as L1, L2, and L3, that sit between a processor and main memory, each larger and slower than the one above it. Recently used blocks, and blocks near them in memory, are kept in the small fast levels, while less immediately needed data lives in the bigger slower levels. When a lookup misses at one level, the request goes to the next lower level, which supplies the data and fills the upper level. This lowers the average time to access memory, but only when programs show temporal locality (reusing recent data) and spatial locality (using nearby data). The levels are coordinated by policies: where blocks can be placed, which block to replace, how writes are handled, whether levels must hold copies of each other's data (inclusion), and how multiple cores keep their copies consistent (coherence). A specific hierarchy is defined by making all of these choices explicit, along with each level's size, line size, speed, and which cores share it.

A cache hierarchy coordinates multiple cache levels, typically L1, L2, and L3, through placement, replacement, write, inclusion, and coherence policies. A miss at one level triggers lookup in, and fill from, the next lower level along a defined miss path. Small fast levels retain recently or nearby accessed blocks, and larger slower levels hold less immediate data, reducing average memory access latency when temporal and spatial locality hold; when locality fails, the hierarchy's benefit shrinks. A concrete hierarchy is characterized by its levels and their ownership (private or shared among cores), capacities, line sizes, latency and bandwidth, inclusion policy, placement and replacement schemes, write policy (for example write-back versus write-through), miss path, and coherence and consistency rules. These parameters are constitutive: two hierarchies that differ in inclusion or coherence behavior can perform and behave quite differently under the same workload. Analysis should declare each parameter, compare against a well-understood reference design, and avoid transferring performance conclusions to workloads or designs that violate its locality or policy assumptions.

Scope of Application¶

Cache hierarchy belongs to computer architecture and is useful where the analyst can specify the typed computer architecture carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, then evaluate the cache levels and ownership, capacities, line sizes, latency and bandwidth, inclusion policy, placement and replacement, write policy, miss path, coherence and consistency rules are explicit. The scope is broad within that domain but bounded by the need for the cache levels and ownership, capacities, line sizes, latency and bandwidth, inclusion policy, placement and replacement, write policy, miss path, coherence and consistency rules are explicit. Conceptual computer-architecture identity only; implementation requires platform-specific correctness, timing, security and coherence validation.

Clarity¶

The abstraction clarifies a crowded vocabulary by making the cache levels and ownership, capacities, line sizes, latency and bandwidth, inclusion policy, placement and replacement, write policy, miss path, coherence and consistency rules are explicit the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test. A bare label is insufficient because the name Cache hierarchy can be used for a formal identity, an implementation, or a neighboring result unless carrier and convention are stated.

Manages Complexity¶

Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Cache hierarchy. Cache hierarchy compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.

Abstract Reasoning¶

Identify the carrier. State what the elements, states, objects, or observations are: the typed computer architecture carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets. Reject examples whose alleged carrier belongs to a different problem.
Lock the constitutive rule. Express the cache levels and ownership, capacities, line sizes, latency and bandwidth, inclusion policy, placement and replacement, write policy, miss path, coherence and consistency rules are explicit independently of one notation or implementation.

Knowledge Transfer¶

Knowledge transfers strongly among subfields of computer architecture because they reuse the typed computer architecture carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, Recently or nearby accessed blocks remain in small fast stores, while less immediate data resides in larger slower levels, reducing average access latency when temporal and spatial locality hold., and type the carrier, state every parameter and convention in the definition, test that the cache levels and ownership, capacities, line sizes, latency and bandwidth, inclusion policy, placement and replacement, write policy, miss path, coherence and consistency rules are explicit, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.

Relationships to Other Abstractions¶

Current abstraction Cache hierarchy Domain-specific

Parents (1) — outgoing typed relations

Cache hierarchy is a kind of Hierarchy Prime

It remains its own entry because its identity is fixed by the cache levels and ownership, capacities, line sizes, latency and bandwidth, inclusion policy, placement and replacement, write policy, miss path, coherence and consistency rules.

Hierarchy paths (4) — routes to 4 parentless roots

Cache hierarchy → is a kind of Hierarchy → is a kind of Network

Show alternative paths (3)

Neighborhood in Abstraction Space¶

Cache hierarchy sits in a crowded region of the domain-specific corpus (4^th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Memory Architecture & Address Spaces (25 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25