Skip to content

Demand Paging

Operating-system method — instantiates Demand-Triggered Deferred Evaluation

Loads a page of memory from backing store only when a program actually touches it, turning the page fault into the signal that the data is finally needed.

Demand Paging defers the loading of data until it is accessed. A program's address space is mapped but not resident; when it touches a page that is not in physical memory, the access traps as a page fault, and the operating system reads in just that page before resuming. Its distinguishing element is that the fault is the demand signal — hardware turns "the program actually needs this now" into an interrupt, with no explicit request in the program at all. This is the read-triggered sibling of copy-on-write's write-triggered copy: demand paging brings absent data in on access, and its whole character is governed by the tension between the latency of that first touch and the waste of loading pages that are never used.

Example

A data tool memory-maps a 4 GB file with mmap. At map time nothing is read from disk — the mapping just establishes the address range. When the program reads the first record, the access to that address faults; the OS loads the single 4 KB page containing it and resumes. A pass that touches 200 MB scattered through the file causes the file to cost 200 MB of I/O, not 4 GB — the untouched 95% is never read.

The tuning lives in the faults. A sequential scan would fault on every page in order, so the OS uses readahead: on seeing a sequential pattern it prefetches the next several pages before they are demanded, trading a little speculative I/O to hide first-touch latency. Push the working set past available RAM, though, and the same mechanism turns pathological — pages are evicted just before they are needed again, every access faults, and the machine spends its time paging instead of computing.

How it works

  • Map lazily, fault on access. Pages are mapped but not loaded; the first touch of a non-resident page traps as a fault — the demand signal.
  • Fault handling. The OS finds the page in backing store (file or swap), loads it into a physical frame, updates the page table, and restarts the faulting instruction. A minor fault only remaps an already-cached frame; a major fault hits the disk.
  • Speculate at the edges. Readahead/prepaging fetches likely-next pages ahead of demand; this is the eager exception to pure laziness, tuned to the access pattern.
  • Budget the working set. Eviction and replacement keep the resident set within physical memory, trading fault rate against footprint.

Tuning parameters

  • Readahead window — how many pages to prefetch on a detected pattern. Wider hides more latency on sequential access but wastes I/O and RAM on random access.
  • Page size — small pages vs. huge pages. Larger pages fault in more per event (fewer faults, more potential over-read); smaller pages are more precise but multiply fault and table overhead.
  • Working-set / residency target — how much to keep resident per process. Higher targets cut faults but starve other work and hasten memory pressure.
  • Eviction policy — which resident page to reclaim first (LRU-ish approximations, dirty-page write-back timing). It sets how gracefully the system degrades as demand approaches capacity.

When it helps, and when it misleads

Demand paging lets a program map far more than fits in RAM and pay only for what it touches — fast process startup, sparse access to huge files, memory over-subscription across many processes. First-use latency is real but amortized, and prefetch hides most of it on predictable patterns.

It misleads when the working set does not fit. Past that point the fault rate explodes into thrashing — the system spends more time servicing page faults than doing useful work, and throughput collapses far out of proportion to the memory shortfall.[1] On latency-critical paths, a single major fault is a multi-millisecond stall hidden inside an ordinary memory access, which is exactly where "memory is free until you touch it" bites. The classic misuse is over-committing memory on the assumption that unused mappings cost nothing, then hitting the thrashing cliff under load. The discipline: size the working set to real residency, tune readahead to the actual access pattern rather than the hope of one, and treat the latency/waste budget as a hard constraint with a cliff, not a smooth dial.

How it implements the components

  • demand_signal — the page fault on first access is the demand signal; hardware converts "needed now" into a trap with no explicit request in the program.
  • latency_and_waste_budget — the mechanism is defined by the trade between first-touch latency and the waste of loading (or retaining) pages that are never used.
  • prefetch_or_eager_exception_rule — readahead/prepaging is the deliberate eager exception, fetching likely-next pages ahead of demand to keep latency off the cliff.

It does NOT defer a copy on write (activation_rule, input_lifetime_and_capture_rule — that's Copy-on-Write) or persist computed results (memoized_result_store — that's Memoization); it loads existing data on read-demand and governs the latency and waste of doing so.

  • Instantiates: Demand-Triggered Deferred Evaluation — demand paging realizes mapped-but-absent data exactly at the access that demands it.
  • Sibling mechanisms: Copy-on-Write · Lazy-Initialization Proxy · Late-Materialization Query Plan · Memoization · Just-in-Time Compilation

Notes

Demand paging and Copy-on-Write both use the hardware fault as a demand signal and often operate on the same mapping, but they must not be conflated when diagnosing a stall: a fault on a read of absent data is a paging load, a fault on a write to shared data is a COW copy. The remedies differ — more RAM or better locality for the former, less sharing or coarser copy units for the latter.

References

[1] Thrashing is the collapse in throughput that occurs when the aggregate working set exceeds physical memory, so pages are evicted just before they are reused and nearly every access faults. It is the canonical failure of demand paging under memory pressure, mitigated by working-set management and admission control rather than by more paging. ↩