Deferred Database Query Object¶

Query tool — instantiates Demand-Triggered Deferred Evaluation

Represents a database query as a composable object that builds up lazily and only touches the database when its results are actually iterated.

A Deferred Database Query Object is a first-class value that describes a query without running it. Filters, orderings, and joins are chained onto the object to build up an intent; the database is not touched until something actually consumes the rows — iterating, counting, or materializing to a list. Its distinguishing consequence is not efficiency but displaced timing: because construction and execution are separated, the moment a query fails and the moment its authorization is checked both move from where the query is written to where it is finally run. A deferred query is therefore as much a contract about when errors and permission checks land as it is about which rows come back — and getting that contract wrong is how lazy queries bite.

Example¶

A web view builds up orders.filter(status="open").filter(region=user.region).order_by("-placed_at"). None of those chained calls talks to the database; each returns a new query object with more conditions attached. The query runs only when the template loops over it to render rows.

Two things follow from that deferral. First, a malformed filter or a permission problem does not raise where the query was written, in the clean controller code — it raises at render time, deep in the template, where the stack trace is least helpful. Second, if that same query object is handed to a background job that executes later under a different principal, the rows it returns reflect that job's authorization at execution time, not the original requester's — so permission must be re-validated where the query actually runs, not where it was built. Used well, the object also lets the ORM collapse the whole chain into one efficient SQL statement instead of many.

How it works¶

Compose without executing. Each builder call returns a new (or mutated) query object carrying accumulated intent; no I/O happens during composition.
Execute on a trigger. A defined set of operations forces execution — iteration, len/count, list materialization, boolean tests. Knowing that trigger set is the whole skill of using the tool safely.
Bind failure to execution. Errors — bad SQL, constraint issues, connection loss — surface at the forcing point, so exception handling belongs around execution, not construction.
Revalidate authorization at run time. Because execution can happen later and elsewhere, the principal and scope in force at execution govern the result and must be checked there.

Tuning parameters¶

Execution trigger set — which operations force the query. Narrower, more explicit triggers make accidental execution rarer and easier to reason about.
Result caching — whether a forced query caches its rows on the object for reuse or re-executes on each pass. Caching avoids repeat trips but risks serving stale rows within a long-lived object.
Eager-load boundary — how many related sets to pull in one execution vs. leaving them lazy. Wider eager loading avoids the N+1 trap; too wide over-fetches.
Authorization scoping — whether permission predicates are baked into the object at build time or re-derived from the executing principal. Re-derivation is safer across boundaries; baking-in is simpler within one request.

When it helps, and when it misleads¶

The object shines when queries must be built up conditionally — assembled from optional filters, passed between layers, reused as a base — while still executing as one efficient statement at the point of need. It keeps query intent composable and lets the planner optimize the whole shape at once.

Its failure modes all trace to the split between building and running. The signature one is the N+1 query problem: a lazy query iterated inside a loop, or a related set forced per row, silently fans one intended query into hundreds.^[1] Because failures surface at execution, they land far from their cause and are easy to catch in the wrong place — or to trigger accidentally by logging or truthiness-testing the object, which forces it. And a query object that outlives its request can execute under stale authorization or a stale transaction, returning rows the current context should never see. The classic misuse is passing a lazy query across a boundary (a template, a job, a cache) and assuming it still carries the context it was born in. The discipline: know exactly what forces execution, handle errors at the execution site, and re-validate authorization wherever the query actually runs.

How it implements the components¶

failure_and_exception_semantics — by deferring execution, the object defines when and where a query's errors surface: at the forcing point, not at construction.
authorization_revalidation_rule — because execution can occur later and under a different principal, permission is re-checked at run time rather than assumed from build time.

It does NOT cache rows for cross-call reuse (memoized_result_store — that's Memoization) or decide *whether the query is needed at all (need_predicate — that's the Conditional Workflow Gate); and it optimizes nothing about how the query executes — that execution planning is the job of query-planner siblings such as Predicate Pushdown and Late-Materialization Query Plan.*

Instantiates: Demand-Triggered Deferred Evaluation — the query object is deferred work whose failure timing and authorization become explicit at the point of execution.
Sibling mechanisms: Predicate Pushdown · Late-Materialization Query Plan · Lazy Sequence Generator · On-Demand Report Generation · Memoization · Conditional Workflow Gate

Notes¶

The defining risk is accidental execution. Because forcing is triggered by ordinary operations — iterating, taking a length, testing truthiness, sometimes even logging the object — a deferred query can turn eager, or turn into many queries, without anyone writing an obvious "run it" call. Fluency with the tool is largely fluency with its trigger set.

References¶

[1] The N+1 query problem is issuing one query to fetch N parent rows and then one additional query per row to fetch related data — N+1 round trips where one or two would do. It is the archetypal failure of lazily executed queries and is cured by eager-loading (join/prefetch) the related sets in the initial execution. ↩