Schema Migration¶
Migration process — instantiates Compatibility Management
Transforms stored data and the interfaces over it from an old structural version to a new one — old-version dependents mapped first — so structure can change without losing or stranding information.
Schema Migration is the executable procedure that carries data, records, and the code that reads them from one structural version to the next. Where a versioning scheme labels a structural change and a policy permits it, this mechanism actually performs it: it rewrites tables, splits or merges fields, backfills new columns, and repoints the queries and jobs that read them. Its defining constraint is that the data already exists and is in use — you cannot start from a blank schema, so the migration must move a live, populated store from shape A to shape B while everything reading it keeps working. That is why it begins not with the new schema but with an inventory of everything bound to the old one.
Example¶
An e-commerce company needs to split a single customer_name column into first_name and last_name across an orders table holding tens of millions of rows. A naive ALTER TABLE that drops the old column would instantly break every report, shipping-label generator, and analytics job still selecting customer_name.
So the migration runs in phases. First, a dependency scan finds every consumer of the old column — 40-odd stored queries, three downstream services, a nightly export. Then the schema expands: the two new columns are added alongside the old one, and application code begins writing all three at once (dual-writing). A backfill job populates first_name/last_name for historical rows in batches, over ≈two weeks, so the database is never locked. Consumers are moved to the new columns one at a time, each verified against the still-present old column. Only once the dependency map shows zero remaining readers of customer_name does the final phase contract, dropping it. The structure changed; not one order record or report broke, because the old shape stayed valid until nothing needed it.
How it works¶
- Map the exposure first. Enumerate every table, index, query, service, report, and scheduled job that touches the old structure. This map defines both the blast radius and the exit condition — "done" means no readers of the old shape remain.
- Expand, then contract. Introduce the new structure beside the old, run them in parallel while both are written, migrate readers across, and only then remove the old — never a single destructive cutover.[n1]
- Backfill in bounded batches. Transform existing rows incrementally so a live store is never locked, and a failure rolls back a batch rather than the database.
- Transform, don't just relabel. Where meaning changes — units, splits, merges, re-categorisation — the migration carries the conversion logic and preserves the information rather than dropping it.
Tuning parameters¶
- Online vs. offline — whether the store stays live throughout (expand/contract, dual-write) or takes a maintenance window. Online is safer for users but far more intricate; an offline cutover is simple but costs downtime.
- Batch size and pacing — how many rows each backfill step transforms. Larger batches finish sooner but hold locks and load longer; smaller batches are gentler but slower.
- Reversibility — whether each step keeps the old structure intact (writing to both) so it can roll back, or commits irreversibly for speed. Dual-writing buys a safety net at the cost of transient duplication.
- Transformation fidelity — how aggressively to clean or re-derive data in flight versus moving it verbatim. In-flight fixes are efficient but can hide errors inside an already risky step.
When it helps, and when it misleads¶
Its strength is that it lets structure evolve on a store that cannot be paused or rebuilt — the normal case for anything real and in production. The expand/contract discipline in particular turns a terrifying all-at-once cutover into a sequence of individually reversible steps.
Its failure modes cluster around what the map misses. An unmapped dependency — the quarterly report no one remembered, the partner reading the database directly — breaks silently the moment the old shape is contracted, which is why the exit condition must be evidence of zero readers, not a guess. Irreversible steps run to save time leave nothing to roll back to when a backfill corrupts data midway. And a migration that transforms meaning can quietly lose information — collapsing categories, truncating precision — under the cover of a "purely structural" change. The discipline is to keep the old structure valid until the dependency map proves it dead, and to keep each step reversible until the one that removes the old shape is provably safe.
How it implements the components¶
Schema Migration fills the components that actually move data across a version boundary:
migration_path— it is the path made executable: the ordered, phased transformation (expand → dual-write → backfill → migrate readers → contract) that takes a populated store from the old structure to the new.dependency_exposure_map— the opening scan produces the map of everything bound to the old structure, which both scopes the work and defines when it is safe to finish.
It does not decide how long the old structure must remain supported (support_window — Backward Compatibility Policy), signal the change through a version number (Semantic Versioning), or write the human-facing instructions for external consumers who must adapt their own code (deprecation_policy, exception_and_waiver_path — Migration Guide). This mechanism moves the data; siblings govern, signal, and document the move.
Related¶
- Instantiates: Compatibility Management — Schema Migration is where a permitted version change becomes an actual, executed transition of live data.
- Consumes: the dependency map it builds is scoped by whatever Backward Compatibility Policy says must keep working during the transition.
- Sibling mechanisms: Migration Guide is its human-facing counterpart — this moves records, that moves people. Rolling Upgrade · Adapter Layer · Compatibility Test Suite · Semantic Versioning
Editorial Notes¶
Form Classification¶
Form family: Intervention, Treatment & Transformation
Rationale: Schema Migration operates by directly transforms stored structures and their readers from the old schema to the new one. That concrete deployed or enacted form is Intervention, Treatment & Transformation under the frozen taxonomy.
Nearest alternative: Protocol, Workflow & Routine — Although Protocol, Workflow & Routine can support this mechanism, the frozen evidence makes its operative form the act that directly transforms stored structures and their readers from the old schema to the new one; the alternative is therefore secondary rather than defining.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Transforming stored data and dependent interfaces across structural versions is database engineering.
Related originating lineages:
- Engineering & Design — Engineering design, reliability, and systems-safety practice supplies a parallel or contributing lineage for the mechanism's defining operation: transforms stored data and the interfaces over it from an old structural version to a new one — old-version dependents mapped first — so structure can change without losing or….
Review resolution: Both blind reviewers agree that computer_science is the primary historical origin. Explicit reconciliation of alternate_origin_disagreement starts from reviewer_a's mechanism-specific evidence: Transforming stored data and dependent interfaces across structural versions is database engineering. Reviewer A proposed alternates=none, origin_mode=single_lineage, domain_reach=specialized, and encyclopedia_synthesis=false; reviewer B proposed alternates=engineering_design, origin_mode=single_lineage, domain_reach=specialized, and encyclopedia_synthesis=false. The final record retains every independently supported alternate from either review (engineering_design) without an arbitrary cap, selects origin_mode=single_lineage to represent the combined lineage evidence, and records domain_reach=specialized and encyclopedia_synthesis=false. Present-day transfer is recorded as reach and is not treated as proof of historical origin.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
Schema Migration is a one-directional transition, not a coexistence mechanism: its job is to end with everything on the new structure. When old and new must run side by side indefinitely rather than converge, that is an Adapter Layer or Rolling Upgrade concern — reaching for a migration there tends to produce a cutover that was quietly never finished.
[n1] The expand/contract pattern (also called parallel change): add the new structure, run old and new in parallel while both are maintained, migrate every reader, then remove the old — the standard way to change a schema without a breaking cutover. ↩