Schema Migration Workflow¶
Process workflow — instantiates Schema Update Protocol
Sequences a schema change into a releasable process — freeze, dual-run, cut over, verify — so live data crosses to the new structure without a period where old and new are both broken.
A Schema Migration Workflow is the orchestration that carries a live, in-use schema across to a revised one without a window in which the system is broken. Its defining feature is that it is a sequenced, releasable process over time — a staged pipeline with a defined order of operations and rollback points — rather than a single edit or a decision. Where other mechanisms decide the new structure or relabel a static corpus, the workflow's problem is change under load: data is being written and read continuously, so the migration must maintain continuity through every intermediate state and prove, at the end, that nothing was lost or mis-mapped in the crossing. Its signature move is to make the two schemas coexist briefly — write to both, read from one — so the cutover is a controlled switch rather than a leap.
Example¶
A product-analytics team is migrating its event taxonomy in a data warehouse that ingests millions of events an hour. The old schema recorded a single page_view event with a messy type field; the new schema splits it into typed events. A hard swap is impossible — dashboards, alerts, and a year of historical queries all read the old shape live. The Schema Migration Workflow stages it. First a freeze on further old-schema changes and a published migration rule mapping each old event shape to its new form. Then a dual-write phase: incoming events are written under both schemas at once, so nothing is lost while consumers are moved. Then a staged cutover: dashboards and queries are repointed to the new events one group at a time, each verified before the next. Throughout, a continuity commitment holds that historical reporting must remain comparable, so a compatibility view lets old queries keep returning correct numbers. Finally, a reconciliation monitor compares row counts and key aggregates between old and new for the overlap window; only when they match within tolerance is the old schema retired and dual-write switched off.
How it works¶
The workflow's distinctive content is its ordering and its overlap phase, which are what let a live system change without a broken interval:
- Freeze and map. Stop further drift in the old schema and publish the migration rule that defines each old-to-new transformation.
- Run both in parallel. Dual-write (and/or dual-read) so the old and new schemas coexist, decoupling data movement from consumer movement.
- Cut over in stages. Repoint consumers group by group with a verify-and-proceed gate, never all at once, so a fault is caught small.
- Reconcile, then retire. Compare counts and aggregates across the overlap window; retire the old schema only after the numbers match within tolerance, keeping rollback available until they do.
Tuning parameters¶
- Overlap-window length — how long both schemas run in parallel; longer is safer and gives more reconciliation evidence but doubles write cost and complexity.
- Cutover granularity — how finely consumers are moved (one dashboard at a time vs. all at once); finer catches faults early but stretches the migration.
- Reconciliation tolerance — how exact the old-vs-new match must be before retirement; tight tolerance catches subtle loss but can block on benign rounding.
- Rollback horizon — how long the ability to revert is kept; a longer horizon is safer but delays decommissioning the old schema.
When it helps, and when it misleads¶
Its strength is that it lets a schema change ship into a running system with continuity preserved at every step and a proof — the reconciliation — that the crossing was clean. Its whole safety rests on the expand-and-contract (parallel-change) pattern: first add the new shape alongside the old and run both, then migrate consumers, then remove the old, so there is never a moment when the system depends on a half-applied change.[n1]
Its failure mode is silent loss at the seam: a migration that "completes" while a fraction of records fell through a gap in the mapping, undetected because nobody reconciled counts across the overlap. The classic misuse is skipping the parallel phase to save time — a big-bang cutover that works in staging and mangles production the moment real concurrent writes hit the seam. The guarding discipline is to insist on the overlap window and the end-of-run reconciliation as non-negotiable gates: retire the old schema only when old and new agree across the window, and keep rollback live until they do.
How it implements the components¶
migration_rule— the published transformation defining how each old data shape maps to the new, executed in ordered phases rather than one pass.continuity_constraint— the commitment that the live system keeps serving reads and preserves historical comparability through every intermediate state.error_monitor— the reconciliation that compares old and new across the overlap window and gates retirement on a clean match.
The workflow orchestrates a crossing but does not decide the new structure or hear stakeholders: it renders no revision_decision (that is Category Split/Merge Review) and stands up no schema_owner governance (that is Schema Change Review Board). Its nearest twin is Knowledge-Base Retagging, with which it shares the migration-rule role; the separator is that retagging is the relabeling act on stored content and holds the backward_compatibility_map and user_transition_path, whereas this is the staged, verifiable process that moves a live system and may invoke a retag as one of its phases.
Related¶
- Instantiates: Schema Update Protocol — the workflow is the carry-a-live-system-across step that preserves continuity and proves the crossing.
- Compare Schema Migration: this workflow is reconciliation-gated release orchestration — the dual-run/overlap window plus a verified cutover gate — whereas Schema Migration is the dependency-map-scoped data-transition procedure that determines what must move and in what order.
- Consumes: Knowledge-Base Retagging can serve as the relabeling operation the workflow schedules within its dual-run and cutover phases.
- Sibling mechanisms: Classification Audit · Category Split/Merge Review · Coding-Frame Revision · Glossary Update · Knowledge-Base Retagging · Ontology Refactoring · Schema Change Review Board · Taxonomy Revision
Editorial Notes¶
Form Classification¶
Form family: Protocol, Workflow & Routine
Rationale: Schema Migration Workflow operates as a repeatable ordered procedure or handoff sequence that coordinates action because it sequences a schema change into a releasable process — freeze, dual-run, cut over, verify — so live data crosses to the new structure without a period where old and new are both broken.
Independent corroboration: The frozen evidence defines Schema Migration Workflow as 'Sequences a schema change into a releasable process — freeze, dual-run, cut over, verify — so live data crosses to the new structure without a period where old and new are both broken', so its operative form is Protocol, Workflow & Routine.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Freeze, dual-run, cutover, and verify are established live schema-migration stages.
Related originating lineages:
- Engineering & Design — Engineering design, reliability, and systems-safety practice supplies a parallel or contributing lineage for the mechanism's defining operation: sequences a schema change into a releasable process — freeze, dual-run, cut over, verify — so live data crosses to the new structure without a period where old and new are both broken.
Review resolution: Both blind reviewers agree that computer_science is the primary historical origin. Explicit reconciliation of alternate_origin_disagreement starts from reviewer_a's mechanism-specific evidence: Freeze, dual-run, cutover, and verify are established live schema-migration stages. Reviewer A proposed alternates=none, origin_mode=single_lineage, domain_reach=specialized, and encyclopedia_synthesis=false; reviewer B proposed alternates=engineering_design, origin_mode=single_lineage, domain_reach=specialized, and encyclopedia_synthesis=false. The final record retains every independently supported alternate from either review (engineering_design) without an arbitrary cap, selects origin_mode=single_lineage to represent the combined lineage evidence, and records domain_reach=specialized and encyclopedia_synthesis=false. Present-day transfer is recorded as reach and is not treated as proof of historical origin.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Expand-and-contract (also called parallel change) is a migration pattern in which the new structure is first added alongside the old and both are maintained, then consumers are moved, then the old is removed — so the system never depends on a half-completed change. It is the standard technique behind zero-downtime schema and interface migrations, described by Fowler and Sato among others. ↩