Data Migration Runbook¶
Execution runbook — instantiates Creative Destruction Management
The executable, step-by-step procedure for moving records off the old store — extract, transform, validate, cut over, and roll back — with every step reversible and audited.
A Data Migration Runbook is the ground-level operating procedure that actually moves the records when an incumbent store is retired. Where a program plan decides that a system will be replaced and when, the runbook is the numbered sequence a named operator executes on cutover night: extract from the legacy store, transform to the new schema, validate row counts and checksums, flip the switch, and — if a check fails — roll back to a known-good state without data loss. Its defining commitment is reversibility of the data move itself: every destructive step is preceded by a snapshot and paired with an abort path, and the run does not begin until pre-declared readiness checks pass. It is a document you can hand to a person at 2 a.m. and expect them to execute, verify, and — if needed — undo.
Example¶
A retail bank is retiring a 1980s mainframe deposit ledger for a new core banking platform. The go/no-go for the whole program was settled months earlier; tonight's question is narrower and sharper — can twelve million account records move in a four-hour window without a single balance going wrong? The runbook opens with a readiness gate: source ledger frozen, new schema deployed, last night's trial run reconciled to zero variance, and the rollback snapshot verified restorable. Only when all four are green does step one begin.
The body is a checklist with owners and timestamps: extract balances and transaction history, transform account numbers to the new format, load into staging, and run a reconciliation that must match the mainframe's control totals to the penny before any customer-facing traffic is redirected. Midway, the transaction-history transform throws a variance of $1,900 across eleven dormant accounts. Because the runbook made this branch explicit — variance over $0 halts cutover — the operator does not improvise; she triggers the documented rollback, restores the snapshot, and the bank opens the next morning on the old ledger with no customer the wiser. The failed batch becomes a defect to fix before the next attempt, and the frozen mainframe image is retained as a read-only archive so any future dispute can be reconciled against the original.
How it works¶
The runbook is distinguished from a generic project plan by four properties that only an execution procedure has:
- Rehearsed on a copy first. The whole sequence is dry-run against a clone until it reconciles to zero variance; the production run is a repeat of a proven script, not a first attempt.
- Every step is idempotent or snapshotted. Each destructive action is preceded by a restorable checkpoint, so any step can be re-run or reversed without corrupting state.
- Validation is a pass/fail gate, not a report. Reconciliation totals, row counts, and checksums have declared thresholds; a miss halts the run rather than being noted for later.
- The old data is frozen, not deleted. At cutover the source becomes a read-only image retained for audit and dispute resolution, so retirement never means erasure.
Tuning parameters¶
- Cutover style — big-bang (one window) versus trickle/dual-write (records migrate continuously while both stores serve traffic). Big-bang is simpler to reason about but has a scarier rollback; trickle lowers risk at the cost of running reconciliation for weeks.
- Reconciliation tolerance — the variance that halts the run. Zero tolerance is safest for financial records but can block cutover on benign rounding; a small band moves faster but risks masking a real defect.
- Rollback horizon — how long after cutover the snapshot stays restorable. A longer horizon buys safety but forces you to keep two worlds reconciled and delays freeing the old hardware.
- Archive retention — how long, and in what fidelity, the frozen source is kept. Deeper retention protects against late disputes but carries storage and compliance cost.
When it helps, and when it misleads¶
Its strength is that it turns the single most dangerous moment of a replacement — the irreversible instant records leave their old home — into a rehearsed, checkable, reversible operation. When it holds, a botched cutover costs a night rather than a franchise. Runbooks pair naturally with a gradual strangler fig approach,[n1] where the new system takes over one slice at a time and each slice's cutover is its own small, reversible run.
Its failure mode is false confidence in a script that was rehearsed against unrepresentative data: the dry run reconciles because the test copy lacks the messy dormant accounts, the malformed legacy rows, the once-a-year edge case — and the surprise arrives only in production. The classic misuse is treating the runbook as the migration when it is only the data-move layer: a flawless technical cutover onto a platform nobody was ready to use is still a failed transition. The guarding discipline is to rehearse against a full-fidelity production clone (not a tidy sample), keep the rollback path live until reconciliation has survived real traffic, and never let a green runbook stand in for the readiness of users and workflows above it.
How it implements the components¶
migration_path— the runbook is the concrete route the records travel: extract → transform → validate → load → cut over, step by numbered step.rollback_or_contingency_rule— each destructive step is snapshotted and paired with a pre-declared abort trigger, so any failed check returns the system to a known-good state.transition_readiness_criteria— the opening go/no-go gate encodes the measurable conditions (frozen source, zero-variance trial, restorable snapshot) that must be true before the run starts.residual_legacy_archive— at cutover the source store is frozen into a read-only image retained for audit and dispute reconciliation.
It does not argue whether the platform is worth replacing (replacement_value_case) or chart the program-level dependency and phasing across users and integrations (dependency_and_compatibility_map) — that scaffolding is the job of its nearest twin, Technology Migration Plan; the runbook is the executable data-cutover it schedules.
Related¶
- Instantiates: Creative Destruction Management — supplies the reversible, audited record-move at the heart of a governed replacement.
- Sibling mechanisms: Technology Migration Plan · Deprecation Program · Infrastructure Replacement Program · Legacy Support Window · Policy Phase-Out Schedule · Product Sunset Plan · Stakeholder Transition Workshop · Workforce Transition Support
Editorial Notes¶
Form Classification¶
Form family: Protocol, Workflow & Routine
Rationale: Data Migration Runbook operates as a repeatable ordered procedure or handoff sequence that coordinates action because it the executable, step-by-step procedure for moving records off the old store — extract, transform, validate, cut over, and roll back — with every step reversible and audited.
Independent corroboration: The frozen evidence defines Data Migration Runbook as 'The executable, step-by-step procedure for moving records off the old store — extract, transform, validate, cut over, and roll back — with every step reversible and audited', so its operative form is Protocol, Workflow & Routine.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Database and IT operations cohered migration runbooks sequencing extract, transform, load, validation, cutover, rollback, and reconciliation around known-good snapshots.
Review resolution: Systems and database operations established rehearsed migration runbooks with cutover, verification, and rollback; data science is later applicability.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] The strangler fig pattern (named by Martin Fowler after the vine that gradually envelops a host tree) migrates a system incrementally — the new implementation grows around the old, taking over one capability at a time — until the legacy system can be removed. Each increment's data move is a natural unit for a small, reversible runbook. ↩