Skip to content

Safe Consistency Migration Workflow

Migration workflow — instantiates Shared-State Consistency Contract Design

Changes a live system's consistency semantics in reversible, observable stages — dual paths, shadow comparison, staged cutover — instead of flipping the guarantee as a silent config change.

Safe Consistency Migration Workflow is the procedure for changing what a running system promises about shared state — eventual to causal, last-write-wins to a merge type, single-region to multi-region — without breaking clients who depend on the old promise. Its defining stance is that a semantics change is a migration, not a toggle: it must be staged, reversible at each step, observed while both behaviors run in parallel, and recorded so that "what did a successful operation mean on this date" is always answerable. This directly counters the archetype's named failure — deploying a semantic change as an ordinary configuration flip, where clients silently start relying on a guarantee that quietly changed underneath them.

Example

A store keeps shopping carts in a last-write-wins register, which silently drops an item when a customer adds from two devices at once. The team wants to move to an add-wins CRDT set that keeps both. Rather than flip the storage type, they run the migration in stages: (1) dual-write — every cart mutation writes both the old register and the new CRDT; (2) shadow-read and compare — reads still serve the old value, but an oracle compares what the CRDT would have returned and logs divergences; (3) staged cutover — reads move to the CRDT one customer cohort at a time behind a flag, each cohort independently rollback-able; (4) backfill historical carts; (5) retire the old path. The audit trail records the exact moment each cohort's semantics changed, so a later dispute ("the cart lost an item on the 3rd") can be answered against the semantics actually in force then. When a cohort's divergence spikes, the flag rolls back with no customer impact.

How it works

The distinguishing structure is expand/contract applied to semantics: add the new behavior alongside the old, migrate consumers gradually, then remove the old — so no single step is a breaking flip.[1] A comparison oracle runs during the shadow stage to catch client-visible behavior changes before any cutover, and every stage transition is reversible and recorded. The workflow does not decide which new guarantee to adopt (that is a design decision made elsewhere); it is the machinery for getting there safely on live state.

Tuning parameters

  • Stage granularity / cohort size — small cohorts shrink blast radius but lengthen the migration.
  • Shadow duration & compare threshold — longer shadowing and tighter thresholds raise confidence at the cost of time.
  • Rollback trigger — how aggressively a divergence or SLO breach auto-reverts a cohort: safety vs. churn.
  • Path strategy — dual-write, dual-read, or both, chosen to match the kind of change (write-semantics vs. read-semantics).
  • Backfill strategy — whether and how historical data is converted, trading completeness against load.

When it helps, and when it misleads

Its strength is turning a scary, seemingly-irreversible semantics flip into an evidence-gated, reversible rollout that is auditable after the fact and catches surprises in shadow before customers feel them.

It misleads when the dual-run stages are themselves mishandled. Dual-writes that are not idempotent or correctly ordered can introduce the very inconsistency the migration is meant to fix; long parallel-run windows are costly and tempt teams to skip the comparison. The backwards version is flipping the flag first and writing the migration story afterward — precisely the silent-config-change anti-pattern in disguise. The discipline is to never change semantics as a bare config value: keep the old path live until the shadow comparison is clean, and record every cutover.

How it implements the components

  • consistency_migration_plan — the staged, reversible plan (dual-path, shadow, cohorted cutover, backfill, retire) that carries a live system from one guarantee to another.
  • audit_trail — the durable record of which semantics was in effect for whom and when, so a change is never silent and past behavior stays explainable.

It does not choose the target guarantee (consistency_guarantee_profile, client_facing_semantics_contract) — that is Consistency Contract Decision Record — nor supply the shadow-stage comparison oracle (Sequential-Consistency Trace Protocol / Consistency History Checker) or the rollout dashboards (Replica-Lag and Freshness Dashboard).

Notes

The migration's own machinery can be a source of inconsistency: a dual-write is a two-target write, so unless each side is idempotent and ordered, a partial failure leaves the two representations disagreeing. Treat the dual-write path as its own small consistency problem — reconcilable on retry — rather than assuming it is safe because it is temporary.

References

[1] Expand/contract (a.k.a. parallel change, Fowler) — evolve an interface by first adding the new form, migrating consumers, then removing the old, so no step is a breaking change. Here it is applied to consistency semantics rather than to a schema or API. ↩