AI Knowledge Hub

How should Product and Engineering plan a data migration for a product change?

Quick answer

Plan a product data migration by defining the user and business meaning of the data, mapping old and new records, rehearsing the move, and agreeing validation and cutover criteria. Product owns the consequences for users and priority; Engineering owns the migration design, integrity checks and recovery route. Treat rollback carefully once users can write to the new system.

What to remember

Key takeaways

Define what must remain correct for users and downstream processes.
Rehearse with representative data and check exceptions, not only record counts.
Separate code rollback from data recovery after new writes begin.
Name the cutover decision owner and a route for unresolved records.

A feature can be ready while its existing data is not. A new account model, pricing structure or workflow may require records to change shape or location. If the team treats that move as a background script, users can encounter missing history or incorrect decisions on release day.

Define the migration contract

A migration contract describes which records move, which fields change meaning, what must remain visible to users and which downstream consumers rely on the data. Product explains the intended behaviour and the cost of incomplete or delayed records. Engineering maps sources, transformations, ownership and integrity constraints. Include records with missing or ambiguous values in the plan rather than quietly discarding them.

Use established migration steps

Teams have long used mapping documents, dry runs, backups and reconciliation reports. For a small, reversible change, a tested one-off migration may suffice. Where old and new application versions coexist, a staged schema change can preserve compatibility while records and readers move. Count comparisons help, but they cannot prove that an amount, owner or date still means the right thing. Sample critical journeys and edge cases with domain experts.

Move in controlled stages

Start with a representative sample, then rehearse the full sequence in a safe environment. Measure duration, error types and the ability to resume. Consider a backfill followed by a short period of synchronisation if the product must stay live. Product and Engineering jointly set acceptance thresholds: for example, which exceptions can wait, which block cutover and what users should see during the transition. AI may help draft mappings or surface anomalous records, but a domain owner must confirm meaning and Engineering must validate transformations.

Agree cutover and recovery

Write down the cutover order, authorised decision makers, monitoring signals and communication plan. Test what happens when a batch fails or a downstream consumer lags. A code rollback does not automatically restore data once new writes have occurred; decide beforehand whether to restore, reconcile or fix forward. After cutover, compare source and target, exercise real user journeys, assign exception owners and retire old paths only when use and integrity checks support it.

Example

Hypothetically, a subscription team changes from one billing contact per account to several authorised contacts. Product defines which person must receive invoices and how users can correct an uncertain match. Engineering maps legacy contacts, rehearses a backfill, checks invoices and records unmapped accounts. The team pauses cutover if a critical invoice route is wrong, and assigns manual review for the remaining ambiguous accounts.

FAQs

Is matching record counts enough?

No. Counts detect omissions and duplication, but field meaning and relationships also need checks against representative user tasks.
Can a team always roll back after cutover?

No. Once new data has been written, the old store may be stale. Plan recovery for that point explicitly.
Who accepts ambiguous records?

Product or a named domain owner decides the business treatment; Engineering records and implements that treatment safely.