Migration Regret: The Assumptions That Turn a Data Move Into a Multi-Month Crisis
Photo by Photo by Growtika on Unsplash on Unsplash
Somewhere right now, a team of engineers is staring at a migration script that was supposed to finish over the weekend. It is Wednesday. The script is still running. Nobody is entirely sure why.
Data migrations have a reputation for going sideways, and that reputation is fully deserved. But the interesting question isn't whether they go wrong — it's where the failure actually originates. Because in most post-mortems, the technology is rarely the root cause. The database didn't betray anyone. The cloud provider didn't introduce a surprise. The failure almost always traces back to an assumption someone made in the planning phase that seemed reasonable at the time and turned out to be completely wrong.
The Assumption Graveyard
Every data migration starts with a set of beliefs about the data itself. How much there is. How clean it is. How consistent the schema has been over time. How well the source system's documentation matches what's actually in the tables.
These beliefs are almost always optimistic, and they are almost never validated before the migration begins.
One pattern that shows up constantly: a team migrating a legacy SaaS platform to a new cloud-native architecture assumes the data model has been stable for years. It has, technically — but "stable" turns out to mean that nobody deleted old columns when the schema evolved. They just stopped writing to them. The migration script treats those columns as active. The new system ingests values that haven't been populated since 2019 and interprets nulls as meaningful state. Downstream logic breaks in ways that are genuinely hard to trace because the data looks valid.
Another common one: assuming referential integrity in a database that was never enforced at the constraint level. The application code maintained consistency. The database did not. Move the data without the application logic, and you discover orphaned records at a scale that wasn't visible in any schema diagram.
Dual-System Purgatory
When migrations hit trouble, the default fallback is to run dual systems — keep the old platform live while the new one catches up. This feels like a safe, measured approach. In practice, it creates a category of problems that can be worse than either the original system or the new one in isolation.
Dual-system operation means every write has to land in two places, or you need a sync process that keeps them aligned. That sync process becomes the most fragile component in your entire infrastructure. It has to handle schema differences, timing gaps, conflict resolution, and error recovery — all while your actual product traffic is flowing through both systems simultaneously.
Teams routinely underestimate how long they'll live in this state. A two-week dual-system window stretches to six weeks. Then three months. The sync layer accumulates workarounds. The team that built it moves on to other projects. Nobody fully understands it anymore, but everyone is afraid to turn it off because they're not confident the new system is actually correct.
This is how migrations become permanent. Not because anyone decided to keep the old system — but because the off-ramp never got built.
The Compliance Dimension Nobody Scoped
For companies operating in regulated industries — healthcare, fintech, anything touching consumer financial data — data migrations carry a compliance surface area that frequently gets scoped as an afterthought.
The assumption is usually something like: "We're just moving the data, not changing it, so compliance requirements don't change." This is wrong in ways that can be genuinely expensive. Moving data across regional boundaries can trigger GDPR or CCPA obligations that didn't apply to the original system's location. Changing the storage provider can affect your SOC 2 or HIPAA Business Associate Agreement status. Altering the data model, even slightly, can require re-validation under frameworks that took months to certify the first time.
The teams that get blindsided by this aren't being careless. They're just scoping the migration as a technical project when it's actually a cross-functional one. Legal, compliance, and security need to be in the room before the first planning document is written — not after the migration is already in flight.
A Decision Framework That Actually Holds Up
Given how reliably migrations go wrong, the most useful thing a technical leader can do is slow down the decision to migrate in the first place. Not indefinitely — but long enough to honestly answer a few questions that tend to get skipped in the rush to modernize.
Is the migration solving a real constraint, or a theoretical one? If the existing system is slow, expensive, or genuinely limiting, migration is probably worth the pain. If the driver is "the new architecture is cleaner," that's a much weaker justification for the risk involved.
Have you actually profiled the source data? Not the schema — the data. Run counts, check for nulls, look for values that fall outside expected ranges, identify records that would fail validation in the new system. Do this before writing a single migration script.
What does rollback look like, and have you tested it? Rollback plans that exist only in a document are not rollback plans. If you can't demonstrate a rollback in a staging environment, you don't have one.
Is there a version of this that doesn't require dual systems? Sometimes the answer is a hard cutover with a maintenance window. That's operationally uncomfortable but often cleaner than months of sync-layer maintenance. The discomfort is finite. The sync layer is not.
Who needs to sign off beyond engineering? Legal, compliance, customer success, and sometimes your largest customers themselves — depending on what's moving and how. Get them in early.
The Redesign Option Nobody Wants to Discuss
Occasionally, the right answer to a data migration isn't to migrate at all. It's to redesign the data flows so that the old data becomes less relevant over time — new writes go to the new system, old data is accessed through a read layer that abstracts the source, and the migration happens gradually as users naturally interact with the platform.
This approach is slower and less satisfying than a clean cutover. It also tends to be dramatically less risky. For platforms with years of messy historical data, it's worth at least putting on the table before committing to a bulk migration that assumes the data is cleaner than it is.
Migrations fail at the assumption layer. The fix isn't better tooling or more engineering hours — it's a genuine reckoning with what you don't know about your own data before you start moving it.
