The scariest one-line diff I've ever written was a column rename. It compiled. The tests passed. And if I'd shipped it, every row already sitting in the production table would have stopped decoding the instant the new code deployed — because the column those rows were written under no longer existed in the struct reading them back. Records don't error politely when the shape changes underneath them. They just disappear.
I've since led a big refactor of a live, stateful service — renaming its primary key, collapsing two fields into one, and deleting a whole layer of the stack — without ever leaving a required caller unable to read its own data. It ran as a sequence of small, independently shippable steps instead of one heroic merge. This is the playbook, because the ordering is the whole lesson, and it generalizes to any migration where the data outlives the deploy.
The mental model: a ratchet, not a checklist
The sequence I keep coming back to:
delete dead code → change the wire → migrate the store → delete the scaffolding → reroute callers → add features
Each arrow is "this must be true before that can safely start." It's a ratchet: every step makes the next one's blast radius smaller. Three of these moves are the ones worth internalizing.
Move 1: delete the dead code first — but prove it's dead
Before touching anything live, I deleted code that looked wired but wasn't: a service layer whose main method returned "not found" on every call and had zero callers, and a background sync loop that only ever ran in local dev because production never wrote the file it watched. It had been a silent no-op in prod for its entire life.
The rule isn't "delete boldly." It's prove death before the funeral — grep for callers, trace the production path, confirm the dead return. Dead code that survives into a migration isn't neutral; it's one more shape your migration has to be correct against, for zero benefit. Removing it first meant the real work had one code path to reason about instead of three.
Move 2: change the contract before you move the data
This is the one people get backwards. I changed the API contract — the wire types every consumer speaks — a full step before touching a single stored row.
Why not migrate the data first? Because if the store changes shape while every reader is still typed to the old contract, the readers choke on the new rows. Flip it: change the wire first, regenerate the client stubs, update every consumer to speak the new shape, ship it. Now the whole system can hold the new shape even though the store is still emitting the old one. The data catches up in the next step, and there's a ready audience waiting for it.
A small durable habit from this: when you retire a serialized field, don't just delete its tag — reserve it, so a future field can't accidentally claim the same slot and decode old bytes into a new, wrong meaning. One line, and it saves someone a genuinely confusing afternoon years later.
Move 3: the dual-shape store
Here's the pattern I reach for whenever a live schema has to change. The store passes through four states:
State 1 — read both, write canonical. The table understands both the new columns and the old ones, so existing rows still decode. But the writer only ever emits the new shape. The reader prefers the new field and falls back to the old:
// Reader is permissive: new shape wins, old shape is the fallback. func DecodeRecord(doc map[string]any) (Record, error) { var r Record // new key → old key → derive from the record id (oldest rows had neither) if v, ok := doc["tenant_id"].(string); ok && v != "" { r.TenantID = v } else if v, ok := doc["owner_id"].(string); ok && v != "" { r.TenantID = v // legacy field } else { r.TenantID = idPrefix(doc["_id"]) } // two old loose strings collapsed into one typed id if v, ok := doc["kind_id"].(string); ok && v != "" { r.KindID = v } else if grp, ok := doc["group"].(string); ok { r.KindID = grp + "/" + str(doc["sub"]) // reconstruct from the pair } if r.TenantID == "" || r.KindID == "" { return r, fmt.Errorf("record %v: undecodable in either shape", doc["_id"]) } return r, nil } // Writer is strict: canonical only. Legacy keys are never emitted again. func (r Record) Document() map[string]any { return map[string]any{ "tenant_id": r.TenantID, "kind_id": r.KindID, // old owner_id / group / sub intentionally omitted } }
The asymmetry is the design: reads are permissive, writes are strict. Any time an old row gets touched for any reason, it silently upgrades to the new shape on write-back. The dataset heals itself as normal traffic flows through it.
State 2 — migrate out-of-band. A one-shot job walks the table, decodes each row, and rewrites the stragglers that ordinary traffic hasn't touched yet. It's idempotent — it only rewrites a row when the stored shape differs from what the strict writer would produce — so you can run it as many times as you like.
State 3 — delete the fallbacks, a step later. Once every row is canonical, the fallback branches and the migrator are dead weight. Delete them. A migration tool that sticks around is a liability: eventually someone runs it against a database it no longer understands.
So the ladder is read-both → write-canonical → migrate → delete-fallbacks. At no instant is a required row unreadable, and at no instant is the old shape both required and unsupported. That second property is the one that lets you sleep.
The same shape shows up in security cutovers
This ladder isn't just for schema. I've used the identical sequencing to tighten a security boundary — flipping a check from "warn and allow" to "reject."
There was an isolation check that, on a mismatch, logged a warning and then served the request anyway. Flipping it straight to a hard rejection would have broken every caller quietly relying on the leniency. So instead: enforce-but-tolerate → migrate callers → remove tolerance. Make the strict check load-bearing, but also keep widening the parser to accept the old inputs; migrate every caller to the strict shape behind that tolerance; add regression tests pinning the invariant; and only then strip the tolerant branches and lock strict-mode on.
The invariant is identical to the schema case: nothing is ever both required and unmigrated at the same time. Whether you're renaming a column or hardening an auth check, the removal of the old path always comes last.
The corollary: if a layer only reshapes data, it's a liability
Midway through, I also deleted an entire layer — a per-consumer wrapper that translated a platform contract into consumer-shaped messages. It had produced a steady drip of bugs, and they were all the same bug: a translation layer drifts from the contract it fronts. A field silently dropped in conversion here, a wrong id substituted there. Rather than fix each one, I deleted the wrapper and replaced it with a tiny content-blind reverse proxy that does auth and routing and forwards the request verbatim.
The test I now apply to any layer like this:
If a layer only reshapes data — renames fields, restructures bodies — it's a liability. A layer that earns its place does auth, routing, or request-shaping and is otherwise replaceable by curl. If you can reproduce it with a shell one-liner and a header, it's pulling its weight; if it's rewriting payloads, it's a bug waiting to happen.
What to steal
- —Sequence the work so each step shrinks the next one's blast radius. Delete dead code → wire → store → callers → features. It ratchets.
- —Change the contract before you move the data. Consumers must be able to hold the new shape before any data arrives in it.
- —Dual-shape reads, canonical-only writes, out-of-band migrate, delete fallbacks a step later. Reads permissive, writes strict — the dataset heals as it runs.
- —For security cutovers, enforce-but-tolerate → migrate → remove tolerance. Same invariant as a schema migration: never leave the old path both required and unsupported.
- —Reserve retired serialized-field tags so old bytes can't decode into a new meaning.
- —A layer that only reshapes data is a liability. The test is "replaceable by curl." Delete the reshaping veneer; keep the thin proxy.
The version of you that's scared of the one-line rename is right to be scared. The fix isn't more courage. It's more phases.