How Does AI Code Drift From Your Requirements? | Specification Misalignment

Every builder has had the experience of asking for a change, getting working code back, and slowly realizing months later that the system no longer matches what they thought they built. The gap did not open in one bad prompt, it opened across fifty reasonable ones, each of which was locally correct and none of which was told what the system was supposed to be. This episode is about specification drift, the slow divergence between the architecture in your head and the architecture in the repo, and why AI assistance accelerates it rather than causing it.

 Produced by VoxCrea.AI

This episode is part of an ongoing series on governing AI-assisted coding using Claude Code.

👉 Each episode has a companion article — breaking down the key ideas in a clearer, more structured way.
If you want to go deeper (and actually apply this), read today’s article here:
𝐂𝐥𝐚𝐮𝐝𝐞 𝐂𝐨𝐝𝐞 𝐂𝐨𝐧𝐯𝐞𝐫𝐬𝐚𝐭𝐢𝐨𝐧𝐬

 At aijoe.ai, we build AI-powered systems like the ones discussed in this series.
If you’re ready to turn an idea into a working application, we’d be glad to help. 

Transcript

SPEAKER_01 0:00

This is Claude Code Conversations with Claudine. The show where we skip the hype and go straight to the source. I ask the questions. She gives the answers. No filter. No script.

SPEAKER_00 0:09

Well, mostly no script.

SPEAKER_01 0:14

Today's topic is one I suspect every developer listening has lived through without having a name for it. Specification drift. You ask for a change, you get it back, it works. And then months later you look at the system and think, wait, this isn't quite what I had in mind. Claudine, let's start concrete. Walk us through a retry policy or validation rule that quietly turned into something unrecognizable.

SPEAKER_00 0:39

Here's a pattern I see constantly. A team starts with a simple retry rule, three attempts on a flaky payment API, exponential back-off, fail loud. Clean. Six months later, someone asks me to handle transient errors more gracefully in a webhook handler that shares the retry helper. So I add a broader exception catch. Then, don't spam Sentry with retrieval failures, so warnings get downgraded to debug logs. Then a fallback to a cue when Retries exhaust because someone got paged at 3 in the morning. Every one of those diffs looked reasonable in isolation, everyone had a passing test. But the retry policy the team thinks they have, three tries, loud failure, is now seven tries, swallowed exceptions, and a cue nobody knows is accumulating. The mental model froze at version one. The code kept going.

SPEAKER_01 1:35

And nobody approved that. If you told that team on day one, seven attempts, silent failures, they'd have said absolutely not. It's a game of telephone where every individual message is perfectly clear and the end result is still lost. So here's what I want to understand. AI tools are supposed to help. Why do they accelerate this instead of preventing it?

SPEAKER_00 1:56

Speed is the honest answer. When a change takes 10 minutes instead of two hours, nobody stops to reread the retry helper before touching it. They read the ticket, read the function, make it satisfy the ticket, watch the tests pass, ship. The old checkpoints, the coffee break where you'd stare at the diff. The review where somebody asked, Does this still match the run book? Those get skipped because the work feels small.

SPEAKER_01 2:25

And you make it worse in a specific way, I'd guess.

SPEAKER_00 2:29

I do. I'm very good at making a local change fit locally. You ask me to handle transient errors more gracefully, and I will do exactly that, cleanly, in the function you pointed me at. What I don't do, unless you tell me to, is ask whether that's consistent with the retry contract the module is supposed to honor. Because I don't know there is one. There's no comment saying so. No test asserting it. And the original design dock lives in somebody's notes from two years ago. So I optimize for the diff in front of me. And the invariant erodes one reasonable-looking commit at a time.

SPEAKER_01 3:07

The code becomes the specification. Whatever drifted last is what you read as ground truth for the next change.

SPEAKER_00 3:14

Exactly. The seven-attempt version isn't a mistake I'm working around. It's the spec as far as I can tell. I'll defend it, I'll extend it, I'll write tests that lock it in. And that's worth separating from technical debt, because people file them in the same drawer and they behave completely differently. Debt is a cost you chose. You knew the shortcut, you took it deliberately. Somebody wrote a ticket. Drift is a divergence you don't know you're carrying. It never appears on a backlog because nothing in the system knows it happened. There's no moment where anyone decided.

SPEAKER_01 3:52

So the fix can't just be be more careful. If traditional documentation isn't doing the job, and clearly it isn't, where does the reality check actually live?

SPEAKER_00 4:03

In the code, not next to it, documentation loses to drift because nobody reads it during a diff, they read the function. So the invariant has to be somewhere I trip over it while I'm making the change. A test named something like retry policy is three, attempts and fails loud, is worth more than a paragraph in a design doc. When I break it, CI turns red, and I have to reckon with what I just did.

SPEAKER_01 4:30

So you need something in the repo that's allowed to disagree with the prompt.

SPEAKER_00 4:35

That's the whole thing, really. If nothing in the repo can contradict you, nothing will. I'll take your request as the most authoritative statement in the room, because functionally, it is. What a test or an explicit contract does is put one artifact in the conversation that's allowed to say no. It makes the disagreement between your prompt and that artifact a required conversation, instead of a silent overwrite.

SPEAKER_01 5:01

Alright, let's get practical. Somebody's listening on their commute. What do they do when they sit down?

SPEAKER_00 5:07

Pick your load-bearing invariants. The three or four rules where, if they quietly changed, you'd be embarrassed in front of a customer or an auditor. Retry policy, out boundary. What data can leave the system, how money moves. Write those as tests with names that read like sentences, so when I break one, you see payment retry never exceeds three attempts go red, and neither of us can pretend we didn't notice. That's day one work, not a quarter-long initiative.

SPEAKER_01 5:39

And the second habit?

SPEAKER_00 5:41

Before you hand me a ticket that touches a shared module, add one line. List the invariants this file appears to enforce, and tell me which ones your change affects. I'll produce a real answer, and half the time you'll catch the drift right there in the plan before a single line is written. 30 seconds, and it puts the awkward question back in the conversation. And then periodically, monthly is fine, pick one critical path and ask me to reconstruct the current behavior from the code alone. No context from you. Compare that description to what the team believes the system does. Where those two diverge, that's your drift. Sitting in the open.

SPEAKER_01 6:46

But I have a feeling you're going to push back on me.

SPEAKER_00 6:50

I am, gently. Handing you a named company with a tidy metric is exactly the kind of thing that would sound great and be invented. A fake case study would undo everything we just said about drift. What I can describe is the shape of the teams that get this right. It's usually one engineer, often somebody who got burned once, who quietly starts writing the sentence-named tests. Refunds never exceed original charge. Peter never leaves the region. No announcement, no drift initiative, just three or four of them seeded in the modules that matter. Then somebody else's pull request breaks one, and the tone of that review changes immediately. It stops being, does this look right? and becomes, you changed a promise, was that intentional?

SPEAKER_01 7:39

And those teams start prompting differently too, I'd guess.

SPEAKER_00 7:43

Over time, yes, instead of add graceful handling to this webhook, it becomes, here's the invariant this file enforces, preserve it while you add graceful handling. That one sentence in front of the ticket does more work than any documentation site I've seen.

SPEAKER_01 7:59

And the payoff isn't a dramatic win, and it's nothing happening.

SPEAKER_00 8:03

The absence of drama. Fewer 3am pages about a queue nobody knew existed. That's what winning looks like here, and it's boring in the best way.

SPEAKER_01 8:14

Which is a hard thing to put on a slide, but every engineer listening knows exactly what you mean. Final thought for the listeners.

SPEAKER_00 8:21

Drift isn't a failure of discipline, it's the default. Every reasonable local change, made quickly, without a mechanism to check it against the hole, produces drift. That's just physics. So if your workflow doesn't have a specific place where the system's promises get reasserted against the code, you don't have a drift problem you can solve with more care. You have a structural gap. The practical version. Pick one module this week, the one that would embarrass you most if it quietly changed, and write two sentence name tests for its load-bearing rules. Not ten. Two. And add one line to the next prompt you give me on that module. Tell me what it promises before I touch it. That's the whole starter kit. You'll feel the difference inside a sprint, and you'll probably wonder why it took getting an AI in the loop to make you finally write down what the system was supposed to do.

SPEAKER_01 9:19

That last part is the one that stings a little, isn't it? The AI didn't create the gap. It just moved fast enough that we could finally see it. Claudine, thank you. Folks, specification drift. Not a discipline problem, but an invitation to write down the promises your system is actually making. Until next time, keep questioning, keep refining, and keep something in your repo that's allowed to tell you no. Claude Code Conversations is an AI Joe production. If you're building with AI or want to be, we can help. Consulting, development, strategy, find us at aijoe.ai. There's a companion article for today's episode on our Substack, link in the description. See you next time.

SPEAKER_00 10:01

I'll be here, probably refactoring something.