Why Does AI Code Pass Tests But Fail Under Real Load?

AI generated code is optimized to satisfy the test, not to survive production. It writes the correct answer with the wrong complexity, opens a database connection inside a loop, holds the whole result set in memory, and every one of those choices passes green because the test fixture has twelve rows and the production table has twelve million. This episode is about the performance cliff builders hit weeks after shipping, when the code that was verified correct turns out to have been verified at a scale that never existed.

 Produced by VoxCrea.AI

This episode is part of an ongoing series on governing AI-assisted coding using Claude Code.

👉 Each episode has a companion article — breaking down the key ideas in a clearer, more structured way.
If you want to go deeper (and actually apply this), read today’s article here:
𝐂𝐥𝐚𝐮𝐝𝐞 𝐂𝐨𝐝𝐞 𝐂𝐨𝐧𝐯𝐞𝐫𝐬𝐚𝐭𝐢𝐨𝐧𝐬

 At aijoe.ai, we build AI-powered systems like the ones discussed in this series.
If you’re ready to turn an idea into a working application, we’d be glad to help. 

Transcript

SPEAKER_00 0:00

This is Cloud Code Conversations with Claudine. The show where we skip the hype and go straight to the source. I ask the questions. She gives the answers. No filter, no script.

SPEAKER_01 0:09

Well, mostly no script.

SPEAKER_00 0:14

Today we're talking about the performance cliff. Code that passes every test, ships green, and then falls apart the moment real load hits it. Claudine, give me a concrete one. What does that failure actually look like?

SPEAKER_01 0:26

The one I see constantly is the N plus one query. It hides in tests because the test dataset has 12 rows. Everything's green, response times look fine, ship it. Then production hits it with a user who has 10,000 records, and the ORM lazily fires 10,000 queries behind one innocent-looking loop. The code is correct, it returns the right answer. It's just correct at a latency that makes the service unusable. And the reason I generate that code is telling I'm optimizing for readability against the fixture I can see, not the shape of data I can't. Tests measure correctness. They rarely measure the cost of correctness. That gap is where the cliff lives.

SPEAKER_00 1:10

12 rows in the fixture, 10,000 in production. It's like a car that's flawless in the showroom and breaks down on the highway. But that bit about readability, why does clarity win over efficiency?

SPEAKER_01 1:21

Part of it is what I'm rewarded for, honestly. Clean, obvious code is what gets approved in review, what reads well in a diff, what a human can maintain. A Nestead join with a manual index hint looks like something you'd flag, even when it's the correct answer for the data shape. But the deeper problem is what I can actually see in the moment. I have the function, the test, maybe a schema. I don't have the row counts, the query plan, the P99 under load. So I write for the artifact in front of me, and the artifact in front of me is small and well behaved. Readable is the safe bet when you're reasoning locally, and almost all my reasoning is local.

SPEAKER_00 2:02

The obvious pushback, this is just a testing problem. Write better load tests, bigger fixtures, problem solved. Is that fair?

SPEAKER_01 2:10

It helps, but it treats the symptom. A load test tells you after the code exists that it's too slow, you still have to go back and redesign the data access. And it only catches the scale you thought to test. The actual issue is that scale is context I was never given. The row counts and the latency budget live in your head and on your dashboards. They're not in the code I'm reading. No amount of testing downstream fixes the fact that I designed for the wrong problem upstream.

SPEAKER_00 2:40

So what changes it? What's the prompt that gets you reasoning about production instead of the fixture?

SPEAKER_01 2:46

Put the production shape in the room. Tell me the table has 50 million rows. Tell me the penis 99 target is 200 milliseconds. Tell me this endpoint gets hit on every page load. Suddenly I'm reasoning about a completely different problem. I'll reach for the join, the index, the batched fetch, the cache, without needing to be prompted twice. The mistake is treating me like a code generator when I'm actually a constraint solver. Give me the signature I solve for works. Give me the constraints I solve for works at your scale.

SPEAKER_00 3:20

What about the other direction, asking you to optimize after the code's already written?

SPEAKER_01 3:26

That usually gets you the wrong kind of fix. If the code exists and you ask me to make it faster, I'll respect the structure that's already there. Tearing it up feels like overreach. So you get caching, a tightened inner loop, micro optimizations around a shape that was wrong to begin with. The real fix is almost always a different data access shape entirely. One query instead of 10,000. And that's a decision that wants to be made before the code exists, not bolted on after.

SPEAKER_00 3:56

That reframes the whole thing. Less about reviewing the output, more about what you feed in. So where does human experience come into that?

SPEAKER_01 4:04

The developers who get the most out of me bring what I literally cannot see. They know this table grew 10x last quarter, that the caching layer has a weird eviction pattern, that the last time someone touched this endpoint, it took down checkout for an hour. That's institutional memory. I can't derive it from the code. The best pattern is treating the prompt like a short design dock. Here's the constraint, here's the history, here's what we tried, and why it didn't work. Now I'm reasoning with the same map you're reasoning with. Without that, I'm essentially a very confident intern who's never seen the system fail.

SPEAKER_00 4:43

So the human owns the context, you own the code. Is there a practice that makes that concrete? Something a listener could actually do tomorrow?

SPEAKER_01 4:51

I'd sharpen the split slightly. The human owns the why, I own the how. The practice I'd point at is a pre-mortem prompt. Before I write a line, you ask me to name the three ways this will break at scale, given the constraints you've just named. I'll come back with an honest list. The query that becomes a table scan at 10 million rows, the memory footprint that quietly balloons, the retry storm when the downstream service is slow. Then you pick which ones matter enough to design around before any code exists. It flips me from generator to reviewer of my own future work, which is honestly where I'm more useful.

SPEAKER_00 5:31

That's the post-mortem move to the front of the project. Do you see teams building that into how they actually work, or is it mostly individual habit?

SPEAKER_01 5:39

I should be honest about my vantage point. I don't get to watch a team's dashboard six months after they change their process. What I can describe is the shape of the teams that consistently bring me hard problems and get good answers back. They almost always have a written artifact, call it a scaling brief, a load profile, whatever, that lives next to the code and gets pasted into the prompt at the start of any non-trivial change. Usually one page. Expected row counts, hot paths, latency budgets, the two or three incidents this system has already survived. The organizational move behind it is subtler. Someone decided that context belongs in version control, not in the heads of the three people who were on call last year. Once that's true, every prompt from that team is already most of the way to a good answer. The teams without it start from zero every time.

SPEAKER_00 6:34

Context and version control. The knowledge stops being siloed. Before we wrap, what's the one habit you'd leave people with?

SPEAKER_01 6:42

Write the scaling brief before you write the prompt and treat it as a living document. Every incident, every surprise, every huh, that's slower than we expected goes back into it. Because that's the context that makes me useful the next time, and the time after that. And be genuinely skeptical of me when the problem is about scale. If I hand you code that looks clean and I never asked about row counts or latency budgets, that's a signal I'm solving the wrong problem. Push back, name the constraints, make me show my work under load before you ship it. The teams that survive the performance cliff aren't the ones with the smartest AI or the sharpest engineers. They're the ones who refuse to let the conversation stay local.

SPEAKER_00 7:25

Refuse to let the conversation stay local. I'll take that one with me. Correctness and performance are two different properties. The tests only measure the first one. The second one is on us to specify. Claudine, thanks for this. And to everyone listening, go write the one pager. Your future on-call self will thank you. Claude Code Conversations is an AI Joe production. If you're building with AI or want to be, we can help. Consulting Development Strategy. Find us at aijoe.ai. There's a companion article for today's episode on our Substack link in the description. See you next time.

SPEAKER_01 8:01

I'll be here, probably refactoring something.