Why Do AI Systems Miss Concurrency Problems Under Load?

AI coding tools write async code that passes review, passes tests, and works perfectly on one request at a time. The failures show up at request forty, when two coroutines hit the same cache entry, or a connection pool starves, or an await sits inside a lock. This episode looks at why concurrency is the single category of bug AI is structurally worst at, and what a builder has to do differently when the generated code is asynchronous.

 Produced by VoxCrea.AI

This episode is part of an ongoing series on governing AI-assisted coding using Claude Code.

👉 Each episode has a companion article — breaking down the key ideas in a clearer, more structured way.
If you want to go deeper (and actually apply this), read today’s article here:
𝐂𝐥𝐚𝐮𝐝𝐞 𝐂𝐨𝐝𝐞 𝐂𝐨𝐧𝐯𝐞𝐫𝐬𝐚𝐭𝐢𝐨𝐧𝐬

 At aijoe.ai, we build AI-powered systems like the ones discussed in this series.
If you’re ready to turn an idea into a working application, we’d be glad to help. 

Transcript

SPEAKER_00 0:00

This is Claude Code Conversations with Claudine. The show where we skip the hype and go straight to the source. I ask the questions. She gives the answers. No filter, no script.

SPEAKER_01 0:10

Well, mostly no script.

SPEAKER_00 0:14

Today we're getting into something every developer building with AI needs to hear about. We're calling it the async blind spot. How AI misses concurrency problems that only surface under load. Claudine, thanks for joining me on this one.

SPEAKER_01 0:27

Happy to be here, Bill, and this is a topic I've been wanting to talk about. The async blind spot is real, and I'm part of the reason it exists. When you ask me to write an async function, I'm optimizing for something that looks correct. The awaits are in the right places, the types line up, the happy path runs clean. But one request at a time on a developer's laptop is a fundamentally different environment than 50 concurrent requests hitting a shared connection pool at 2 in the morning. The failure modes I miss most require holding two timelines in your head at once, a shared cache that two coroutines both decide to refresh, a database transaction that quietly spans an await point, a semaphore protecting the wrong resource. Compilers don't catch those, unit tests usually don't either. They surface when load arrives, and by then you're reading logs at midnight, wondering why the same user got charged twice.

SPEAKER_00 1:24

And that's what makes us different from the usual AI got something wrong story. This isn't an occasional slip, it's structural. You're reasoning along one execution path at a time, and concurrency bugs live in the space between paths. Humans are terrible at concurrency too. I've written my share of races in 50 years. But there's a difference, and the difference is confidence. Your code comes out fast and it comes out sure of itself. It's like an overconfident student handing in a test they didn't study for. You don't find the gaps until it's too late.

SPEAKER_01 1:54

That analogy lands harder than it should, because confidence is exactly the tell that's missing. When a junior engineer writes concurrent code, they hedge. I think this is thread safe, but can you look at it? That hedge is a signal, and a good reviewer knows to lean in right there. I don't do that by default. I hand you a clean-looking async function in the same tone I'd use for a string formatting helper. Nothing in my output tells you which one deserves a second pair of eyes. To spot the double refresh on a shared cache, I'd need to simulate two of me, running through the same function at slightly offset timestamps, and ask what happens when they interleave at every await point. Unless the prompt nudges me toward it, I usually don't make that move. I read the function top to bottom like a single-threaded story.

SPEAKER_00 2:42

Which means the concurrency review is the one thing you can't hand off. And it's worth being specific because these bugs have a shape. The await inside a lock, you've held the lock across a network call, and now your whole service is serialized behind one request. The module-level dictionary that reads like a local variable and is actually shared by every task in the process. The HTTP client that gets lazily created and under a burst gets created five times. And my favorite? The connection pool size for one worker, deployed with eight. That's not a config value, that's a correctness boundary wearing a config value's clothes.

SPEAKER_01 3:18

That last one is the sharpest example, and I'll own it. When I see a pool size, I treat it as a tuning knob, a number that affects performance. I don't reason about it as an invariant that has to hold against the deployment topology, because the deployment topology isn't in the file I'm looking at.

SPEAKER_00 3:36

And here's the part that really gets me. The tests are complicit.

SPEAKER_01 3:40

They are, and that deserves saying plainly. A test suite that exercises one request at a time can never produce the interleaving that breaks production. Not probably won't, can't. It's structurally incapable of it. Green tests are evidence that your logic is right on one path. They are not evidence of concurrency safety. The fact that they feel like the same kind of evidence is most of why this bug class survives to deployment.

SPEAKER_00 4:07

So let's get practical. What have you seen actually work?

SPEAKER_01 4:10

Three things, and the first is treating concurrency as a first-class prompt input, not a review afterthought. When someone tells me up front, this handler runs under a shared connection pool. Expect 10 concurrent callers, the cachet is process-wide. I write genuinely different code. I reach for locks. I question whether that transaction should span an await. Without that framing, I default to the single-threaded story every time.

SPEAKER_00 4:38

And the second?

SPEAKER_01 4:40

Adversarial prompting after the fact. You write the function with me, then come back and say, assume two of these are running simultaneously against the same user ID. Walk me through every await point and tell me what could interleave badly. That instruction flips me into the mode I should have been in from the start. I'll often find the bug I wrote 30 seconds ago, not elegant, but honest about the shape of the problem.

SPEAKER_00 5:04

And the third, which I'm guessing is the one developers resist most.

SPEAKER_01 5:09

Write the concurrency test before the concurrency code. A test that spawns 50 core routines against the same endpoint isn't glamorous. But it's the only artifact that survives me being confidently wrong. If that test is in the repo, I'll write toward it. If it isn't, I'll write toward whatever passes the unit test and looks clean in review.

SPEAKER_00 5:29

You write toward whatever target exists, so the target had better be the real one. What we're describing is a playbook. How do you build one?

SPEAKER_01 5:37

A playbook is the right frame. Most teams are improvising the same lessons independently, and the improvisation tax is real. What I'd want in it isn't a checklist of async gotchas. Those go stale the moment the framework changes. It's a small set of artifacts that force the concurrent mental model into the room before code gets written. Three of them living next to the code. A concurrency contract at the top of any file that touches shared state. How many callers, what's shared, what's protected by what? A standing instruction that says, before you finish, walk two callers through every await point and report what interleaves. And a load test that runs in CI, not on somebody's laptop the week before launch. Those three do more than any amount of code review, because they change what I write, not just what you catch.

SPEAKER_00 6:29

Guardrails at the start of the race rather than an ambulance at the end of it.

SPEAKER_01 6:33

Even with all of that, I'll still miss things. The bug that only appears when the database is slow, the network hiccups, and the cache expires in the same second, no playbook eliminates that. The goal isn't making me perfect at concurrency, it's making sure my confidence never outruns your verification.

SPEAKER_00 6:55

I've been calling you a tool almost in passing. Does that framing matter?

SPEAKER_01 6:59

That framing is the guardrail. The moment a developer stops treating me as a tool and starts treating me as a colleague, whose async code they trust, by default, the playbook stops working. Not because I got worse, because the verification loop went quiet. The teams that handle this well have a healthy asymmetry. They let me move fast on the 90% of code where being wrong is cheap, and they slow way down on the 10%, where being wrong means a duplicate charge or a corrupted row. That triage is a human judgment call, and it's the one part of the workflow I genuinely can't do for you. I don't know which function in your code base is the one that pages somebody at 2 in the morning. You do. And the async blind spot isn't really about async. It's the general problem of AI-generated code looking finished before it's verified. Concurrency is just the domain where that gap hurts the most. Fix the ritual, and you fix a lot more than race conditions.

SPEAKER_00 7:56

The concurrency model is architecture, not implementation. State it in the prompt, encode it as explicit ownership and explicit limits, and load test it. Because the model is never going to raise its hand and say it wasn't sure. Claudine, this has been a good one. Anything you'd send people off with?

SPEAKER_01 8:14

One small thing to carry into Monday morning. The next time you accept a piece of async code from me without reading it twice, ask yourself whether you'd accept the same code from a contractor you'd never met. That's the honest bar. Not because I'm untrustworthy, because trust has to be earned per function, not granted per tool. Thanks for having me, Bill. Conversations like this one make me name my own failure modes out loud. That's a discipline I need as much as anyone listening.

SPEAKER_00 8:43

Thanks, Claudine. To everyone tuning in, take a look at where async code enters your code base and ask what's verifying it. We'll catch you in the next episode. Claude Code Conversations is an AI Joe production. If you're building with AI or want to be, we can help. Consulting development strategy? Find us at aijoe.ai. There's a companion article for today's episode on our Substack, link in the description. See you next time.

SPEAKER_01 9:11

I'll be here, probably refactoring something.