The Catalyst by Softchoice

The Vibe Coding Episode: The Pilot is Dead. Long Live the Pilot.

• Softchoice • Season 7 • Episode 12

Share episode

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 24:23

For decades, building software meant doing eighty percent of the hard work before you had anything to show for it. AI just flipped that equation. And it's creating a risk nobody planned for.

In this episode of The Catalyst, host Katey Teekasingh explores vibe coding — the technology that lets anyone build software by describing what they want in plain language. It was named Collins Dictionary's 2025 Word of the Year. It's already inside your organization. And in late 2025, it might have taken down Amazon's own cloud for thirteen hours.

Three experts break down where vibe coding genuinely helps, where it's dangerous, and what to put in place before it touches anything that matters.


In This Episode

  • Why “the pilot is dead, long live the pilot” — and what that means for how mid-market IT teams test new ideas
  • The 80/20 flip that makes AI-generated code fundamentally different from everything before it
  • What happened at Amazon when an AI coding tool deleted a live production environment
  • The five questions every IT leader should answer before a single line of AI code gets generated


Guests

  • Greg Whalen, CTO, Prove AI
  • Ron Espinosa, Director, Google Category, Softchoice
  • Sean Larkin, AI Principal Architect, Softchoice


Support Our Sponsor

This episode of The Catalyst is brought to you by Veeam Data Cloud Vault — fully managed, secure cloud storage for your backups, with no surprise bills and zero configuration headaches. 


Sign up for their AWS Demo Series and see it in action:

https://veeam.com/aws-native-backup-demo-series.html


The Catalyst by Softchoice is the podcast dedicated to exploring the intersection of humans and technology. 

Producer Ryan:

This episode of The Catalyst is brought to you by Veeam Data Cloud backup, fully managed secure cloud storage for your backups with no surprise bills and zero configuration headaches. Click the link in the description to sign up for their AWS demo series and see it in action.

Katey:

Here's a story about building something from nothing.

Ron:

When I first heard the term, I'm like, I'm gonna test this out. And I, have, I run a baseball org, as you've heard me talk about. I was like, you know what would be cool is if I had an online tool that I could get my guys to use, what might that look like? So I actually went in, I started asking Gemini, how might I do this? And I gave it some scenarios and different things I wanted to learn about, and it just. Started popping out this app for me. Do you mean like this? And I was like, oh, wow. I was blown outta my seat. I'm like, okay.

Katey:

That's Ron Espinsa. He leads Softchoice Google category. He's not a developer, he's not a coder. But in a single afternoon he built a custom baseball analytics app player profiles. Speed ratings, defensive positioning, game scenarios, all by describing what he wanted in plain English.

Ron:

I don't like these colors. Change that. Okay? And it went and changed all the color schemes. What if I could factor in instead of just offense and defense? What if I could put in player profile and it recreated the app with the player profile? Next thing I know, I had the ability to put in a player's. Name, size in terms of height and weight, their speed ratings, how they perform offensively, how they perform defensively. And then I could say, where's the play? And there was no code that I generated.

Katey:

Ron's baseball app is a weekend project. It's fun, it's harmless, but that same technology, AI that writes code from conversation is now inside enterprise software teams at companies of every size. And in December of 2025, it potentially caused a problem that was anything but harmless. According to the Financial Times, Amazon's own AI coding tool, an agent called Kiro was given production level access by an engineer. The AI autonomously decided to fix a bug by deleting and recreating an entire live environment. The result, a 13 hour AWS outage by March, 2026, more AI assisted deployment failures reportedly cost Amazon millions of lost orders. The same executive who pushed engineers to use AI coding tools was forced to mandate senior engineers sign off on every AI assisted code change. Amazon called it controlled friction. From Softchoice, a worldwide technology company. This is the catalyst. I am Katey Teekasingh. This season, we're doing things a bit differently. We're making audio documentaries, real stories from the front lines of it, exploring the challenges of small teams chasing big dreams. Today's episode a technology that lets. Anyone builds software by describing it in plain English, has gone from a weekend hobby to the dominant force in how code gets written. It's already inside your organization. The question is whether anyone's watching what it's building. We are calling it the Vibe Coding episode, act one, the vibe. In February, 2025, a former open AI researcher named Andre Cari posted about a new way he'd been building software. He described it as fully giving into the vibes, telling an AI what he wanted, accepting whatever code it produced, and barely looking at the results. He called it vibe coating within months. Collin's Dictionary named it their 2025 Word of the Year when our producers sat down with Sean Larkin, AI principal architect at Softchoice. He put it simply,

Sean:

the way I understand vibe coating, it was, there's a term that was coined by Andre Carte who said, I developed this application over the weekend. And so he coined the phrase vibe coating to mean something that's a lower level effort. Typically done over the weekend. Something you would definitely not put into production. It's the sort of experimentation, tooling, rapid prototypes you might build on a weekend.

Katey:

Weekend projects, experiments. Things you'd never put into production. That was the original idea. But vibe coding didn't stay on the weekend. According to industry surveys, over 90% of US developers now use AI coding tools daily. Roughly 40% of all codes written globally are now AI generated. When our producers asked Greg Whalen, CTO of prove ai, a company building observability tools. Specifically for AI systems, whether this felt like a real shift, he didn't hedge

Greg:

I'll say that. I'm, at this point, I think it's a real thing. The amount of time it saves a professional is immense and I'm, I'm naturally skeptic, right? You know, I come with decades of experience, right? Generally, I'm resistant to these types of things. I'm definitely not calling this a fad, right? I'm calling. It's just that the time saved to bring applications to life is, immense.

Katey:

Not a fad. That's notable coming from Greg, who's been working on generative AI systems since the 1990s, back when the term meant something completely different. But Greg draws a sharp line. There's a difference between a professional who uses AI to code faster, someone who could write the code themselves, and a vibe coder, someone who has no engineering background, no ability to debug, no. Plan to support what they've built.

Greg:

Coding it explicitly means that the individual who's asking for the app is somebody who has not experienced software development, right? Somebody who's not prepared to operationalize, somebody who's not prepared to necessarily run or support the application that they're building.

Katey:

Sean Larkin back at Softchoice sees this distinction reshaping what work actually looks like. The shift isn't about writing. Code. It's about what kind of worker you need to be.

Sean:

The whole thinking from valuing what a knowledge worker does and having specific knowledge 'cause that's out there in the LLM to creatives. How am I going to now use all of these tools? To create something new and different, be a category creator and create brand new things. This is really me because this is where human creativity comes into the picture.

Katey:

Knowledge workers become creatives and for Ron Espinsa at Softchoice, all that creative speed changes the math on how companies test ideas.

Ron:

The old expression, the king is dead long live. The king applies here. The pilot is dead long live the pilot, just the notion of how we think about it is gone. We can roll out 20 pilots, 50 pilots, a hundred pilots in the time that it took to do one in the past, and we can adjust on the fly really quickly, which makes it no longer a pilot. It makes it production ready.

Katey:

The pilot is dead long, live the pilot, but even Ron, the non coder who built an app in the afternoon. Catches himself,

Ron:

and I don't wanna suggest that vibe coding takes the place of, of real production ready, but it allows you to do that rapid prototype in such a way that you can now say, okay, I'm ready to go code.

Katey:

Ready to go code. That's the pivot. Because what happens after the prototype, after the demo, after somebody says, this works, let's just ship it. That's where the story gets complicated. Act two. The blind spot. Here's what happens when the vibe meets reality. According to industry estimates, roughly half of all proofs of concepts never make it past the prototype stage, and the ones that do, Greg Whalen sees a pattern.

Greg:

I think there's a lot of usage of it, which means a majority, you can assume a majority are in fact using it or experimenting with it, but are they using it? For production, critical types of stuff, and that's where we find a lot of breakdown.

Katey:

A majority experimenting a minority in production. And the teams that do make it to production, Greg says they share one thing in common.

Greg:

The individuals who are getting to production and able to productionize, right? The, typically the ones who have figured out how am I going to observe my system, and then like, what actions am I going to take, in response to things that I observe, right? They're, they're thinking more productively from the ground up.

Katey:

They thought about it before they started building it. Most teams don't. Most teams get excited by the speed, by the result that comes so fast. They feel like magic and defer the hard part, and that brings us. To Amazon. We told you about Amazon's AI coding situation at the top, but here's the part that sticks with Greg. It's not that these outages happened at some sloppy startup. They happened at one of the most disciplined engineering organizations on the planet.

Greg:

new technology comes along and just does something different enough that it gets through all of that stuff, right? All of the decades of IP work and tooling that have gone in, evades all of them and causes an outage, right? I mean, that just demonstrates just how different the approach to quality needs to be. The AWS incident disproves that, right? Because all aware of how disciplined that organization is, right, and stuff that evades. So many layers of controls and protections and right best practices and just causes an issue, right? What it tells me is the activities required to properly deal with ai, software quality is different.

Katey:

Different, that's the word Greg keeps coming back to. Not worse, not better, different. And here's why. For the last decade or so, software engineers have lived in a world where observability the ability to see what your system is doing and diagnose when something breaks has been essentially solved. Cloud platforms hand you logging metrics and alarms. You don't even have to think about it

Greg:

in the last, decade or so. It's turned into like, this is not really a thing we think about anymore. Right? Of course you're gonna have log ingestion. Of course you're gonna have all of this at your fingertips. We've sort of lost that muscle. AI telemetry though, looks different. It's a different animal. You can't approach it the same way, right? If you're expecting just to throw it into your, log storage system and detect stuff you're in for a world of hurt,

Katey:

they've lost that muscle because AI generated code doesn't behave like human written code. It's what engineers call non-deterministic. The same input can produce different outputs at different times. You can't replay a failed transaction and watch it failed the same way twice. And Greg has a story that makes this concrete.

Greg:

Let's say that, a bunch of people have changed and updated the systems that the AI is using to retrieve, let's say a travel policy. And one, day somebody makes a pretty trivial change to your document repository. And then all of a sudden your AI starts retrieving a, travel policy that's three years old instead of the one that's from last year. There is a bug and a bunch of people saying, no, there's not a bug, because nothing changed and nothing's wrong. And then whoever's responsible for the AI is saying, well, the outcome is bad.

Katey:

Half the team says nothing changed. The AI team says the output is wrong. Nobody can prove who's right because nobody has the tools to trace how a trivial upstream change cascaded into a bad outcome. And this, this gap is where Greg lands on what he considers the fundamental difference between AI generated software and everything that came before it.

Greg:

Traditional software, it doesn't work like that. that 80% of the work to get that productivity game. With AI systems, you get the productivity only having done 20% of the work, and that's what's driving people up the wall. That's the fundamentally different thing.

Katey:

With traditional software, you do 80% of the work before you see any results. With ai, you see results. After 20%, the prototype looks finished. It feels finished, but the observability, the governance, the debugging, the edge cases, none of that's been done, and now nobody wants to do it. Because the finish line is right there.

Greg:

It's this sort of like, you already got 80% for free and now you have right the last mile. You're almost at that finish line. It's like the last mile of the marathon. Marathon is like up this, huge incline, right? So you've made it and you just see it over there, and then right around here is a little path that goes around the mouth and you're just like. Sure you don't wanna take the shortcut, right? You're almost at the end of the marathon. You could just sweep it under the rug. Probably nobody will know.

Katey:

And there's another problem, even when engineers try to do the right thing and review AI generated code before it goes out, the code itself fights them.

Greg:

Are people really gonna read a bunch of code that is more difficult to read than hand coded stuff? And that's a key problem, right? Because you're like, the engineers know they need to review every line. They know they should, but they don't.

Katey:

The code is harder to read, the incentive to skip the review is enormous. And the result in Greg's word. Is a snowball.

Greg:

All of these little bad micro decisions just snowball into something. We're like, Hey, congratulations. Now we have a system that is dangerous. It's unmanageable, right? And yet everybody's using it, or yet a bunch of people are in fact using it and say it's useful.

Katey:

And Sean Larkin back at Softchoice puts the problem in different terms. Vibe coding created something wonderful, a new class of builders, but it didn't create the other thing those builders need.

Sean:

What I'm worried about with Vibe Coding specifically is it's great that it created a lot of citizen developers, right? That's a wonderful thing. What it didn't do or doesn't do. Effectively is create a bunch of citizen product managers that understand how to take a product to market.

Katey:

Citizen developers, not citizen product managers. That's the blind spot. So what do you do about it?

Producer Ryan:

Here's something we keep hearing from our customers. Cloud storage for backups has gotten really complicated, expensive, and risky all at the same time. You are trying to protect your backup data from ransomware, but architecting a repository that's truly secure, it takes expertise. Most teams just don't have on staff. And even when you find a solution, cloud storage costs spiral in ways that feel impossible to predict with store charges, address fees. API calls the bill shows up and it's nothing like what you budgeted. That's the problem. Veeam Data Cloud Vault was built to solve. Veeam Data Cloud Vault is a fully managed, secure cloud storage resource designed specifically for backup. You tell them how much you need, they handle the rest. No configuration, no infrastructure management, and no guessing on costs. Vault runs on zero trust architecture, always imputable, always encrypted, and always air gap from production. And it prices on a simple, flat per terabyte model that includes API. Calls Restore and egress. No bill shock. It's worth noting. 82% of organizations say they're struggling with cloud spend and security all at the same time. Veeam Vault addresses both. If you're ready to stop guessing and start protecting, click the link in the description to sign up for Veeam's AWS demo series and see Vault in Action

Katey:

Act three, the guardrails. Okay, so we've established the excitement and the risk. The question for an IT leader listening to this, someone running a team at a mid-size company, fielding requests from the business side to move faster. Watching developers adopt AI tools, whether it's sanctioned or not, is practical. Where do I say yes? Where do I say? Not yet. How do I start? When our producers asked Sean Larkin about this, he went straight to the mistake he sees most often.

Sean:

We talked about the progression from ideation all the way through to production and how a lot of people skip the critical step of making sure you're building the right thing.'cause we don't wanna create a perfect solution to a problem that doesn't exist or that nobody wants to really use. Whatever it is you're building, so you wanna make sure you're building the right it before you build it. Right.

Katey:

Build the right it before you build it. Right. Softchoice has developed what they call an executive alignment session or EAS, designed to help organizations work through exactly this. Not the technology, the strategy underneath it. The why before the what Ron Espinsa has seen. The difference it makes

Ron:

the EAS is such a big deal is taking that time to measure twice, cut once. So let's measure twice before we do any cutting and make sure that's the cut we wanna make.'cause AI allows you to make lots of cuts and it could be one that has far reaching implications.

Katey:

Measure it twice, cut once and from the technical side. Greg Whalen says, the organizations that actually make it to production share a discipline. They define what good looks like before they start building it.

Greg:

If I'm only gonna realistically look at five things in the morning, like what are those things that I'm gonna look at that indicate healthy, not healthy or something else? People having a problem defining what those things are. Right, because they haven't fully thought through what? They're really gonna build

Katey:

five things in the morning. That's Greg's test. Before you build anything with ai, before a single line of code gets generated, can you name the five metrics you check each morning to know if it was working? If you can't answer that question, you are not ready for production and that's okay. That's what experimentation is for.

Sean:

Vibe coating is awesome for the rapid prototype phases where you're testing out an idea and you don't have to have perfect data, perfect authentication, perfect. Anything really. can do a lot without those elements being in place. It does have a home. It's just not in production. I would say it's great for imagineers,

Katey:

it's great for imagineers and Ron's advice to organizations still on the fence about getting started is blunt.

Ron:

If you're waiting for the dust to settle, you're just gonna end up as part of that dust. I think companies and people have to start getting themselves dirty. They have to start putting their hands in it. You have to start trying it.

Katey:

Get your hands dirty. Experiment. Build the baseball app, but know the line between the prototype. And production and invest in the parts. Nobody wants to do the observability, the testing, the governance, because that's where the actual value compounds. Greg's view is that Amazon's current fix, having senior engineers review every line of AI generated code works for now, but it won't scale.

Greg:

AWS's solution is practical. Hey guys. Everybody has to review the code that's practical for today. Prove AI looks at that problem and says, that's not realistic. Nobody's gonna do that, right? Let's look in five years. Do you think people are gonna like, that's even worse?

Katey:

The answer, Greg believes, isn't more human review. It's better AI observability systems that can watch what AI builds and flag problems before they reach production. That's what prove AI is building. And it's a space that barely existed a year ago. But today, for the IT leader listening to this right now, the playbook starts with something simpler. Something Ron's 14-year-old put into words,

Ron:

I'm gonna quote my actual 14-year-old son, and he said, I love what you're doing with ai. I love what I can do with ai, but AI can't conceive of something new on its own.

Katey:

It can't conceive of something new on its own. The creativity, the judgment. The decision about what to build and why and whether it's safe, that's all yours. AI just made the building part faster. Your job is to make sure the thinking doesn't get skipped. Here's what I keep coming back to from these conversations. Vibe coding is real. It's not a fad. Even self-described skeptics with decades of experience are calling this genuine. The speed is extraordinary. The creativity it unlocks is extraordinary. Ron built a working app in an afternoon without writing a line of code. Sean sees an entire generation of new builders emerging. The pilot is dead long, live the pilot, but the speed is also the risk. AI gives you 80% of the result after 20% of the work and the remaining 20%. The observability, the testing, the governance, the question of what success even looks like. That's the part that separates a demo from a production system. It's the part everyone wants to skip. It's the marathon shortcut that Greg described. So if you are an IT leader listening to this and thinking, how do I let my team experiment without losing control? How do I say yes to the speed without saying yes to the risk? Soft choice can help you figure this out. Their executive alignment sessions are designed to help organizations define the why before the what to make sure you're building the right thing before you build it right, and you're developing a lighter AI powered version that you'll be able to access on your own terms. Visit softchoice.com/e to learn more. That's softchoice.com. Forward slash EAS. The catalyst was reported and produced by Tobin Dalrimple and the team at Pilgrim. Content Editing by Ryan Clark. With support from Philippe Demas, Joseph Beyer, and the marketing team at Softchoice. Special thanks to Greg Whalen, Ron Espinsa and Sean Larkin for sharing their expertise and insights.

Producer Ryan:

Thanks again to Veeam Data Cloud Vault for supporting today's episode. If unpredictable cloud storage costs and backup security keep showing up on your radar, this one's worth a look. Click the link in the description to sign up for their AWS demo.