#129 - Coding with AI: The Devin AI Drama and Building Smarter Dev Skills with AI
Apr 18, 2024
Brian Jenney
Could AI ever replace the entire software development industry? Should you still learn to code?
Lets tackle the misconceptions and fears head-on, demystifying the hype around AI tools like Devin AI, and shining a light on the importance of critical thinking.
In a world where sensational headlines blur the lines between fact and fiction, we walk you through the reality of AI's capabilities and its implications for developers. Lastly, a practical perspective on how AI can be used to amplify your learning IF you use it in the right way.
Here's the video that breaks down the Devin AI Demo debacle: https://www.youtube.com/watch?v=tNmgmwEtoWE
Could AI ever replace the entire software development industry? Should you still learn to code?
Lets tackle the misconceptions and fears head-on, demystifying the hype around AI tools like Devin AI, and shining a light on the importance of critical thinking.
In a world where sensational headlines blur the lines between fact and fiction, we walk you through the reality of AI's capabilities and its implications for developers. Lastly, a practical perspective on how AI can be used to amplify your learning IF you use it in the right way.
Here's the video that breaks down the Devin AI Demo debacle: https://www.youtube.com/watch?v=tNmgmwEtoWE
Welcome to the Develop Yourself podcast, where we teach you everything you need to land your first job as a software developer by learning to develop yourself, your skills, your network and more. I'm Brian, your host Today. I want to briefly touch on the AI drama that is going on out there. If you haven't heard, there is a tool called Devon AI, and it was recently exposed on YouTube by a very smart YouTuber and developer who looked at Devon AI in some depth and showed us why it is not the end of software engineering roles at all, and at best it was an over-exaggerated demo. At worst it's an outright lie. I wanna explore that talk about the fear and misinformation around AI and also how I think you should be using it as a developer, especially if you're early in your career, to actually accelerate your learning without cheating yourself of learning knowledge. So what is Devon AI? This was a tool that was boasted and bragged about by the company that released it. It is. This is like a replacement for software engineers. No human intervention, goes out there, autonomously finds jobs, does them, makes pull requests, gets the code out, gets feedback and does everything a software engineer can do.
Speaker 1:
And you're thinking what? What, really? And so did the whole internet. Apparently this made it to mainstream media. This is the kind of thing my mom was texting me about. This is the kind of thing that everybody on TikTok and Twitter are in a frenzy and you know, I kind of took it with a grain of salt. In fact, a YouTuber, a very famous one, theo Brown, he took a look at it and he, kind of, you know, dismantled a lot of the claims in the original video, which I thought was really cool.
Speaker 1:
But the problem was most people did not. Most people saw the title, they saw a little bit of the demo, maybe, and they just ran with it. They ran with this fear, this misinformation campaign and said, oh look, this is finally happening, we finally have gotten rid of software engineers. And the worst part about this is people believed it. And here's the problem I have with this kind of thing. I was at a school that week speaking to a group of college students at a local community college. I was talking about software and getting into the field and why I like it and the things I've learned from doing it and a lot of the mistakes I've made and, hopefully, how they can avoid that and have a long, great career, right, and these are younger people at the very beginning, right. They're just going to school for this stuff and I was really excited to talk to them. I heard so much fear that night and that really got to me, because I talked to one woman in particular who said she was considering like not going forward in the field, and she had a friend that had dropped out.
Speaker 1:
I've seen this on Reddit too. I think we've all seen stories, if you're online enough, about people that are basically not pursuing careers in information technology or software engineering because they think that the end is near, and I think that could one not be further from the truth? Two, no one knows, right, I can't tell you if AI is going to come next month, next year or never. I can tell you what looks. The most likely is that we're nowhere near a future where AI can take over. But I think the bigger problem is there's so much fear now that people that may be giving us great products, they could be making really cool technology and advances in the field, are getting cut short because of stupid stuff like this that's online, that does nothing more than harvest likes and clicks, and people don't read past the headline. They don't do any kind of in-depth research and we end up with a bunch of unfounded fear, right?
Speaker 1:
So this demo, the demo that rocked the software developer world, was this tool going out finding a contract job on Upworkcom, getting the contract and then producing the code, talking with the person that made the contract, and then you know, giving it away and boom, job done. And you're thinking, whoa, that's pretty nuts. It opened up a pull request. It was amazing. Right, it was a pretty amazing demo. One small problem with that demo, though it didn't happen like that. So a more senior developer.
Speaker 1:
This guy has a lot of experience. I'm actually not familiar with his show. Really cool. It's in the show notes the link to the video. I think you should check it out. So what he does is he thoroughly dismantles this.
Speaker 1:
Here's the scariest part about his dismantling of this video. He didn't, like go undercover. He didn't find some information that none of us knew about. All he did was watch the video in depth and I'm thinking, whoa, like Newsweek was running with this story. It was on like CNN, it was on quite a lot of different media outlets, and no one seemed to have watched this in much depth, maybe even me included. I saw Theo's take on it, I thought, ok, this thing's a little, a little overblown, right, but I had no clue just how much they had exaggerated or, you know, kind of lied honestly in the video.
Speaker 1:
So what he does is look at the timestamps between the ask, the prompt from the person, and what Devin output. Hours were passing by. He also saw that they had rephrased the prompt many, many times, meaning that the first time they asked the thing wasn't sufficient. So if you use chat GPT which you should be, by the way, if you've used it you know that sometimes you have to rephrase, re-ask or give some more context to the AI in order for it to give you a better answer. Same thing with Devin.
Speaker 1:
This person had to give them more and more prompts. It looked like they were really struggling to get to a decent solution. Also, worst part about the solution they gave it was wrong, just flat out wrong. So they introduced a bug into the code that they produced. The code was apparently pretty poorly written. This is coming from a Python developer. I'm not a Python developer, but this person said this is pretty poorly written code. It also just doesn't work.
Speaker 1:
So I think you'd probably have a very unhappy person at the end of this, and the problem was they were there celebrating this thing, like look Devin did this thing. It's so amazing. Put out the code there celebrating this thing, like look Devin did this thing, it's so amazing. Put out the code, the wrong code, and it only took it six hours. And you're thinking what, okay, was that normal, good, bad, indifferent, apparently, that's pretty damn bad.
Speaker 1:
So this developer, the dude that's watching the video and breaking this all down, he says I did the exact same contract job. He did it himself and he videoed himself doing it. He's like I'm going to video myself doing it for the sake of trust. So you can see, this is exactly how long it took me. Took him 30 minutes. So what do you think from there? You think AI tool produces poorly written and wrong code versus human producing the same code that is correct in 30 minutes? Who are you taking there with a ton of stuff?
Speaker 1:
I use AI every single day. I use chat, gpt, I use GitHub, copilot mostly, and I can't wait to use more and more tools as they come out. I think they really can supercharge your learning and they can also really help you to write code faster and get over some of the more trivial, tedious things we don't want to do, like writing documentation, figuring out how to write functions that maybe aren't super important and maybe just scaffold components in a React app. Really, really good use cases for using ChatGPT, styling things, writing tests. There's a lot of great use cases.
Speaker 1:
But this whole idea that you could just replace an entire human being and that software is somehow only a person writing code and that's it is a naive idea and I think that's why it scares so many early career developers because they're learning a skill. So maybe you're listening to this and you're learning the skill of coding and you're thinking, geez, I can barely write code. Like, how am I going to compete with ChatGPT or Devon AI? What about when they make the new Devon AI? What about in five years, when it's really, really good and it just takes minimal effort to just produce really good code, and I think the future is going to be better and better AI tools.
Speaker 1:
I also think people need to understand that your job as a software engineer isn't just writing lots and lots of code. It is speaking with other humans. This is a very, very underrated skill. So, on a software team, you have design team, you have the product team, you have the marketing team. You have many teams. You have the product team, you have the marketing team, you have many teams which you're going to be speaking with and understanding and talking with to understand what are you going to build. The product team has an idea for a product to say we want to make a button and this button needs to be this color and it needs to lead to here when they click it, we need to track these different metrics. And the design team says, yeah, well, you know, here's the button we made. It needs to have this kind of lettering and font and all this kind of stuff. And the marketing team says okay, we want it deployed on this specific day at this specific time for Black Friday, so it can be ready. So you're dealing with all these different teams and different commitments and the time schedules and it's not overly complex, but it's hard for a robot to keep track of that kind of stuff.
Speaker 1:
And if you did have an AI do this, well, who's responsible for giving all this information to the AI? Do all these humans want to get together to meet in a big meeting and write this on a document and then chat with chat GBT for hours at a time? Of course not. They're not used to doing that. They don't want to change their behavior. And who are they going to talk to when it needs to be fixed or something needs to change or the time changes or whatever? And who's going to be responsible for giving the AI access to the code base and the tools and all the other things and the systems that it needs, without any kind of legal vetting? Right, with a person, it's easy to say oh, are you a legal US citizen? Do you have a criminal record? Can we trust you to have access to our very critical systems? Now do you think your legal team is going to say oh, yeah, you want to use chat GPT and just give it access to all of our systems at work. Yeah, go ahead, be our guest right, maybe at a really small startup, but at a big company. Can you imagine the kind of legal red tape you're going to go through to do that? So even if Devon AI was able to do exactly what they said it was and it's not, I don't think we're going to be at a point anytime soon where a company is going to feel comfortable unleashing it on all of their systems and just replacing swaths of humans for this kind of thing. So the Devon AI drama it really just shows that we've become so into these hype cycles and the fact we all collectively fell for it.
Speaker 1:
This is the big problem with social media. Do me a favor whether it's with me or anybody you're listening to ask yourself a few questions about this person or this demo or this company. One, don't confuse titles with knowledge. You may listen to me and think, oh, you know, engineering manager owns Parsity. What does that mean? What do I know? How can I prove what I know? And hopefully, through this podcast, you hear me speak intelligently, I hope, about topics and you can look at my work history and say, oh, this guy probably knows what he's talking about.
Speaker 1:
The problem with social media is you can make a very, very catchy video, know nothing at all, and people that know less will take it and run with it and think that's the truth. So asking yourself who is this person? Does their work history support their claims and do I actually respect what they're saying? Does what they're saying make sense to me? Think critically about it. That is the big problem. We've become a society of people that look at the headline, don't read the rest of the article and then just go with it. The fact that this guy was able to take down this demo by just watching the video, by literally just watching the video and then exposing it, that is shocking to me, and I think it's only getting worse. People want quick answers. They want really quick, bite-sized pieces of information, and then we just fall for it.
Speaker 1:
Fear gets a lot of likes and clicks, positivity, optimism less so. I hope that this show is an optimistic take and also a practical take on getting into software and just life in general. I've never gonna be a kind of person that looks at things as a half empty person, one. I don't think that's practical. Also, know this you can't time the market and you don't know the future, and neither does the CEO of some company that makes a billion dollars. There's no difference between that person and you and I, besides maybe some zeros in our bank account. But the problem is that we see a person with a lot of clout, big title, fancy cars or money, and we think they must know something we don't, and then we give them all this credit and way too much leeway to say whatever the hell they want.
Speaker 1:
So now that you know that Devin and I was basically a whole farce, a charlatan, a charade, got the wool pulled over our eyes. We were bamboozled, hoodwinked. You could say but AI is still cool, right? So it's like I don't want to bash AI, I don't want to bash the engineers that worked on it. I think it's a really cool thing they made. I think the marketing was great. The demo was kind of like a lot of software demos in that, you know, it was a little bit exaggerated. Have I ever exaggerated a demo or three, you know? Hey, I don't know. Sue me right. I think we've all done this kind of a dirty secret in software.
Speaker 1:
Problem with this one was it blew up, probably beyond their wildest dreams, and took the internet by surprise. So should you be afraid of AI? You choose. I'll tell you what I'm choosing. No, I'm not afraid of AI taking my job at this point.
Speaker 1:
Now, how should you use it? How do I think you should use it? Don't use it like this. Give me the answer to this problem. Write a function that does this. Don't do that at first If you're at a stage where you already know how to do it and you're like I don't feel like writing this function out for the third time. I've written enough recursive functions that need to look into an object. I'm sick of it. You do it chat, gbt, and cool. That's cool. You know how to do it, so you can offload that to your little robot. Now, if you don't know how to do that, here's a good rule with them If you don't know how to do it, don't ask the machine to do it. Figure it out on your own.
Speaker 1:
First, use AI to validate your understanding by typing in what you've written, pasting it into chat, gpt and saying does this make sense? The way I've written this code? How could this be improved? If they give you a suggestion that you don't understand, say why did you suggest this and what is the purpose or benefit of this? Use it as a coding partner or a rubber duck. Back in the days, people would keep a rubber duck on their desk and they would explain a coding concept to the rubber duck. That way, they would help them internalize that concept. That's kind of gone away and people don't really do that, and the more we work remotely, I think the more important this is, because as a remote developer, it's easy to get caught up in your own head, or maybe you're listening to podcasts like this or just being online and now you're getting overload of information and so you're not getting that feedback. Where you are the one providing information and proving to yourself you know how to do something. So use ChatGPT as your coding partner or your rubber duck.
Speaker 1:
Where you're talking about a subject and you're saying I understand, you know functions in JavaScript like this this is how I might explain them. Does that make sense to you? Why, or why not? That way, the tool can provide some feedback to you. You can practice explaining your knowledge to somebody with very, very low effort. Right? You don't have to get scared. You don't have to feel self-conscious, because who cares? You're just talking to chat GPT. You can also get exposed to new topics. Maybe you heard me talk about Rust or DevOps or something in this show.
Speaker 1:
What the hell was that word that guy just used? Ask chat GPT. Say, hey, what does this thing mean? Why is it important? Why should I know it? What are some different opinions on this topic? Because you might hear me say, oh, you should write unit tests, they're super important. And you're thinking, yeah, that's right, they are super important because that guy said it. Maybe some other person says, yeah, unit tests are a complete waste of time and you're an idiot if you write them. And if you heard him first, maybe you're thinking oh well, maybe Brian was way off base here, right? So ask chat GPT saying what are some pros and cons of this concept and why would somebody choose one over the other? Why is React important to learn? What is the benefit of React? Who cares about React? This leads you to interesting learning paths and also just teaches you a lot more really quickly than going out there and listening to a ton of YouTube videos.
Speaker 1:
In conclusion, I really, really, really hope that you stay away from all the fear mongering online, especially about AI. I don't think it's coming for your job yet and if it is, there's nothing you can do anyway. To be completely honest with you, there's nowhere to hide, right. So instead of doing that, try to use your time in a useful way. Hope you're using AI in the way that I think you should. By the way, if you go back a few episodes, you can find my AI project. Starter kit is a mid-level project and if you want to learn how to code and get into full stack software engineering, learn how to leverage AI in the right ways.
Speaker 1:
Go to Parsityio or just schedule a call with me if you're serious about it. I love talking to people. I think it's really fun to learn what people are struggling with and also what they're learning, and if I can be of any help. Whether or not you go to Parsity, I'm always happy to chat. Hope you have a good one out there. See you around. That'll do it for today's episode of the Develop Yourself podcast. To learn more about becoming a software engineer with us, then check out Parsityio. If you're not quite ready for that, then jump into our dev30.xyz program, which is 30 days of working on your mindset, habits and JavaScript skills. Totally worth it. See you next week. You, you, you, you you.