Pybites Podcast
The Pybites Podcast is a podcast about Python Development, Career and Mindset skills.
Hosted by the Co-Founders, Bob Belderbos and Julian Sequeira, this podcast is for anyone interested in Python and looking for tips, tricks and concepts related to Career + Mindset.
For more information on Pybites, visit us at https://pybit.es and connect with us on LinkedIn:
Julian: https://www.linkedin.com/in/juliansequeira/
Bob: https://www.linkedin.com/in/bbelderbos/
Pybites Podcast
#180 - Transforming DevOps with Generative AI
Generative AI is revolutionising DevOps practices by streamlining incident management, enhancing collaboration, and enabling automation. In this episode, Bob chats with Chris Williams - Developer Relations Manager for HashiCorp - about the future of Ai in the tech industry.
Throughout the episode, we discuss the practical applications of AI, address concerns regarding job security, and learn about the importance of staying up-to-date with ever-changing advancements in AI.
Article:
https://community.aws/content/2rRTRRuo2Tj6r0FXZmWJH8gNHjP/supercharge-your-devops-practices-with-generative-ai
Talk:
https://youtu.be/POn5WYFw4xU?si=jtBMmPwe9yb-fe8I
Repo:
https://github.com/aws-samples/genai-for-devops
Chris social media links:
Chris Williams☁️🐍 (@mistwire.com)
Chris Williams (@mistwire) on X
Books / what we’re reading:
Chris: https://www.amazon.com/Mastering-Terraform-practical-deploying-infrastructure-ebook/dp/B0CZKFZ6NK + https://www.amazon.com/Go-Programming-Beginner-Professional-everything-ebook/dp/B0CZ6WFKNB
https://www.amazon.com/Machine-Learning-Generative-Marketing-data-driven-ebook/dp/B0D6BFJ1WK
Generative AI is really cool and it's really fun and it's this bright, shiny new toy that everybody has. But now we're at the point where we're trying to figure out okay, how do I genuinely apply it, like instead of just like a tack on chat bot in your website or whatever like that how do we use it to make our lives better in real, substantive ways? We use it to make our lives better in real, substantive ways.
Julian Sequeira:Hello and welcome to the PyBytes podcast, where we talk about Python career and mindset. We're your hosts. I'm Julian.
Bob Belderbos:Sequeira, and I am Bob Beldebos. If you're looking to improve your Python, your career and learn the mindset for success, this is the podcast for you. Let's get started. Welcome back everybody to the PyBytes podcast. This is Bob Bellows. I'm here with Chris Williams again. Chris, how are you doing?
Chris Williams:Hey everybody, I'm doing good Thanks.
Bob Belderbos:Welcome back. Chris was with us on episode 147. And I think there we spoke about cloud therapist.
Chris Williams:Oh, you went back to the old episode to see which one I was yeah, but that was a lot of fun.
Bob Belderbos:And, um, yeah, in this episode, um, I asked you to come back. Uh, well, first of all, how are you doing today? That's, uh, that's been a while.
Chris Williams:I'm doing good. Thanks, uh, the, the job is going really well. Uh, I, I do. I spend a lot of time doing what I love studying, learning, going out and talking to people about. You know, I work for hashi corp. I'm the oh, I'm. My name is chris williams. I'm the developer relations manager for hashi corp of north america, and uh, I I uh spend my days um going to conferences, doing talks, learning, writing blog posts, uh, creating content and and having a good time talking about. You know the, the pitfalls of it today and how you can fix them using cashew cork tools nice, nice.
Bob Belderbos:Yeah, you're living a dream and see your activity on blue sky and what you're up to I appreciate that I am having a good time.
Bob Belderbos:Yeah, I am having a good time so I invite you today to um talk about an article and a conference talk about devops and gen ai. Supercharge your devops practices with generative ai, right? And there was a conference talk. You guys did ados reinvent 2024 same title, and it was julie gunderson and chris williams and chris williams, so it sounds like you duplicated yourself, but no, it's actually. There are two chris williams right so.
Chris Williams:So julie uh pinged me and she was like, hey, do you want to do a talk with me and another chris williams? She was like I want to have two chris williams's on stage. So this will be the first time we've had a twice named chris williams at reinvent and I was like I'm totally in. So so she pulled me in to this project, um, because you know it's, it's, there's, there's a lot of pain points in devops writ large and yeah, gen ai can, can help solve, um, a number of the things that historically have been a little bit, uh, you know, tough um, and and we'll, we'll get into it. We can talk about dora, we can talk about um. You know the, the metrics and and and all the things that we did and the demos that we wrote to you know help make life easier yeah, so.
Bob Belderbos:So that's what I really wanted to dive into. But maybe for getting into the nitty gritty, what sparked this interest in adopting more Gen AI in DevOps?
Chris Williams:Well as with most people in tech, I'm a very lazy individual If I can automate things and make my life easier. In previous lives I've done a number of things. I was in incident management, I was heading retros and doing a number of different things. That isn't the fun stuff. The fun stuff is building stuff, know, building stuff, making making things go forward. But around that wrapper of the fun stuff are all of the things that you kind of have to do, like, like if.
Chris Williams:If you have an incident, then you should write things down afterwards to prevent the incident from happening again. If you have communication with people that is important, you should capture it somehow and turn that into either a runbook or a workbook or a how-to. If you create code, you should write some tests to test it to make sure that you don't explode it in production. If you build things, you should create monitoring and observability wrapped around it. And because we're humans, sometimes we don't think about those things.
Chris Williams:When we're under pressure, when we've got time constraints and we've got to just get something out because somebody promised somebody that it would be done in a certain amount of time, we've all been there. It's nice if we could leverage large language models and foundational models to do some of that for us. Take the first swipe at it, create 80% of the documentation, and then we fine tune it or create some of the tests or make sure that it does a code review in a very methodical and stamped out way, taking into consideration all of the things. So we're like, well, okay, so we have these large language models, let's see what we can do with them. Let's point Bedrock at some of the common problems that we find in DevOps and let's see what can happen. And we got some really fun use cases out of it and I think we had some good results too.
Bob Belderbos:Yeah, cool, you want to talk about some of these use cases.
Chris Williams:Yeah, sure. So, like I said, I did spend time in incident management in a previous life, a thousand years ago, and one of the things that we came up with was what if we could capture all of the conversations that happen in a chat thread? So something goes bang. It's three o'clock in the morning, the chat thread starts in the Slack channel. We have chatbots, so let's have a chatbot watching the channel and capturing all of that information. It is also tied in and it's got timestamps. So let us capture all of the metrics that are coming out of CloudTrails. So we're capturing the events, we're capturing the issues and errors and everything that's happening in CloudTrail. We're capturing the conversation. That is all fantastic information. Let's point a large language model at it, in this case Bedrock, and have it scrape all of that information from the start of the incident to the time that the service comes back up again. Grab all the conversations that happened in between and turn that and use that.
Chris Williams:As you know, assume the role of an incident manager. Answer these 10 questions what happened? Why did it happen? What are we going to do to fix it? Blah, blah, blah. And we did. And so if you go into the GitHub repository, if you look at a couple of the functions you'll see embedded in there are the prompt engineer calls to Amazon, bedrock that that say, ok, now take all this information and do this with it. It's just like talking to chat GPT. Here's all the information Give me, give me this synopsis and it spits out a pretty good response. And I was like this is sick, because I mean, you know that that blameless retro that we have at the end of the day, when everybody's like sitting there trying not to point fingers but you really want to point fingers and and and people are bringing up stuff, that that all comes out in the wash, the, the, the incident management. Chat GPT takes care of a lot of the heavy lifting so that you can just review it and make sure that you know going forward, those things will happen. Just review it and make sure that you know going forward, those things will happen.
Chris Williams:The next step after that was okay, now take that incident that happened the first time. If it happens again, craft a runbook to quickly fix the problem instead of spending, you know, two hours going through the issues at three o'clock in the morning. So that was another prompt that we fed into it. It grabbed the information from the previous incident and then crafted an actual runbook from the incident report that says okay, here's steps one, two, three, four, five go fix a problem. And it really it doesn't. A lot of people are kind of like, oh, gonna, it's gonna be taking jobs, uh, generative, ai is gonna be, like, you know, getting rid of people. And it doesn't do that. But it makes things go way, way faster cool.
Bob Belderbos:So, um, yeah, so you're saying run books. So, apart from jenna, I already working on the first instance. You're also building up a continuous, a process that make it continuously better, right by by reinforced learning, almost right. So, um, yeah, what? What are some of the the long-term, if you can already tell um learning and improvements?
Chris Williams:well, I mean this was this was very much a proof of concept that that we threw together and and uh, thought, like I mean we DORA. So, going back to the talk, if folks are curious about, like, the DORA metrics I'm not going to go into like, what the DORA metrics are here, but if you're curious about them, they're the four metrics that the folks at the DevOps Research Institute created to figure out, like what makes what makes a good DevOps team? Uh, and and so we use those four metrics for the lens of the pain points that we wanted to fix with, with our proof of concepts, and so that made it really go bump in the night, and how to shorten the lifecycle from getting something from first commit to production. All of those things were the things that we wanted to figure out. Okay, well, if I had infinite money and infinite coding skill, how could I leverage generative AI to help shorten these life cycles? And that was where we came at the demos from that perspective and it was really interesting to see the response from the folks in the audience. It was a very well-received talk. The last time that we checked it, I think it was like number five out of out of all of the. So there was like 900 talks at reinvent and, um, after everything was published on YouTube, we were, we were watching the views and everything like that, and I think we were like at number five for a little while. It was, it was doing really really well, um, and I think that the reason why it resonates is because generative AI is really cool and it's really fun and it's this bright, shiny new toy that everybody has.
Chris Williams:But now we're at the point where we're trying to figure out okay, how do I genuinely apply it, like, instead of just like a tack on chat bot in your, in your website or or whatever like that how do we, how do we use it to make our lives better in real, substantive ways? Um, and so that's, that is the approach that we took when we were, when we were talking about how do we? What kind of pocs and demos do we want to do for this? And um, yeah, it's I, I want to a lot of the folks that were in the audience. They were like OK, well, can you please share the code with us? Can you please show us how to deploy it and do it in our environments so we put the, the repo out there. There's there's how to's and walkthroughs on how to stand up in your own environment.
Chris Williams:It is it is an AWS talk, so it's leveraging all of the AWS services AWS chat bot, bedrock, all that stuff and, um, I think that it would be really cool for somebody to create like a service around that, and there are some companies that are actually looking at that, some incident response companies that are trying to like OK, now that, now that we can capture all this information, how do we turn this into a fully featured platform for incident response? Yeah, yeah, so yeah, yeah, I think's. I think it's a, it's a good application.
Bob Belderbos:Yeah, go check out that repo right and see what you can can do it and build around it, because it's just amazing how much opportunity you're now is with these models. But yeah, one thing is to be able to use chat, GPT and do your prompt engineering and get better at that. The other thing is to to work with their APIs and, in this case, AWS infrastructure and to really piece together an end-to-end solution right. That is way more involved, Exactly.
Bob Belderbos:In the talk, you show like diagrams and there's lambdas and there are buckets of cores and the bedrock and it's a pretty complex infrastructure, right? So? And the other thing I'm curious about like how yeah, how do you validate? Right, because usually ai tools these the scenarios you're describing seems very clear-cut. Where you have a, you can really provide a strong prompt with a lot of detail, hence the, the output from ai pretty reliable, but I guess they can't have it wrong, right? So how do you manage fault tolerance and checks and validants with that?
Chris Williams:And that's a really good point.
Chris Williams:I mean AIs can hallucinate.
Chris Williams:If you look at some of the prompts that we have inside of the lambdas and the calls and the code, you'll see the places where we've said don't make things up.
Chris Williams:If you don't know the answer to this, don't say it. Coming out of reInvent 2, that would have been really nice to have been able to leverage in the demo itself on how not to self-hallucinate and how to be careful with information and everything, but we didn't use those just for the POC. In a production environment, you're going to have to be very careful with the type of model that you're going to use, the type of information that you want it to be looking at and ingesting. If you have, like you know, corporate secrets that you want to be careful of, then perhaps you want to have your own on-prem large language model that is doing the ingestion and regurgitation of information for you and not rely on a CSP's giant public model and not know where the data might wind up someday. So there's a lot of questions around that that need to be answered when you're figuring out how to deploy this in a production environment.
Bob Belderbos:Yeah, the data privacy. That's definitely a concern. Very much so. Yeah, the data privacy, that's definitely a concern. Very much so. Yeah, cool, yeah. What excites you the most about the future of Gen, ai and DevOps?
Chris Williams:Well, actually we're doing a Gen AI boot camp starting February 1st Me and a couple of the other AWS heroes well, let me walk that back. Andrew Brown is doing a generative AI bootcamp. I am one of the guest instructors that is going to come on and talk about the enterprise, the ramifications of how to do a design. I'm tackling it from the enterprise architecture perspective, but we also have AI engineers and AI architects that are going to come on and talk about their niches of generative AI and how to build things. We're going to be crafting a translation application from English to Japanese. I am in the process of learning Japanese right now, so this is perfectly in tune for me and, yeah, so I'm.
Chris Williams:I'm very excited about how we're going to leverage it. It is a fantastic translation model right now, but what really excites me about it is like capturing the nuance of like phrases, like like. We have phrases in English that we say that don't have translations, but there are other similar types of phrases in other languages that mean the same thing from a, from a semantic perspective, but not from an actual like what the words written down mean. So I love having the translator module models to me, like the differences between them and also then explain like the actual word translations for stuff. So, yeah, there's I'm using it a lot for language type stuff, but there's there's so many applications out there. I think we're currently at a point where there's so many things that we can use it for. We're paralyzed with what we should use it for. So, yeah, I'm actually really excited to get this boot camp going, because I'm going to see a lot of really cool ideas come out of that as well.
Bob Belderbos:Yeah, no, it's almost like a whole nother dimension got added right. It really is. Other dimension got like, got added right in there. You get almost paralyzed, brother, like what we couldn't do like two years ago. Now all of a sudden all opened up and it happened so fast and it's evolving so fast that it's it's astonishing I I actually uh right when uh chad gpt you, you could first subscribe to it.
Chris Williams:I created a, an agent called the devops mentor. Um, I dumped a whole bunch of my old articles into it from my website. Um, I I took a bunch of uh, you know I I turned them all into PDFs and uploaded them and I said now, start, you know, teach back to me Like, so it has a much larger body of knowledge, of course, um, and I said, you know, assume the role of a mentor, use the Socratic method when you're teaching people to not just give the answer but to ask questions, to like, lead them to the, to the right answer and everything. And it is, it is currently one of the top like, like, if you, if you go to chat GPT and look for the DevOps mentor, mine is is the most used one with, like you know, x number of queries on there.
Chris Williams:Um, and it's it really. It really speaks to how people are leveraging these models to like, uh, the the devops mentor, if you ask it like terraform questions, it'll give you some, like you know, good, validated design stuff that came out of hashi corp. If you ask it go questions, it gives you stuff from from the various you know, public, publicly published authors information it's. It's fascinating because it is like a one-stop shop for really, really good information, whereas previously you'd have to spend, like you know, half an hour using Google and getting stuff from a bunch of different articles, you can now coalesce that down into very, very good prompts and get amazing answers.
Bob Belderbos:So it's exciting. So, yeah, exactly, you would Google around, you had to do a lot of vetting and of course now you have to still vet the answers that the AI gives you back, right, but it feels so much more targeted and so much faster.
Chris Williams:And that's an important part. I mean people always, you know, cut and paste from stack overflow and uh, they're they're now kind of doing the exact same thing with, with uh, agents. You still shouldn't do that. You still need to make sure that the answers that it is giving you are legit, because you will get bad answers Even if you say I mean, uh, just the other day I was, I was playing with a co-pilot and I said, hey, I want to, I want to create this application to do this thing, and it, you know it gave me the file structure, it created the all the systems and everything. But if you, if you actually like, tried to Terraform, apply that into an environment, it would cost you thousands of dollars a second because it picked really, really expensive instances and services. So, buyer beware you still have to be careful.
Bob Belderbos:Yeah, yeah, you definitely have to know what you're doing. Yep, and you're going to just, yeah, use those tools without pre-knowledge. That can go pretty bad.
Bob Belderbos:So yeah, no.
Bob Belderbos:AI is you're. You're all into it. Um, do you have a couple of quick tips for people that are newer to it and want to? You know either? You know, improve their prompting skills, but maybe more relevant in this context is build their own ai apps. But what is your advice for them to to get up to speed?
Chris Williams:um. Check out the gen ii boot camp that is starting february 1st. Yeah, uh, actually there. There there is a. There's a really good um free code camp 22 hour primer course that andrew brown put out. That has some good information in there on like you know how to get your feet wet there. There is a really good free articles on AWScom, on the AWS tutorial site to to get going with that as well. I mean there's there's so much information out there. I mean there's. If, if you go on LinkedIn and you and you do you do a search for AI tutorials or learn AI, you'll get a lot of really good responses to tons of articles. Devto has really good sections in there. I mean there's just an embarrassment of riches out there when it comes to learning, honestly, yeah.
Bob Belderbos:Awesome. Well, thanks for sharing all this insight. Super interesting and definitely something people should start looking into. Right Like this is really where the industry is going and yeah, it's always that saying right, ai won't replace you anytime soon, but the engineer that does might right, so this is a lot of fun and there's a lot of potential. So, yeah, we've covered a lot. I would recommend people to go watch the talk and read the article. Super interesting, thank you. Anything you want to add on the whole AI discussion?
Chris Williams:Embrace it, enjoy it, have fun with it. It's not going to go away anytime soon. That's very obvious now. So it's not something that is um, to be feared. I I would say this this is something that can can be a very powerful tool in your tool belt.
Bob Belderbos:Um, and it's uh, yeah, it is what it is, yeah yeah, you said like there were a lot of people interested in the code and stuff, but did you also notice a bit of fear, like like wow, this is going very fast now and maybe my DevOps job might be automated? Did you notice that when you presented these ideas?
Chris Williams:I think that that is a constant underlying tone. There is fear-mongering out there. There's people like hey AI is coming for my job and I can't answer that I don't know what's going to happen two years from now, how much better these models are going to get and what kinds of things are going to happen. But I think that if you are always learning and always staying on top of things, I think that you can ride the crest of this and stay up to speed. The old analogy is if you're the last horse and buggy cart creator in a world full of fords uh, ford model t's then you're going to get left behind. So start learning how to make ford model t's. So I mean, I don't know, it's, it's a, it is a. It is an uncertain time right now. So just keep learning and keep you know, trying to better yourself.
Bob Belderbos:Yeah, definitely interesting times as well, and, um, but yeah, usually we have seen with these trends in history that's, we all feared for jobs to be lost, but then a lot of new jobs got created as well, right, so it might even right each other out. So, right, exactly, cool. But it's also a logical trend with programming, right, they will always find new ways to automate more. So it's, it's also kind of what's expected from what developers typically will do, right, like to build bigger systems.
Chris Williams:So I actually found a meme on uh the programmer humor subreddit, and it was. It was a, a black and white photo of a lady holding a punch card and and, uh, she, and underneath it it was like the compilers are coming for my job. And I was like you know what it's? We've, we've been, we've been, uh, dealing with this fear for a long time.
Bob Belderbos:Yeah, that's awesome. We'll link that as well. Cool, we always wrap up with the books. What are you reading at the moment?
Chris Williams:Oh boy, okay, so I am currently reading Samantha Coyle's Go programming book, mark Tinderholt's um mastering Terraform book and um that. That's. That's it for right now. I've, I've, I've actually I've got a couple of other ones that I'm dabbling in, but nothing that I'm like really diving into. So, samantha Coyle, go. Mark Tinderholt, terraform.
Bob Belderbos:Yeah, awesome Packed one, right, I guess?
Chris Williams:Yes, yes, packed publishing. Yeah, they're both packed, I think yeah.
Bob Belderbos:Yeah, yeah, cool. Yeah, I'm all over the place Still reading that Python concurrency book about async IO. I picked up a couple of packed books because there was a promo so you could get books for five bucks, and definitely picked more ML and Gen AI. There's one, generative AI, in marketing and there's another one, gen AI, in finance, so I'm definitely on your wavelength Like, let's read up more what Gen AI is going to do in these disciplines, right To get ideas.
Chris Williams:Oh, Gen AI in marketing and finance. Are you going to put the links for those too?
Bob Belderbos:Yeah, different books, but working with somebody in PDM who works in finance. So I just found it interesting to see what Jenna could mean for his industry. And well, I'm officially, or in a previous life I studied finance Right. So it could be interesting to watch back or, you know, see what could happen there. And then marketing yeah, marketing is just what we always have to do right as as a business owner. So if we can automate anything there or streamline it, then kind of the same passion you have for devops, um, that will, that will be really good.
Chris Williams:So yeah, drop those links. I'd be curious to read those as well.
Bob Belderbos:That sounds interesting yeah, cool, I will do. Yeah, it was a pleasure catching up. Any final shout out or words, things you want to share?
Chris Williams:I would be remiss in not saying thank you to Julie Gunderson and the other, chris Williams, for helping make this happen and dragging me onto stage at reInvent. It was a huge honor and I them uh thinking of me to do that. We had an absolute blast doing it. It was. It was, uh, I think, 400 people in the audience. Uh, there's there's a really fun picture of us like doing a selfie with the audience. It was like the first um, uh, first talk of the first morning of of the monday of reinvent. So so we had crazy attendance and I was very, very nervous going into it.
Bob Belderbos:Yeah I think you can see the selfie moment that's at the start of that talk. It's on youtube again. So, yeah, yeah, check it out it's. It's a great. It's a great talk and I'm glad I I spotted your post on uh, on blue sky. Is that also where people should follow you, mainly on blue sky? Is that where you're active?
Chris Williams:yeah, yeah, if you go to blue sky, I am mistwirecom over there. Or or if you just google, mistwire, uh, I'm like the first three pages of hits, I'm all the things in there too yeah, you own that.
Bob Belderbos:Yeah, easy to find you. Yeah, yep, it is well, always a pleasure to catch up. Thanks for hopping on and sharing and, yeah, people reach out to Chris if you have any questions, feedback. I'm going to talk about DevOps and Gen AI. You're also in the PyBets community, so people can write yeah, I am. Yeah, they can talk to you there too. They can talk to you there too. Hit up chris there and uh, yeah, thanks again and we'll we'll talk soon again.
Julian Sequeira:Awesome cheers, bob cheers. Hey everyone. Thanks for tuning into the pie bites podcast. I really hope you enjoyed it. A quick message from me and bob before you go to get the most out of your experience with pie bites, including learning more python, engaging with other developers, learning about our guests, discussing these podcast episodes, and much, much more please join our community at pybytescircleso. The link is on the screen if you're watching this on YouTube and it's in the show notes for everyone else. When you join, make sure you introduce yourself, engage with myself and Bob and the many other developers in the community. It's one of the greatest things you can do to expand your knowledge and reach and network as a Python developer. We'll see you in the next episode and we will see you in the community. Bye, thank you. Thank you.