Develop Yourself

#287 - From Smoothie King to AI Engineer

• Brian Jenney

Share episode

Ryan is a current student at Parsity who build an app for his employer, Smoothie King, to suggest drinks in a chat interface using a powerful and lesser-known AI technology: RAG.

RAG stands for retrieval augmented generation. Basically, providing information (like smoothie recipes) to an AI model so it can return a highly specific response.

Ryan breaks down how he finds the time to build side projects like this and how he built this app.

Want to build your own AI-powered app? Check out this project: parsity.io/ai-with-rag

Connect with Ryan here: https://www.linkedin.com/in/rhardin378/

Send us a text

Shameless Plugs

Free 5 day email course to go from HTML to AI

Got a question you want answered on the pod? Drop it here

Apply for 1 of 12 spots at Parsity - Learn to build complex software, work with LLMs and launch your career.

AI Bootcamp (NEW) - for software developers who want to be the expert on their team when it comes to integrating AI into web applications.

SPEAKER_01:

Welcome to the Develop Yourself podcast, where we teach you everything you need to land your first job as a software developer by learning to develop yourself, your skills, your network, and more. I'm Brian, your host. Today I have a special guest on the show, Ryan, who works at Smoothie King, and he's going to talk about this really interesting app that he's built, which is so much cooler than the majority of projects you see out nowadays. And I think it's really important that Ryan shows us that you don't have to quit your job to start working in tech. So he's at Smoothie King, which as you can probably imagine, not a tech company, it's not a startup, it's a normal business. And he ended up building this internal AI tool that has helped make his team more efficient. And he's using a technology which I've spoken about on this show quite a lot: Rag, Retrieval Augmented Generation. We're going to break down how he's using it, the product he's built, and how this is going to help him advance in his career. Thank you so much for being on the show.

SPEAKER_00:

Hey, thanks. Thanks for having me on. I'm I'm excited to talk about what I've built and excited to be on the show.

SPEAKER_01:

Dude, me too, man. So earlier today, you actually gave me the link to this particular project you've built. Can you just kind of walk people through this chat interface, this app that you've built?

SPEAKER_00:

Of course. It's it's very cool. So it's um built something I wanted it to be very easy to use, um, some so much so that anyone from like a high schooler to adult to an older age uh adult would be able to use it. Um it is a AI-powered search engine tool that allows me to search different menu items through a chat interface similar to something like Gemini or Chat GBT. Um it allows a user to ask questions that are smart, which are much higher than your keyword searches. So I can ask it things like what's a smoothie that has strawberries and at least 30 grams of protein, and it's gonna look through a vector database that I have set up that's been trained on data from the Smoothie King website that I've had to scrape, and it's able to give me back a strong response that I send through a re-ranker, which allows me to get even better results and return them in a friendly way to uh UI. So I'm able to show a top five recommendations, macros for it, and then also recommend other things through a share link, which it will say something like, for instance, when I copy and send it to someone, it will say, Hey, try out the for instance, uh, one smoothie maybe called like the pineapple surf, the pineapple surf, and it'll send you a link that will go to my uh chat bot and send back the same responses that were seen from the original link.

SPEAKER_01:

This is this is really cool. And it's funny that like all the things you just said, I've been asking these kinds of questions in an interview. I'm trying the for a company I just left, we were trying to hire for an AI engineer, and you just said all the things we were looking for. It's pretty nuts because we were interviewing people that like had you know Google experience, you know, five years, seven years, ten years experience, and most of them didn't have much experience with RAG. And I'm not saying this to down anybody, I'm just saying it's such a new thing that a lot of people don't have a lot of practical experience with it. And you just like name drop some things that I think are super important, like re-ranking all this stuff. Before we get into like more about this app, because I was using it earlier today and it's really cool because I was typing in like, what's a good drink for like a hot summer day or something? And it gave me back a pretty interesting response. And I said, I don't like bananas. And it gave me back a better response. I was using it, I'm like, this is actually super useful. Before we go a little bit more into the details of it, for people that don't know what rag is, a lot of people are now hearing this term. It's a term I've been throwing around quite a lot, but rag stands for retrieval augmented generation. But like in your own words, like how would you describe rag to somebody that doesn't really know what rag is?

SPEAKER_00:

Um, so a cool thing I've been able to do is um explaining this to some of my high schoolers that may not have a strong technical background or any kind of a technical background. Yeah. I've been I'm kind of describing it like it's a rag is a way for you to get chat GBT like answers, but they're grounded in a source of truth. So rag is able allows you to kind of use a vector search, get things that are gonna be in your original data, it's gonna prevent hallucinating, which is the biggest thing. Yes, especially in something with nutrition.

SPEAKER_01:

Yeah, that's a huge one. I'm glad you said too, because hallucinations are another thing that I I think that a lot of people seem to be unaware of this. That like when you ask ChatGPT a question, it's not guaranteed to give you a correct response, it's guaranteed to give you a response. And the reason why we use something like retrieval augmented generation, which you like really eloquently put it, is to ground it in a source of truth. So you're doing something with proprietary data, right? Like data that ChatGPT just wouldn't know. If I if I ask ChatGPT, like, hey, um, there's no Smoothie Kings, I don't think, in California, right? So if I said, hey, I want this kind of thing from Smoothie King, it may need to go do an internet search. There's no guarantee that it's going to get the accurate information. Uh, and even if it does do the internet search, it could just return me either outdated data or something that makes no sense, or it could just make up something based on whatever's in its knowledge base. So to give it the most accurate and up-to-date information, you would need to feed that to the large language model at the time that it's returning that response. And so, how did you get the data? Because this is the part I think that is usually the most complex, tends to be where people have the most difficulty because storing it in a vector database, as you probably saw, it's fairly trivial, but getting the information and then breaking it down so you can feed it into a vector database is where most people have a lot of trouble. So, how did you go about this?

SPEAKER_00:

So, this this was actually a a bit of a process. Yeah, I bet. But well, well worth it. Um I used puppeteer to scrape it's a JS uh uh headless browser. Right, right. It's a headless browser that can be used with JavaScript. Something you typically, when you think of scraping, you think more of like a language like Python. Um so I use Puppeteer, so I could use JavaScript, which I'm more accustomed to. And I was able to go to the Smoothie King website and find all their publicly available information on different smoothies and run a uh a bunch of different functions together, or also known as scripts, to get exact smoothie names, ingredients, all the macro details I needed from the website. And since it was a well-designed website, the the data was very normalized. So once I could access all the different properties that we need through the DOM for one section, it was kind of I just had to add some edge cases and I can run through the page pretty quickly.

SPEAKER_01:

Nice. And how'd you break up and like and the next part is so you you get the data, um, and this is probably I'm guessing large amounts of text, but I don't know. Did you have did you have any way that you had to break it up? Because this is what's called chunking. And chunking is when you break down data because a vector database, you can't store something like um like uh a novella, for example. You can't store like a 20-page document. So did you have to figure out how to break these up? And if so, how did you decide how to do that?

SPEAKER_00:

So the the cool thing I did with this was I um I decided the way I wanted my data, the data I would need to be formatted, and I saved each and every uh different smoothie as a JSON object in a big JSON file. Okay, and then from there I could normalize the data as I uploaded it into Pinecone. So it was ideally I was able to chunk different smoothies together and have it be different categories and then a small s something that's only gonna be maybe 10 fields and a very small payload.

SPEAKER_01:

Okay, okay, very cool. And what was the vector database you chose?

SPEAKER_00:

Um, I went with Pinecone. Um I've been hearing a lot of good things with Pine Cone, and I really wanted to get my hands dirty with it.

SPEAKER_01:

Yeah, I've I've used Pine Cone, I've used Quadrant, and I've used Vertex AI so far. And uh if I'm being opinionated, which I am, I wouldn't use Vertex AI. Pinecone is really easy to get started with and great. I think quadrant is really cool. The one thing I like about quadrant, maybe more than Pine Cone, is that it has a visualization graph, which I find fairly useful. So I can see like where clusters of data is. So you can like, for example, in yours, certain smoothies may be clustered together and you can see does this and does this make sense intuitively because you're gonna be familiar with the data and say, okay, this does make sense or ooh, something's off here. Maybe I need to actually look at how I'm doing this. So how did you how did you learn this stuff? We're we're not teaching this at parsity like right this moment. I I added a bunch of curriculum that I'm gonna release soon, but how did you learn this?

SPEAKER_00:

Um, so I so Brian did actually he uh you did. Uh you you let out a project not too long ago.

SPEAKER_01:

That's right. Yeah, it was fun.

SPEAKER_00:

Um, that was a rag bot trained on a bunch of your posts for LinkedIn, that's right. And so I I kind of like thought of that and I thought I've really I need to find a good use case for me. So I I came up with this idea for the search engine and looked further into where I would need re-ranking different trade-offs I had to make to be cost effective. So for instance, I had to use a model from Chat GPT, like a chat 3.5, which is a bit of an older model, but for what I needed if for it was good enough that it would still provide me a nice readable feedback because the rag implementation was doing all the heavy lifting. All I needed was something to make it a little more human readable.

SPEAKER_01:

Yes, that's why it's probably so fast. I was noticing that I got pretty fast responses. That was really clever, man. Dude, it's it's it's really wild hearing you say all this stuff because, like, after just doing this interview process, and then I was just doing actually a video before we did this, looking at AI engineering roles, and all the things that you're doing are what a lot of companies, especially in San Francisco, are looking for right now. And I believe this must be the case in in Austin too. You didn't use any Python at all, right? I think this is another big like misunderstanding. Everybody thinks, oh, I I gotta use Python if I want to like use AI or they gotta become a machine learning engineer or something to build something like this. What was the stack? Like, what was the exact stack you've used for this?

SPEAKER_00:

So for this, I use Next.js um for my for my uh framework with TypeScript on the front end, a little bit of TypeScript on the back end, um Pine Cone for my database. I used OpenAI embeddings, I use a re-ranker from the Pine Cone and the GPT from OpenAI to get my human readable responses back.

SPEAKER_01:

Dude, this is uh yeah, that that sounds like basically what I've done at the last two companies where I've left. Obviously, a bit more complex just because you know they have we have more users and things like that, but ultimately what you've built is a version of what I've just built at the last two startups with like larger teams of people. This is like kind of blows my mind because uh I haven't met a lot of people doing this in general, uh, even at a professional level. So it's even cooler to see somebody earlier in their career doing this kind of stuff. I mean, now that you've seen it, it you see probably how useful it is, right? Like, because you can immediately see, yeah, how do you plan on using this at work? Do you see this as more a customer facing or something that you could use internally or both?

SPEAKER_00:

I think it's definitely something that could be used for both for sure. Um I think I I implemented the share feature in there for the customer facing, especially. The plan is if I can get uh the plan is to make a pitch to corporate to see if they like the idea and implemented. I could be having a customer side and then uh internal side as well.

SPEAKER_01:

Yeah.

SPEAKER_00:

Kind of allowing us to make better recommendations for the customer while also allowing us to upsell a little bit more and have that extra kind of it factor, if you will.

SPEAKER_01:

Yeah. Um, and I and I can imagine it like like most companies, there feels like there's this top-down push for using AI. I I don't know if you've seen or heard this, it it's Smoothie King, but is this something your company is like actively pursuing? Are they like looking like how can we use AI in our day-to-day operations? Or is that even on the radar?

SPEAKER_00:

Well, as as far as on the ground, I'm I'm not seeing it anywhere really on the radar right now, maybe in the far future, but I'm not seeing any kind of like implementation uh currently right now. So I think there's really a great opportunity for it.

SPEAKER_01:

Yeah, yeah. And I think this is like the the big thing that I wish more people would do. Like you're doing all the I feel like I feel like you're like my salesperson or something like that, because this is all stuff that I think is really important. Like it's it's kind of interesting because you, even it very early in your career, have built something not only useful, but something that a lot of CEOs and people higher up in the food chain at these corporations are like desperate to start using because they know at many companies I need to use AI. Uh, I've been told to use AI, we have investment in AI, but they don't know what to do. And and there's been some studies out there, like 95% of AI projects fail. And and essentially it boils down to people at big corporations saying, hey, just use Chat GPT or we're gonna buy co-pilot for business, which is, you know, it's it's all right. But but things like this are where you have that cool wow, that it factor, and then if you can um sell it to the higher ups, it it puts you in a really, really cool, advantageous position because then it's like, oh, this guy's the expert, right? And then you become like elevated in status, which I think is a really smart move, to be honest with you. Um, how hard was it to learn this stuff? Because I think this is also a big misconception. Like, I I don't want to make it sound like, oh, it's easy, but like at Parsity, we we've done a lot of work on teaching traditional full stack software engineering, and I think people think that like AI engineering is so wildly different, but how different was it for you to pick up this skill? Like, was it how what was the difficulty level of adding this new skill to your tool belt?

SPEAKER_00:

Um, I'd say I'd say it was probably a like a six or a seven, not too bad. I um I used a lot of in in uh in editor like co-pilot assisted for a lot of the kind of more structure that allowed me to really dig into it harder into like the more important parts.

SPEAKER_01:

Yeah.

SPEAKER_00:

Um a lot of questioning what you get back is the big part. And so as as someone who learned through a really strong foundation at Parsity, I was able to use my fundamentals to kind of like see what's what's good code from bad code kind of to navigate where I need to research more outside of an LLM response and diving into documentation and being able to connect the two.

SPEAKER_01:

Yeah, that's uh that's a really solid approach. And you said you said copilot are you not using um uh cursor or claude, or you are you mostly just using copilot?

SPEAKER_00:

Which is actually I'm I'm mainly just using uh copilot and uh VS code just because it it has a lot, it honestly has a lot of good models for I pretty cheap.

SPEAKER_01:

I I I feel you like I got kind of spoiled on using cursor and claude, and uh I'll just be honest. I look back at some of the code I've written in the last year, which has been very much AI assisted because we were rushing to meet some deadlines and things like that. And I'm just gonna be completely honest, I'm not proud of that code. I'm really not. So I think I think you're doing, I think I need to take, I might need to take a page out of your book and slow down and go back back to maybe using Copilot as my default for a while. I I don't know. It thinks things are changing so fast uh with this kind of stuff. Um, if somebody's out there listening, you know, you you're much closer um in experience to many people that are listening to this show. What are some things like you might recommend they do, or or maybe in hindsight that you would have done with this project? For other people out there, it's like, oh, I really want to learn this too. You know, like what what do you think they should do?

SPEAKER_00:

Honestly, just don't be afraid to get started. Um, there's if if you can do a Google search, you can honestly go to Parstee's site, and I'm pretty sure Brian still has a tutorial on the rag for free. Honestly, it's just the mainly thing is getting started and staying consistent with it. It's so easy to fall into the trap of really diving into something for a day than leaving it in the dust for a couple of weeks to come in back to it.

SPEAKER_01:

100%. Yeah, that's the big thing. And and last question, um, which I think a lot of people are gonna be more interested in as well, is you you work full time. Um, how do you find the time to do this?

SPEAKER_00:

Um, yeah, so I I definitely work full time. I work, I work uh 50 hours, sometimes more a week. Damn. Okay. Um so a lot of it's just finding the right time to work on stuff. So uh a lot of it's done on days off or shorter days. Um it's it really has to become a priority, but it's not undoable. It's not it's not uh something that's can't be done. It's it's just you gotta have that will, that grit, really you gotta you gotta have a passion for it somewhere in there.

SPEAKER_01:

I I couldn't agree more. I think that's the big thing. It's like as much as I think you can you can almost teach anybody, I don't really believe you could teach anybody. Um, but I do but I do know that the people that are successful, like I you don't look you don't look um down and out, you look excited, you show this to me, you're a juice to talk about it. I was excited to talk about it with you too. And that kind of like genuine excitement you you can't fake. And I think that's honestly what keeps people moving forward. And that's one big thing. I'd say if you don't have some passion for it, then I I might just pick something else to do. But if you do, this is like really, I think a cool time to start building this because there's so much doom and gloom, but it's really cool to hear you building stuff like this. For people that want to connect with you, where can they find you? Oh, you can find me on LinkedIn. I'll have a link in the show notes, and I don't want to like blow up your your app by having it in the show notes, but like, do you think it'll be publicly available? Are you just gonna have it like an internal tool only?

SPEAKER_00:

Um I I'm I'm I'm probably gonna post it to LinkedIn with and see how it does as bar as usage. Um put a rate limiter, I put a rate limiter on it smart on the API end. Very smart. Only allow a certain amount of tries per hour. So we'll see how that works. And I want it to be publicly available to be used, but if not, um the plan is to have a really nice demo. You can always request access to it.

SPEAKER_01:

Awesome. Well, hey man, thank you so much for for sharing this. Uh, really cool, and I'm looking forward to to seeing how it evolves. I appreciate it so much. It's been a pleasure. Thank you. That'll do it for today's episode of the Develop Yourself podcast. If you're curious about switching careers and becoming a software developer and building complex software and want to work directly with me and my team, go to parsity.io. And if you want more information, feel free to schedule a chat by just clicking the link in the show notes. See you next week.

Podcasts we love

Check out these other fine podcasts recommended by us, not an algorithm.