Let's talk about the elephant in the room when it comes to AI. That one word. Hallucinations. Right? When you rely on an AI chatbot to provide you something and it straight up lies, or it tells you a very non-truthful version of what you might be looking for. And if you don't keep up with the advancements of AI, you might still think that AI lies all the time. And well, if you're using the wrong model, if you're not using best practices, hallucinations, even in 2026, can still be a huge problem. But here's what not enough people are talking about with the latest technology in today's thinking models and the ability to very easily ground responses in your company's data without much tech know-how. Essentially get rid of at least the high rate of hallucinations. So that's why we're going to be diving into this sticky topic on the fifth volume of our Start Here series, talking about AI hallucinations, what they are, why they happen, and the right ways to reduce the risk. All right, let's get into it. I'm excited for today's conversation. I hope you are too. If you're new here, this is our fifth installment of the Start Here series. This is the essential podcast series to both learn the AI basics and to double down on your AI knowledge. So whether you are a beginner trying to understand where do I start with AI, or you're really just trying to deepen your expertise, our Start Here series is definitely for you. And if you haven't already, please go to starthearseries.com. Here's why. Well, number one, uh, that's gonna give you access, uh, free access to our inner circle community. All right, and you're gonna get straight into uh our Start Here Series uh channel there. So you can go back and listen to all of our uh Start Here series uh right there in our free community, but you can also get free access to our uh Prime Prompt and Polish, uh Prime Prompt Polish uh Chat GPT prompt engineering course. All right, and if you miss our last episode of Start Here series, that was volume four, we talked about the human AI collaboration and best practices for working alongside AI, which leads us right into today's show. Because if you are doing the best practices of human AI collaboration, what we tackled in volume four, then that leads you straight into reducing hallucinations. Because if you are working with AI in the right way versus blindly trusting its outputs, well, you're gonna uh be able to fight back against the hallucinations. So here's what we're gonna cover on today's show. First, we're gonna tell you what hallucinate what hallucinations are uh and why they are there. Then we're gonna show you how to assess the risks and talk about how models have improved on hallucinations, but they still run into them. And then last but not least, I'm going to suggest to you a kind of four-layer method to reduce the errors and show you how you don't really need to be that scared of hallucinations as long as you are being a smart human. All right, let's jump into it and start at the beginning. What the heck are hallucinations and why do they still happen? Well, in general, here's as as bluntly as I can put it: large language models, right? And go back and listen to uh volume one and volume two uh if you need a refresher on how large language models are built. But essentially, they scrape the entirety of the internet, online, offline data sets, and then humans train these models. So when you and I ask a question, uh right, through this process called reinforcement learning with human feedback, these smart people at you know, OpenAI and Google and Enthropic and all these other labs have trained models that when someone asks about XYZ, here's what you should respond. However, for the most part, AI models are very super smart next token prediction. Uh, right. So sometimes that means if something was maybe incorrect on the internet, or if a model is confused about what you're actually asking, it might present a made-up answer and it might do so very uh confidently, because at their core, AI models are trained to be helpful assistants. That is in almost every single system prompt uh that an AI model uses. So that's why sometimes they are gonna make things up because they want to be helpful more than anything else, uh, because they predict the next word from patterns. And that's kind of why they exist. So it's almost like maybe you've heard this uh this saying uh that hallucinations are a feature, not a bug. Because at the same time, kind of that same methodology of how they're built help large language models be extremely creative, strategic. And even if we're talking about you know, scientific discovery, drug discovery, uh, mathematics, that's actually how they're able to solve new problems that humans haven't been able to. Uh, but I think a lot of people, number one, are using the wrong model. They're not going through the you know context engineering 101 of how you should be working with a model. And they're using, right? They're using, like I said, they're using the wrong one. That's why hallucinations have been a huge problem. And they can get you and your company in a lot of trouble if you're not doing the best practices. But uh, here's what's changing models now they can think and they can reason, much like humans do, and that has led to a drastic reduction in hallucinations, right? Uh, you're obviously the average person is using way more inference, right? They're eating up way more tokens uh to go through basic problems. But that's why I think hallucinations are really on the uh decline. So the combination of better models that can think and reason like a human do, combined with kind of this rag-esque uh features that front-end large language models give you, uh, are really leading to that reduction. But this is how they work, uh, right? Uh, because people are always confused. Like, hey, if I give a large language model a simple problem, why does it get it wrong? Or, you know, a riddle, or asking it how many R's there are in the word strawberry, why does it get it wrong? Why is there this, you know, this really jagged, um, you know, almost polarizing output that a large language model can do? It can do something that's absolutely genius, but then it can get something absolutely wrong, right? So that is, well, because they are next token or next word predictions. They are based on patterns trained by humans, they don't verify the truth against reality necessarily. And it's that same capabilities that allow AIs to be extremely creative and to generate code and to write poetry. Well, at the same time, that can lead them to lie to us, right? Um, and by definition, a hallucination is just a confident answer containing either fabricated claims because the model just generates the text, or it can make up sources, right? Which is a big problem as well. So I'm gonna be talking about uh when we go over our kind of four-layer uh approach, how you can avoid that type of hallucination. So uh there's really different categories of hallucinations, right? One is it just lies, one is it it's not really lying, uh, but it's sounding overly confident and just giving you some generic information. And then other times it can just make up sources, right? So it might give you true facts, but it might make up where it got those facts from, right? And in most cases, like I said, hallucinations, if I'm being honest, in 2026, they're more human error than AI error. And I know a lot of people won't agree with that. That's fine. But if you get me a room full of humans that are using AI models the right way, right? Get uh a room full of humans that have taken hour free, you know, prime prompt polish uh courses, they're gonna run into very few hallucinations, right, compared to the average user. Because if you actually know how models work, how to feed them the right data, and how to check and verify on the back end, being a smart uh you know, human user, you're not gonna run into them very much. But let's talk about the early days because it was bad, right? So go back to the early models, the GPT-3 or the GPT 3.5, uh, you know, the early days of uh Chat GPT and late 2022. Uh studies showed that GPT-5 fabricated up to 40% of academic citations, right? Fast forward to the next version of GPT-4 that went down to about 29%. So uh hallucinations got a little less, but still rampant. Uh, fast forward to today, GPT-5.2 reports a 6.2 error rate on general queries, and open AI claims a 30% reduction in errors in GPT-5.2 versus GPT-5.1. So we're not talking, you know, of 30% fewer errors between GPT-5.2 and that version from three years ago. No, we're talking about in a three-month period, and that jump is huge. And I'm gonna explain why uh that jump has occurred and I think why I, you know, in a year from now, we may be not even talking or talking very much about hallucinations. And the main reason is just the ability for these labs, right? So I'm gonna give an example here from OpenAI, but I think that uh Anthropic and Google and OpenAI have made tremendous strides uh with their models and uh through a lot of different techniques, which I'm not gonna get into the technical side. Um, they've made AI models that have much more reliable outputs. And one of the reasons um is their ability to handle longer context. All right, so for our live stream audience here, I have a screenshot from uh OpenAI's GPT-5.2 um model release. And I want to kind of talk about this long context. So there is a test, it's essentially called the four needle test. So, what this is, they have it um pull out and they ask the model questions and kind of it's it's a needle in the haystack test. And they see over a wide range of a conversation, right? If you're using uh a model into the hundreds of thousands of tokens, right? Like at 256,000 tokens. So a very long conversation because what has happened in the past, a lot of these hallucinations come uh when you are hitting a longer point in the context window, right? So think of the 3 p.m. brain fog, right? Let's just say you work nine to five. At 3 p.m., you're probably not as sharp as you were at 9:30 a.m. Right when that second espresso hits and you're like, let's go, and you're firing all cylinders, right? For the most part, that's how large language models had been, I would say, even late into 2025. But think of that nine to five instead as a context window, right? Because all large language models have their constraints, they have their um kind of confines that you can't break through. And one of those is the context window. That's how much information a large language model can retain until it starts to forget. And potentially when it starts to forget, it will start to hallucinate. So even great models like GPT-5.1, as I go here on the results of that four needles test, uh, you saw its ability to properly pull out those facts over a large context window. When it started out, right, it was, you know, about 95%. It was very good at GPT-5.1 thinking. But then toward the end of that context window, it dropped down to like 45%, a 45% uh ability to recall that information. Whereas GPT-5.2 thinking, hardly any decline at all, right? I believe it was at like a 95 or 96 percent at the end of the context window. Okay, so think that very drained human at 3 p.m., they might not be able to recall facts, right? But with today's, and when I say today's, I'm saying the latest generation Gemini 3 Pro, Opus 4.5 from Anthropic, Claude Opus 4.5, and GPT-5.2 thinking from OpenAI, their ability to recall information across that context window has greatly improved, which is one of the main reasons why that hallucination rate has gone down, because now models can think. They can't only think and plan ahead and reason and call tools on their own to uh provide you more accurate information, but they're able to do that across the entirety of their context window, which is one of the main reasons why hallucinations are going down. Here's why this still matters for your business. Well, let me cut it to you straight. One of the biggest gaps right now, and one of the most blaring uh things that are going wrong with AI implementation across the enterprise is a lack of training and education. Because you have companies that I've talked to personally, there's plenty of case studies out there that are rolling out access, whether you're talking about, you know, Microsoft Copilot 365 licenses, uh, you know, ChatGPT Enterprise, Claude Enterprise, Gemini Business, Gemini Enterprise, et cetera. They're rolling out AI access, you know, a lot of this was in 2025 to thousands, tens of thousands of employees, but not giving any best practice training or even education, which is why hallucinations are still rampant. People don't even understand. Oh, I need to uh, you know, click this model selector and choose the best model for the job. Or I should be having this model uh, you know, call a certain tool. It should be running Python. I should be uploading files. People don't know the basics. Yeah, they're just trying to save as much time as possible with AI. And that's led to a lot of hallucinations with a lot of uh high profile and a lot of press around it, right? So, uh, as an example, there was uh the AI Hallucination Cases database at HEC Paris, documented 486 different legal cases worldwide involving fabricated AI content. Yeah, a lot of these like hallucination stories, they come at the most embarrassing point, which is in the legal sector, right? So uh there's been over 128 lawyers that have been cited for filings with hallucinated cases. One of the most popular is the, I think it's the MEDA uh versus Aviancha. I don't know if I got the pronunciation of that right. I might have hallucinated the pronunciation, but that's where you saw uh attorneys sanctioned for six fake citations. That was one of the more uh infamous cases of AI illegal early on. And even uh Deloitte reportedly refunded part of a $300,000 Australian government contract after it was uh they reportedly found some AI-generated phantom citations in their report. So this isn't just people who are, you know, using AI to write blog posts, right? A lot of times some of these hallucinations make their way to the forefront in very high, uh, high value and um just extremely visible places, right? Like consulting companies, like lawyers. We've seen plenty on the financial side as well. So that's why this still matters, right? Because maybe what I laid out for you, and like saying, hey, if you use the right model and uh look at these, you know, the needle and the haystack, that's improving the context. When well, most people aren't using models the right way, and that's why this is still extremely important for your business. Don't worry, we're gonna lay it out for you here. But I want I want to be very clear. Just because I am extremely optimistic about hallucinations decreasing with proper human involvement and training, it doesn't mean they're going away, right? The very nature of what a large language model is, how it works, means that hallucinations will be there for a while, uh, right? Uh, because the models are always going to optimize for what word or what sets of words could be next, even if those things are incorrect. Think of how many times that you've been on the internet researching something and you're like, ah, this isn't right. That's not right, right? In the same way, well, large language models take all that information from the web. So let's say you are a domain expert and there's something, you know, uh, that in your field people constantly get wrong, or there's some uh some pieces of information out there that exist that are circulating that aren't exactly true. Well, in the same way that you might find incorrect data on websites, or you might go to a conference or hear colleagues speak about something that you're an expert in, and you're like, nope, that's not right. Well, a lot of times large language models are reflecting inaccuracies uh or uh, you know, half lies or half truths that exist in the real world. Second, uh, the second reason why hallucinations probably are gonna go away is well, when their needed information isn't there by default, by default, large language models are gonna fill in the gap, right? They're gonna do everything they can to be a helpful assistant because that is the default behavior, unless you change it and you should, and I'll tell you how. And then third, the chat inference, the chat interfaces rewards that fast, confident answers, right? I'm not gonna get too much into uh you know reinforcement learning with human feedback and uh kind of scaling laws and inference and all that, but for the most part, chatbots are trained to quickly give you an answer and being token efficient, right? So if it thinks, oh, I can spit out an answer rather quickly, it may just do that unless you've told it not to. Right. So uh they'll at times will reward kind of uh giving you an answer even if it's not confident versus just saying, I don't know. I will say that's models more of late 2025, because the models from 2026, you'll see by default, again, using the right models uh in the right context, they are gonna say more. Uh, and I'd love to hear if you've seen this more. I have models that actually say, I don't know, or I'm not sure, right? Even without giving uh you know anything extra in your prompt, anything in the special instructions. Today's models are more likely to say, Hey, I'm not, I'm not sure, which is a good thing. So here's how to spot and cut down on hallucinations, a quick four-step plan. All right. Number one, you need to change the model's behavior. Uh, like I said, I do think that um, I think that in the future, the AI, the AI labs are gonna find that sweet spot between uh instructing models to be helpful assistance, but not too helpful and not fabricating things. But in the short run, you can do that, right? So whether it's in your prompting sequence when you are working with a large language model, or what I would recommend is setting custom instructions, right? So uh there's different kinds of places that you can set custom instructions. But uh, if you are non-technical, if you're not really sure what that means, that's in there's settings in most of the big providers that you can essentially put in your own rules and any chat that you use, any response uh that you are uh using the model, it will always go through and read and usually adhere to the own set of custom instructions that you put in there. So, you know, even putting something simple, like if you're not certain or the information isn't provided, say I don't know rather than guessing. Something simple as that. I I have a set of uh custom instructions uh that I've done a lot of testing with over the years that are much more robust. But even putting something simple like that, like hey, if you're not 100% sure, say you're not sure, right? Or to require every factual claim to include a source, or you know, labeling everything afterwards, right? Saying, hey, this is what I'm 100%, you know, telling the model, this is what I'm 100% sure on, this is what I'm not sure on. You can even do something like uh in each um response, having it give you giving you a confidence score. That's another great thing that I that you could do. And then have it to separate facts from inferences, right? Because here's the thing, especially if you're using it as a strategy, creative partner, brainstorming, not all those things are black and white, right? Maybe 99% of what you might use a large language model for is in that gray area, right? It's strategies, creative, right? Being creative. So you have it separate facts from inferences. You can ask for a table with columns for confirmed fact versus assumption in a structured output. So number one is doing that either in your prompt or in your custom instructions. Number two, my gosh, the fact that we have this and you don't have to pay any extra, is wild. Uh, right. Make it retrieve the information, right? This whole context engineering, something I've been teaching since 2023 before it was a thing. Uh, right. If if you've taken our free prime prompt polish course, this is the refine cue in the priming. So uh we call it the fetch in the insights, but essentially using a smaller simplified version of RAG, retrieval meta generation. Now you have a simplified version of RAG available with a couple of clicks by connecting your company's data, right? So both in obviously Microsoft 365 Copilot, uh in uh Claude, in Google Gemini, in OpenAI's chat, GPT. They have different ways for you to connect your business data in a few clicks. So you can essentially uh ground it in a way you can ground it in a way you kind of can't. But the combination of number one, those kind of custom instructions, and number two, first putting your company's information. If you combine those two things and you say, always check, you know, this ABC document uh before you respond. And then if you make sure you check and verify it that it did, I mean, those two things right there, amazing. But it doesn't matter if you're, you know, a Microsoft organization, you can connect your, you know, your OneDrive and SharePoint data uh in OpenAI's products, right? If you're using uh, you know, Microsoft 365 Copilot, like online, their online version, you can connect your if you use Google, if you use Google Drive, right? So being able to connect your company's data to a large language model and then instructing it to always um look at those things first is huge and it makes a big difference. So there was a 2024 Stanford study that found RAG combined with reinforcement learning with human feedback and guardrails, achieved 96% hallucination reduction versus the baseline, right? Just doing those basic things, right? Like having a version of your company's data and proper uh instructions are going to cut down on hallucinations, just those two things alone. And then steps three and four, kind of combined into one, is just the verification workflows and agent safety. So it's kind of like three and three B. All right. So what is that? Well, you have to be able to catch errors before they escape. This is the uh the expert-driven loops that I always talk about, not the lazy, passive human in the loop. I'm talking about the active, proactive, expert-driven loops. So, as these models, they think by default. You know, I say GPT-5.2, uh, right. And if you're listening to this uh this episode in July, right, maybe it's GPT-5.3 or Gemini 3.5. I don't know. But today's latest models are agentic by nature. They're gonna go through, they're gonna think, they're gonna decide how much to think. Should I spend five minutes on this answer? Should I spend 30 seconds, right? But you have control over that. Uh, and you can always kind of build, and I always recommend doing something like this build a second pass review where that model's only job is checking claims. That's why I also do kind of a mixture of models set up. But I have systems set up where you know, I maybe have a Google Gemini Deep Research run, and then I'll take some of those results and I will verify it. And I have something set up. Uh, the model that I like doing this for is GPT-5.2 Pro. Um, I will then use GPT-5.2 Pro only to go through and verify every single thing that I get from a different model. That doesn't mean I don't trust model A versus model B, right? And sometimes I'll flip-flop it. It means you should always be doing this, right? Especially for high value, highly visible projects, right? If you're just trying to see, like, you know, what's the weather next week or, you know, something topical, right? Like, what's the best, you know, what are the three best softwares for tackling this issue? I don't think you necessarily need to go through all those steps. But if this, if you are using an AI model as part of a high value workflow, you should definitely be doing this second pass review. Uh, and then again, requiring even that second pass to show the sources. And then more than anything, you need to be tracing and observing how these models are getting to those conclusions. And what that means is well, you should be using a thinking model and then reading the summarized or the chain of thought, right? So, what that is, uh most of the models, you know, different models give you different level of uh visibility on what they're actually doing under the hood. But just if you were to hand off an important assignment to a brand new employee, hopefully you wouldn't just hand that off to the client, right? You would take a step in between and be like, hey, new employee, tell me how you got to this answer, right? Let's say they spent an entire week on this big project. I would hope you would take at least an hour or so to sit down with that new employee and be like, okay, how do we get to this conclusion? Walk me through, right? So the good thing with these agentic models by default, you can see all that. And you, smart human, need to be able to go through and check and see what it did at each and every step. Did it follow your instructions of, you know, uh, here's here's what's factual, here's what I inferred. Did it make sure to look at the right documents at the right time? You can go through and sequentially check all of those things. And if it didn't do it, then you can course correct. But that is layers three and four, and especially as AI can take action, right? As we're talking about agentic AI, that's when you really need to treat it like a junior employee. But no, they can still make things up. And like I said, most of the time, if you are doing the proper context engineering one-on-one, if you're using the right model, and then if you're going through kind of layers three and four right here, with the second pass in the observability and traceability of going through that chain of thought, hallucinations aren't gonna be the biggest problem for you, right? You still need to have those expert-driven loops because as you're looking at that chain of thought, as you're providing data on the front end, you have to do that at an expert level. You can't just say, here's a thousand files. Good luck, agent. No, you need to, you know, point them in the right direction. The same thing, checking responses on the back end. You have to know what all of that means. But if you have expert-driven loops and if you go through those four steps that I talked about, I think hallucinations are no longer going to be the big elephant in the room. You're gonna be able to reduce them. All right. But remember, it's not going away. Hallucinations are a property that you need to manage, not something that you hope will get fixed one day. And it's not that the winners right now are picking the best models, they're just building those four steps of verification into every single uh episode or into every single piece of work that they're doing. All right. Speaking of episodes, I hope this episode was helpful. Uh, because the start here series, I want you to know all the details, right? Um, as someone that's done this everyday AI thing now more than 700 times, I've got to speak with some of the smartest people in the world. I've realized if you're educated, if you're trained, if you understand how these models work, and if you keep up, that's that's the big caveat there, right? If you keep up, you don't have to be as worried about hallucinations. I'm not saying you can write them off. You shouldn't. But if you're going through the right steps and what we went over in today's episode, you are definitely able to reduce the risk and to get more out of large language models because that's what we're all about here at EverydayAI, cutting through the fluff, giving you the facts and the right information to grow your company and your career. So I hope volume five of the Start Here series was helpful. I hope you know more about hallucinations now, why they happen, and how you can reduce the risk. So if this was helpful, please go to starthearseries.com. That's gonna give you free access to our inner circle community, uh, the prompt engineering course, as well as an easy spot to go listen and catch up and engage with others who are going through this Start Here series with you. So thank you for tuning in. I hope to see you back later for more everyday AI. Thanks, y'all.