Don't Make KPMG's AI Mistake

KPMG recently withdrew an AI report after concerns were raised about its content... 

In this week's episode of Glitchd, we break down what happened, why it matters, and what businesses can learn from it.

As more organisations rely on AI to help write reports, proposals, presentations and thought leadership, one question is becoming increasingly important:

This episode explores the risks of relying too heavily on AI-generated content, why human oversight still matters, and the practical governance every organisation should have in place to protect its reputation.

If AI is part of how your organisation creates content or makes decisions, this episode is for you.

🎙️ New episodes every Tuesday.

Find out more about Sharon and SKS Professional Services at skspsl.com.

https://skspsl.com/

Transcript

SPEAKER_00 0:06

Sharon here, welcome back to Glitched, the podcast for businesses navigating AI. New episodes drop every Tuesday. Let's get into it. Earlier this month in June, KPMG published a report on agentic AI. Then the organizations named within the report started speaking up questioning the accuracy of the report. Interesting. These organizations included the NHS, Transport for London, UBS and Swiss Federal Railways. Then GPT-0 ran an analysis on these citations, and out of the 45 references within the report, only five were found to be accurate. Five out of 45. The rest contained paraphrased titles, fabricated sources, attribution errors, or links that pointed nowhere meaningful. KPMG pulled the report soon after, and this is, I have to say, I have to remind you who KPMG is. KPMG is one of the most recognized professional services brands in the entire world. Like this is just insane. So we are talking here about what happens when AI generates content and humans put their name on it without checking it. There's no proper human review. There is no effective human in the loop. This is not just about KPMG making a mistake. I have to make that clear. So what really happened? AI models generate plausible sounding citations the way they generate everything else. By predicting what should be there, not by verifying that it is. The businesses named in these case studies did not consent to being used as proof points. They read about themselves in a published report and didn't recognize what was being written about them. That's a different category of a problem from a typo or a misquote. This is big reputational damage. And not just on KPMG, but the organizations that they mentioned within this report that they failed to put the right processes in place, such as effective human review and human in the loop. And I think this is what should concern every business leader listening. KPMG have resources, plenty of them. They have lots of reviewers and lots of processes. So if this can happen at that scale, it can happen everywhere. But the question is, is anyone checking it before it is used or goes out to the public? Is there human review and is there human in the loop? Obviously, with KPMG, there wasn't here. But for you, do you have effective human review and human in the loop? Worth pausing on that because this is exactly where governance becomes practical rather than theoretical. At SKS Professional Services, I work with businesses on exactly those types of questions. How do you use AI responsibly in a way that protects your reputation and keeps you on the right side of the law and the public and your clients? If you want to have that conversation, head on over to skspsl.com or use the contact page to get in touch directly. The link is in the show notes. Right, let's keep going. Here's where things get quite interesting. When a business publishes content, it owns that content. Right? The fact that it's AI generated is not a defense and it's not an excuse. If your marketing report, for example, contains a fabricated statistic, your company said that. If your thought leadership piece attributes a quote to someone who never said it, your company attributed it. If your case study names a client in a context they did not consent to, your company named them. You can't point the finger to the AI because your business is using that AI. We are not yet in a world where regulators have caught up with every scenario, but we are absolutely in a world where reputational damage moves faster than legal process. The KPMG story was picked up globally within only a few hours of the report being pulled. This kind of exposure does not wait for a court ruling. But there's also a subtle risk. Businesses are increasingly using AI to produce internal reports, board papers, market analysis, and investment proposals. If those documents contain hallucinated sources of fabricated data and decisions are made based on them, the liability question does not disappear just because the document was internal. I think it actually gets more complicated. But enough of the issues. Let's talk about what good practice looks like. You see, the answer is not to stop using AI for content. AI is very, very helpful. AI presents an amazing opportunity for businesses worldwide. The answer is to treat AI outputs the way that you would treat work from your very capable colleague. You see, for a long time people have said treat AI like a very junior intern, someone who's just joined the business and is very junior, they don't know what they're doing. I don't think that's the right approach. Treat it as a very capable colleague because AI is very, very intelligent, much more intelligent than an intern or a very, very junior colleague, in my view. So every citation needs a human eye on the source. Every statistic needs to trace back to something real, something legitimate. Every named organization or individual or attributed quote needs explicit sign-off before it goes anywhere. Do your research. Do not take what AI gives out as fact. You need to fact-check it. And I don't think this is an unreasonable burden. I think it's very worth it. If KPMG had taken the time, a few hours, maybe a day or half a day to do this, we wouldn't even be talking about this. Checking with human in the loop, human review that is effective, is the minimum standard you would apply to anything else published under your brand. So why aren't businesses doing it with AI? There's also a structural point worth making. The businesses that will come out of this period are the ones who build clear internal rules and processes about where AI can operate without close oversight and where it cannot. Processes are so important. It's really interesting that when I mention the word processes, people think it's going to slow everything down, that it kills innovation, it kills speed. It doesn't. It just needs time at the beginning to be well thought out, to be communicated throughout the business, but the right processes really accelerate success. They accelerate opportunity and they accelerate the ability to overcome challenges. So clear processes or clear rules are very important guardrails when it comes to AI, especially. Content that goes external, content that names third parties or content that makes factual claims, these are categories that absolutely need human verification as a fixed step, not an optional one. Maybe the easier or maybe the more straightforward way to go about it would be simply to ensure there is human review in every AI output. No matter what you're using, always keep human in the loop. Right, with KPMG, they will recover. They have a brand equity to overcome this and weather this. A smaller business, however, maybe a consultancy, etc., this would turn out quite differently for them. So in closing, take away this. Five citations out of 45 were found to be correct. The rest were misleading. Four organizations read their own names in a report that said a lot about them that wasn't true. And the tool did what AI tools do. It generated something convincing, and KPMG put their name on it, nobody challenged it, nobody checked it, there was no effective human review or human in the loop process. That is a huge lesson for us to learn. It's not that AI cannot be trusted, but it reminds us that trust without verification is not a strategy. It is not an effective process. I'm Sharon, this is Glitched. New episodes every Tuesday, and if this one was useful, share it, like, and if it raised questions about how your business is handling AI governance, please do get in touch via skspsl.com. The contact page is there, and I would love to hear from you. I'll see you next week.