SPEAKER_03

What we see is consumers check out when they're reading AI generated content.

SPEAKER_01

What do you think, Max, that the internet looks like in two to three years if we don't do anything about detection or verification or distinguishing synthetic from authentic content?

SPEAKER_03

A couple of years ago, the internet was 20, 30% AI generated, and by mid-2025, it's already 50% AI generated. And I think we're really we're gonna be approaching like 80 or 90%.

SPEAKER_01

So where is this line between smart assistance, let's say, and outsourcing your thinking too much?

SPEAKER_03

So I actually think the best work to outsource the AI is the work that you understand deeply and you're very knowledgeable about because then you can tell when AI is bullshitting you.

SPEAKER_01

So with this manipulation, this flood of slot that's happening, where is it showing up first?

SPEAKER_03

I I think the biggest one I'm seeing is SEO. There are some companies that let you fully automate your blog writing process with AI. At first glance, it's like, oh, you know, I can put out as many blog posts as I want. And then what we see in reality is once you do that, you start ranking really well, you start getting a whole bunch of page views, and then at some point Google notices and it drops to zero.

SPEAKER_00

Max is the co-founder and CEO of Panground, an AI detection startup fighting to protect human authenticity in a world flooded with AI slot. A Stanford-trained machine learning expert from Google and Neuro, he's helping people tell what's real from what's machine made.

SPEAKER_01

Welcome to Using AI at work. I'm your host, Chris Dag. Each week we'll be learning how today's business owners, entrepreneurs, and ambitious professionals are getting more done with smart use of tomorrow's tech. Let's get started.

SPEAKER_00

Right now, every business leader is asking the same question. What are we going to do about AI? If this is you, ChiefAIOfficer.com has the answer. We give you a simple path forward where we provide executive and team training so your people know exactly how to safely use generative AI in their day-to-day.

SPEAKER_01

We also manage the deployment and implementation to make sure tools actually get adopted and deliver results. And we'll also guide company-wide transformation so AI becomes part of your operating system, not just another shiny object.

SPEAKER_00

The companies that act now will increase productivity, cut costs, and grow faster than their competitors. Those that wait will get left behind. So if you want to make AI work in your business, visit chiefaiofficer.com and see how we're helping companies of all sizes finally get results from AI.

SPEAKER_01

Hi, everybody, and welcome back to another episode of Using AI at Work. My name is Chris Dagel, and I am the host of the show. Our guest today is Max Sparrow. Max is the co-founder of Pangram, which is an AI text detection company. They're based in Brooklyn. And a growing team. At the time that we last spoke in February, I guess you guys had around 10 people working in person, which is not the standard, I think, in the AI space. There's a lot of remote teams here. But one of the things that Pan Graham is doing is, I guess, one of your claims to fame is that you guys uniquely detected an unreleased creative writing model output that everybody else just assumed was human or authentic output, and uh not according to the Pan Graham algo. So one of the things that I think is important for our listeners today is that leaders are drowning in AI slop, whether it's on LinkedIn, whether it's in what's being sent to you via email from your clients or your vendors or anything like that. And it's important for you to be able to truly judge what's authentic, what's real, what's human output, and what's synthetic, especially to be able to do this at scale. And that's what Max's company does. So, Max, welcome to the show. Um, we'll dig in in a minute, but is there anything that you think that the audience needs to know about why they should listen to this episode?

SPEAKER_03

Oh, thanks so much for having me. I mean, I think this is really important, especially for people who use AI in their daily lives, like I do. I think there there is a lot of there's a wide spectrum of what people believe is acceptable, AI use or not. And I think, you know, I think that's gonna probably be a lot of what we cover is like what are how are we how do we use AI in ways that empower us to uh be better and more effective without just flooding the people around us with AI slop.

SPEAKER_01

So uh I mean, especially now that I talk to executives regularly, and even if they don't even have an AI use policy, even if they've never trained anybody in their organization, the first thing they're saying is, hey, this open claw thing, how can we set that up, right? They all want agents.

SPEAKER_03

They they they heard about open claw somehow from LinkedIn?

SPEAKER_01

Yep. You know, I I think what's happening is that every like business peer group, there's one or two enthusiasts in there for sure that are like way ahead of the pack of everybody else. And you know, executives and CEOs and stuff that they like the shiny stuff for sure. Um, I mean they they appreciate substance as well, but when they see somebody else, you know, saying, Oh, my agent's doing this for me, my agent is writing all my content, my agent is handling all of my you know, my thought leadership posts or sending the uh the updates to our team or whatever, the executive is thinking, ooh, I want that, but they're not necessarily um seeing the output and going, oh, that's the qual kind of quality output I would want to be associated with my brand or my name, right? They're uh they're assuming AI output, AI slop, is something's better than nothing. And I don't know that that's necessarily the position that Pan Gramp takes. What's your what's your thoughts on where these executives are heading with this like enthusiasm for AI generated output, even though it may not necessarily be uh in the long or short term, supporting their brand or supporting their narrative?

SPEAKER_03

Look, yeah, I think it's easy to get shiny object syndrome around AI. It's it's new and it enables things that truly were not possible before. Um, and I think this is yeah, the the like efficiency gains, very exciting. But I do think uh quality, like if if you're letting your bar for quality drop because you're using AI, then I think this could come back to bite you, especially because you're not the only one. I think like everybody out there who's like tons of people are using AI, uh kind of like if you're ever on LinkedIn, you can tell like things like the feed is flooded with AI generated content. And like I I think people are pretty discerning, like like they can tell, and like there's it's it's not like there's a shortage of things for me to read. Like, like I'm all yeah, I'm already like picking and choosing like the the best of the best to to read and or or summarize. And so if there's you know twice as much content, but it's half as good, then like, or 70% is good. Like, like I don't care. I'm just I'm not reading anything but like the highest quality content already.

SPEAKER_01

So we're gonna talk about like some things that people should be looking out for for sure. But I think uh listeners, yes, you can produce more content, but so can the bad actors, right? AI has become this force multiplier for those folks too. So the the uh flood of synthetic content and these autonomous agents that can manipulate platforms at scale is something that you as an executive, you need to be concerned about this, not just so that you're not wasting your time reading you know AI generated content, but you could face brand or reputation risk and definitely operational noise like fake reviews, fake content, fake engagement that is going to distort the decisions that you could be making based on the data points that you're receiving from web traffic or responses to certain marketing pieces or whatever those things are. So um what do you think, Max, that the internet looks like in two to three years if we don't do anything about detection or verification or or uh distinguishing synthetic from authentic content?

SPEAKER_03

Oh yeah, we've we've seen the trends. Uh it's gone from uh a couple of years ago, the internet was 20, 30% AI generated, and by mid-2025, it's already 50% AI generated. And I think we're really we're gonna be approaching like 80 or 90 percent. And the the really amazing thing, I I guess not amazing, but the the the thing that's like not that great about this is that I I think what we see is consumers check out when they're reading AI generated content. Uh, we we ran a study, um, and people just once they once they clock something that they read is AI generated, they start just carrying less, they they go to platforms less with AI generated content, and they have like this like pretty like high distrust around like misinformation and hallucinations in content that they can tell is AI generated.

SPEAKER_01

So, okay, in that case, what would be the the business cost of um certainly you as a as a business owner or something like that, not knowing what's real? Um, but I guess you kind of touched on that for the consumer or for your your your clients. Um, but what what's the cost for you as a business owner, not you specifically, but a business owner or somebody who's a decision maker in a business, for them to not necessarily have this discernment to be able to tell what's real and what's not?

SPEAKER_03

So I think where I'm seeing this is these like untrusted layers of communication where like you don't necessarily know that the person on the other side is a good actor. So like one example of this is like RFPs. Like I see people all the time, like in government elsewhere, they're putting out like a a a request for proposal, like RFP. Like, you know, I want to hear from you know several different businesses, but what we're seeing now is like just more and more just like fake AI generated RFPs, like like there's no actual substance behind it. There's not even often a real business or individual behind it. It they're just like there's people who are trying to like completely fake it till they make it, like fill out fake RFPs, and then you know, if once they get a meeting, then then they're scrambling to put something together. But but it's just a waste of everybody's time. And and and this is just one example. I I see this, I get AI generated uh emails in my inbox. This costs me time. Yeah, that's like very quantifiable in terms of the cost of the business. Uh and it it's starting to just be like a flood.

SPEAKER_01

You know, I'm noticing it, uh I don't spend a lot of time on content and thought leadership. We're too busy doing the thing, I guess.

SPEAKER_03

That's good.

SPEAKER_01

But in in LinkedIn in particular, man, I like you know, I was encouraged by some, I was like, all of a sudden some of my posts are getting responsive. I'm like, oh, this is great, it's good to see. I'm I'm I'm engaging with the audience. But it turns out, I mean, I've got a I kind of talk to to business owners, it's almost like uh a wine drinker. If I put if I'm if I don't have a palate, if I haven't spent the time like really um understanding the nuance of wine, if somebody puts a glass of red wine in front of me, it's just red wine. Compared to the individual who spent, you know, uh went through the Windows of the World Somalier course or whatever, like they can tell you the terroir, where it was grown, what vintage, what grape, all that stuff. Same thing with AI. If you're if you're not paying attention, it's just all red wine. You don't really have a palette, right? You can't discern that that synthetic from something that I actually wrote. Now, I've got a palette that's developed, and what I'm seeing in platforms like LinkedIn, I'm not on Instagram or Facebook or anything like that, is that that um that cadence of your reply, the the words you chose in your reply, that seems a little I don't trust it. And that's what I'm seeing. I don't I don't know, is anybody even putting out real content on these platforms anymore? Or or do you have data points that say like how much of what I'm reading on something like LinkedIn actually came from the mind of the person that's posting it?

SPEAKER_03

So I guess two things to pull out here. So one is yes, there have been papers that show that people who use AI regularly, they're pretty robust at detecting AI themselves. Yeah. They they can tell. Uh, and then when we run studies, we've seen it's about 30 to 50 percent of people can tell. Uh about 30% of like the older generations can tell. And by Gen Z, it's it's 50 to 60 percent can tell. Um with that said, how much of the internet uh like place like LinkedIn is AI generated? Um, I actually don't have a number for you on on LinkedIn, but I have ballpark it. Ooh.

SPEAKER_01

Just a basic on your understanding of like the 30 to 40 percent.

SPEAKER_03

I I think I don't think it's the majority yet, but I think it's it has grown pretty rapidly. And I actually think that like LinkedIn is starting to realize this and backpedaling a little bit. It used to be I I try to compose a post on LinkedIn, I write 10 words, and there's a little like right with AI button that lights up. It's like, yo, well, you don't have to finish writing your don't finish your thought. AI can finish it for you. Yeah. Uh and now they actually removed that. Like, I don't see it anymore. And I think it's because they realized like it was hurting the quality of content on their platform.

SPEAKER_01

Very good. I I hope that's the case.

SPEAKER_03

No, yeah, yeah. The the replies are are actually the worst for me. The replies when I write a LinkedIn post and then I read it sounds like somebody's engaging with my content. They're like, oh, that's really interesting. They mention uh something from my actual post, and then they ask a follow-up question, and then I like squint for a second, and I'm like, oh, this is just like AI reply slop. Like I they're they're literal time wasters, they're wasting everybody's time by uh having AI auto-reply to every post that they see.

SPEAKER_01

So for the listeners, if you're on these platforms, especially LinkedIn, please be aware that this is happening, right? Like, and don't be don't be one of those people.

SPEAKER_03

So yeah, hey, don't be one of those people. Like, yeah, the the number of people I've talked to that said, I want to get my open claw to go like run my LinkedIn or Twitter account for me. I'm just the I I understand it's it's exciting technology, but it's not great. Uh, and then I can also tell what what we're doing to try and combat this and like help you like not spend time on your time waster things. So, so Pangram, we're a platform to detect AI-generated content. You can go to Pangram.com, paste in a piece of text, and we'll tell you if it's AI or not. We have a one in 10,000 false positive rate, so it's pretty low. So we're if we say something's AI, we're pretty sure that it is. Uh, but obviously, like you're not going on LinkedIn and like copy pasting everything you see into another website. That's absurd. So what we did instead, we put out a Chrome extension that as you scroll on LinkedIn, Substack, Twitter, Reddit, it will automatically tag the uh comments and posts and replies that you see as AI if they're AI. And so you see that little badge next to reply, and I'm just like, okay, I'm I'm just not gonna engage. It's it's not worth my time.

SPEAKER_01

I'm on the extensions, the Chrome Web Store, and I'm adding it to comment now. I didn't realize you had a um a download. So um for my team, let's add in the show notes for sure a link to this um in the Chrome store. This is huge, I mean that's a lot. Like I would love to have it as a copilot rather than copy and paste it. So it's good to know.

SPEAKER_03

That's awesome. I yeah. It's been supremely supremely useful for the team. And we just put it out there uh on the at on the Chrome Extension store last month. So nice. I hope people find it useful.

SPEAKER_01

I'll let you know. I'll be using it, leaving a review for sure. Um yeah. So with this with this manipulation, this this uh flood of slop that's happening, where where is it showing up first? Is it in reviews? Is it in, you know, obviously we talked about social. Is it in customer support? Is it like where are you seeing that it's showing up?

SPEAKER_03

I I think the biggest one I'm seeing is SEO. So so people want their website search engine optimized. And so to do that, you want blog posts that are relevant to the topic and keywords that your target users are searching for. So what people did in the past, they'd hire blog writers, uh, they'll they'll get some some blogs up, and then hopefully some of the some of these start ranking for the keywords they want. But then there start coming around these some companies like AirOps, if you've heard of it. Uh, there's some companies that let you fully automate your blog writing process with AI. Like sounds like a dream in like it at first glance, where it's like, oh, you know, I can put out as many blog posts as I want. And then what we see in reality is, you know, once you do that, you start ranking really well, you start getting a whole bunch of page views. And then at some point, Google notices and it drops to zero. So I saw this on Twitter uh just the other day. Click up, they were have like you see this exact trend with their search results, where they their their search traffic was going up and up and up and up, and then it dropped off a cliff. And then their like head of SEO was on Twitter commenting, yeah, we just like we relied too hard on AirOps, we we we flew too close to the sun and and we got burned.

SPEAKER_01

Now, one of the things that we talked about um on our kind of our pre-call was this idea about uh the impact of AI in education, this concept of skill formation, right? That um, and one of the takeaways from me was that you were talking about how AI can offload the cognitive work that students need to develop that kind of widens this gap between motivated and self-driven learners and those who are just looking to get through and get the class, right? Absolutely so where is this line between smart assistance, let's say, and outsourcing your thinking too much?

SPEAKER_03

Um, I mean, I think this line, I I don't think it's just for students. I I think it's actually for everybody. Like there's oftentimes like skills that I'm trying to learn, and I would much rather like learn the skill myself and put in some reps so I like fully understand yeah, like exactly what I'm doing. And then I start to outsource some of this work to AI. So I actually think the best work to outsource to AI is the work that you understand deeply and you're very knowledgeable about, because then you can tell when AI is bullshitting you. If if you're if you're going into something and being like, I have no idea where to even start, I think AI is also great for like getting you up to kind of average, but you're never gonna like know if if there's a hallucination, if it's it, if it's not doing it as well as an expert would. Like you just have like no no sense of that. So I think I think it's really important to, in my opinion, what we're doing, we continue to work with experts. I'm always trying to hire experts in different fields, like SEO experts and like design experts, because what they the people I hire, I hire them because they can do what they do way better than Cho Claude or Chat GPT can. So, so yeah, uh, I still think there's a ton of value in having experts and then uh letting them use AI to make themselves more productive rather than like me as the CEO being like, we're getting rid of all designers and we're replacing them all with Chat GPT. That I think that's yeah, that's just a disaster waiting to happen.

SPEAKER_01

No, I like this perspective a lot that encourage them to use AI on the things that they're already good at. Because the human in the loop will be able to detect, you know, slop or hallucination or whatever, right? You know, I saw something, it was probably on X or whatever, but it was indicating uh some measurement of uh is your you know cognitive skill declining from using AI. They said, can you recall the last five requests you made to an LLM? And I was like, man, I don't even remember the last one I did.

SPEAKER_03

So I I don't know if that is that relevant. Do you remember the last five Google results you did? Remember the last five times you you you know you you called your mom? I don't know. It's like I I I think that that's fine. Like, like that's not necessarily a problem. I do think it's a problem if like I am putting together like a yeah, uh a proposal, a document, something, putting it, I'm sending it to somebody else, but I don't fully understand what's in it, and I couldn't tell you what's in it by heart. I think that's where you you get a problem.

SPEAKER_01

So I I don't know, do you have any any tips or how are you making sure that you're staying uh cognitively challenged and you're not relying too much on the models?

SPEAKER_03

Um you do anything in particular? Well, so what I do, my background is in software engineering. So I I actually take I I block off my calendar every Tuesday. I take no meetings and I just use this time to code. Some of the coding is with AI, some of it is without, but the I'm trying to still stay close to the product and uh uh make sure I'm still like really deeply involved. I think you know it's it's great to take a step back and be in manager mode, but I also think staying close to the core product is really, really important. Uh and as well. As like I think grounding in experts, like I mentioned before.

SPEAKER_01

Yeah.

unknown

Yeah.

SPEAKER_01

Yeah. You know, I guess that's that's one of the quotes. I I don't remember who it was, but it was, you know, um a scion of industry. He was being interviewed, and the interviewer um was kind of taking a a swipe at the the person, it wasn't Thomas Edison, it was somebody of that stature, talking about how um, you know, how how can you lead this company when you don't even understand the you know the dynamics of the product that you're making or the engineering behind it? Exactly. And he said, He said, Oh, all I have to do is pick up a phone. This this line rings the expert that you know, the PhD on that. This one. So it was almost like the same concept, but just with actual humans as compared to what we are able to do almost with with models. But um, so Pangram certainly I like you said, copying and pasting into something and to determine if this is like am I reading synthetic or authentic, or is it predicted to be authentic information? Not viable. Using the um plug-in certainly makes a lot more sense. It's on demand, it's right there with me as I'm going through stuff. But where else in like maybe a more sophisticated stack would Pangram fit? And for what to what end for uh different industry, I guess.

SPEAKER_03

Yeah, so we have a pretty cool stack in recruiting. I think this is the one that I'm most proud of. Uh so if anybody uses Ashby, we've we have this integration. Uh and the reason I think it's valuable, again, like it's again with the bad actors. There's a million uh there's these websites that will auto-apply for you. They'll they'll mass apply everywhere. Um and you know, why would I hand apply to 10 jobs myself when I could use this auto applier that applies to 10 or a thousand jobs? And then, you know, whatever interviews I get, then I can decide if I like really even want to work there in the first place. Uh so the the problem with that though is I think there's a lot of candidates that are actually like low interest or low signal. So what I've been seeing people do is they just add a little text box at the end so like upload your resume, you know, give your information, but also just like, hey, please write a few sentences about why you're interested in this company, like don't use AI. And if they use AI, uh when the instructions say don't use AI, you can just actually automatically disqualify them, which I think is really great because then you're only getting the high high-intent candidates who actually wanted to work there.

SPEAKER_01

Yeah. How did it come about that that that was one of the areas in you know the spectrum of business that Pangram started getting real traction?

SPEAKER_03

I I think this is just like something that customers had been asking for for a while. Like we were seeing recruiting teams that were actually just like buying Pangram subscriptions and doing the copy paste themselves. And they're like, hey, can we like can I instead like upload a CSV and have you like uh or yeah, can I upload a spreadsheet and then have Pangram give you give me a result for every candidate? And I'm like, oh well, we could just like work inside your um inside Ashby instead. We'll we'll work directly with you on this. And so that's how this came to be. Uh I we work pretty closely with a lot of our early customers still. And I think this is this is really like this product shape has hit out of the ballpark. Um I we're still, I think, looking for more like this. I think something that I really want to do, but just haven't gotten around to doing yet, is building this uh email integration, which is the same thing. I just want AI generated emails to go to a different folder. I'm not gonna delete them, not sending them to spam, just putting them in a different folder so I could I can take my time and and see if they're relevant.

SPEAKER_01

You know, that's like my uh like a lot of others, my inbox is scary, right? And I would assume that a lot of it is certainly augmented, if not outright, synthetic content.

SPEAKER_03

Totally. Especially there, there's these cold outreach pipelines that of course it just uses an LLM. Right, write a quick paragraph about like why product X is relevant to person Y.

SPEAKER_01

Yeah. And for those uh of you listening who have a business account, like I don't think that you can't get in trouble for spamming business accounts. Not quote unquote, not like I know that there are different rules for me as a business sending to a B2C, a consumer, as compared to me as a business sending to another business.

SPEAKER_03

I think right.

SPEAKER_01

So I can I can be a lot more aggressive, or I don't have to worry, I can just get a list, scrape a list from Apollo, and send to everybody on there and not worry about consequence other than you know brand damage or something like that. So if you're a business, like please understand that uh as this this AI slop um increases, you are gonna be seeing a lot more of those, what Max was talking about, those cold email outreaches that are what seems human, but again, if you you have that palette, you can be like, nope, this is this was AI generated. Um so what should a full trust stack look like for a business if I want to make sure that I mean hell, I'm in the AI space, right? And I want to make sure that um our clients understand that we're behaving in an authentic manner, that we're not just outsourcing everything to AI, which I guess would kind of be eating the dog food, but not necessarily the the brand experience I would want to do. Yeah, yeah. I don't know.

SPEAKER_03

It's uh it's like if if if outsourcing to AI is like even cheaper than outsourcing to the Philippines, then like I think that's sort of what it's gonna indicate to customers.

SPEAKER_01

Yeah. Um well what would a full trust tech stack look like for a business then?

SPEAKER_03

Yeah, so I think the getting getting the Chrome extension for the the socials and just like the general internet is really good. Getting the um the recruiting stack, like that, that's what we use ourselves. Um I guess hit me up if you're interested in the email thing. I'm all I'll probably yeah, I'll probably code it up using AI uh some weekend and then put it out there. Uh and and then beyond that, I think we just see uh I see some people who use the Pangram API to automatically check some things that come in. So like publishers, anybody who has like submission cues, I think that's a great way to kind of just like validate that what you're getting inbound is a real person and you should you want to take your time and treat them as such.

SPEAKER_01

So uh what are your thoughts on like when would detection be enough versus maybe more robust approach such as watermarking or identity verification or internal policy on usage, stuff like that?

SPEAKER_03

I I love watermarking. I think watermarking is one of the most interesting technologies of this time. I think it's really cool how you could look at an image and it looks just like real life, but then it has this invisible Gemini watermark and it comes back as AI. Super cool. One of the problems is watermarking on text is a lot less robust just because of how text is often a lot shorter. There's a lot less there, there's no like pixels to hide this uh this invisible watermark in the same way there is for images.

SPEAKER_01

Yeah, yeah.

SPEAKER_03

Uh the other issue with watermarks is it's an opt-in system. So open so Google Gemini, like Google images are all watermarked. And I think only recently, like OpenAI said they're going to start watermarking images too. So open AI images also watermarked. Great. Uh, if I want to create some like Iran disinformation, I'm just gonna use a model that's not Google and not OpenAI. I'm gonna use one of the unwatermarked uh models. So I think it's the the trust issue is you can't trust that a bad actor just happened to use a watermarked model because there's so many unwatermarked things out there. Um I think it's really great for uh like giving additional validation and I think it's cool technology, but I think it's not the only thing that you can rely on.

SPEAKER_01

So now let's let's kind of put a little twist on this. What if I as a company want to understand how my knowledge workers are using AI-generated output to for both internal and external communications? How do you recommend companies deploy detection internally?

SPEAKER_03

That's a great question. I've had some people come to me and ask about this. I we don't really have a product solution for this, like just but I think, yeah, I yeah, we we've had a couple companies who have just gotten Pangram for all their employees. And they're they're just they talk about like they they want to fight against work slop. They don't want people sending other people like 20-page documents that are like fully AI generated that they haven't even read themselves. Like they're very like anti-work slop. And so what they do is they instead just like empower all of their employees to have pangram and to to kind of use it liberally on internal docs and make sure, like, you know, it if something's AI, like there's no shame in it, but you should kind of just disclose it as such. Like, like, hey, this was uh like I generated this with Claude, uh, it might have some hallucinations, but like, you know, I double-checked it for hallucinations, so like we should be good. Versus like, this came straight out of Claude, I haven't really looked too deep into it.

SPEAKER_01

Uh, you know, you should approach it a bit differently if that's the case. So this is interesting because I hadn't really thought much about it prior to asking you that question, but uh, this is something that maybe should be in like an AI use policy for a company. Because one of the things we do is we help companies, because a lot of them, they don't have any guardrails. They're just like, go use AI, right? And this would be one that I think by default, uh a leader that was concerned about risk is is concerned about a breach or IP leakage from somebody you know putting something into the models and whatever. But they're not necessarily thinking about a use policy about how do I want internal communication that's generated by AI to be uh you know indicated or how do I like what do we what's our stance on that? And I think that that's something that probably should be a part of of a use policy. Like if you're gonna use AI, um it's not a copy-paste situation. We still want quality output from you.

SPEAKER_03

Yeah, and and I also like I think transparency is so great because it works, it helps in both directions. One, like it encourages people to not just um pretend that AI content is their own. Uh it encourages people to be transparent uh and put out quality work. And then on the other side, it also shows other people like what they can be capable of if they're using AI properly. Being like this if not only if you disclose, hey, I use some AI, but this is how I use AI. This is how I still produce uh a quality document, even though I I leverage AI. Like, yeah, great, do it.

SPEAKER_01

That's something that we need to think about internally at Chief AI officer. So um, I know you guys have some big clients. Quora is a client, right?

SPEAKER_03

Yeah, Quora is a client.

SPEAKER_01

And they're processing obviously large volumes of content with you guys.

SPEAKER_03

Millions and millions, yeah.

SPEAKER_01

So I guess were were they using detection prior to their relationship with you guys?

SPEAKER_03

They they had some stuff internally. Uh it it just like wasn't as accurate. Yeah. They were they were one of our earliest customers, so they they've uh they've got a pretty good deal uh at high volume. But I think we we work with we worked with a couple other social media platforms. Um, we work with some publishers. Uh we obviously work with a whole bunch of schools and universities.

SPEAKER_01

You know, and that kind of we'll get into that education question, but I have I have a question about uh Core's experience. What does a major platform like that see once it turns detection on? Like how does that like how does that change their economics and the way that they operate?

SPEAKER_03

Um I mean, I think it's as always, it's a quality issue, right? If if I suddenly like am able to turn on a ultraviolet light and see like, oh, actually my my users, 10% of what they see on my platform is spam or or is inauthentic, then that's like suddenly that's a huge problem. That's like, you know, we also it's just such a huge win if we say we're gonna take this inauthentic content and just like silently remove it from their feeds. Suddenly everybody's experience gets 10% better, uh, people are happier, and you're they're gonna see higher retention. So I I think that was sort of uh like 10% is not the number, but like that was kind of the process that we went through where we we did a full audit where they ran a month's worth of uh content through Pangram. You know, they had like actual data labelers going and like looking at the content, trying to see like, is does does the Pangram prediction match what we think? Um is this actually like inauthentic content? And and they they went super deep because they are obsessed with quality. That's why they're still one of the websites with the most traffic. I I think they're still like top 25 trafficked um websites in the world. Um so many years after they launched, they they still have a lot of users, and I think it's because they care a lot about quality.

SPEAKER_01

I like that I like to know that they do care that much about quality. That's they'll they'll be in it for the long haul. Um so uh I don't know if it was you and I when we were on the the original uh conversation, but but this an AI agent had tried to contribute code to an open source project and then got rejected because it was humans only. And then are you familiar with this?

SPEAKER_03

I I think I saw that, yeah.

SPEAKER_01

Yeah, and then the agent generated a long blog post takedown of the person that had rejected them. So how do we like and if you think about this, like somebody, uh a malicious actor in your industry, I mean, content warfare could become cheap and scalable if this, you know, this is a small preview, but what happens when that's 10,000 posts?

SPEAKER_03

Exactly. Yeah, yeah, what's stopping them from stopping at one blog post? Let's make 10,000. Let's reply everywhere on the internet where this individual has a presence or or where they're named and you know, bad mouth them. Oh, you could just run that with an agent for a few bucks. And so I think that that's where like I'm really worried about the future of the internet. Like, you know, a lot of people talk about debt internet theory where it's just bots talking to bots. And I think like we've never actually been as close to this being true as we are today.

SPEAKER_01

All right, now let's talk about the education side. A lot of the the people that we work with typically they're executives, they got kids, and they ask me, so what should I what like what should have my kids studying? Or what what is this, what's what jobs should my kids be focused on? And I was just listening to a podcast today about how in 2023, it was like, you're gonna, you're, you'll fail if you use, like, we're gonna give you a failing grade if you use AI, right? And but to me and to you, it's like, do you want to discourage or or plant the seed in these students' minds that AI is bad, you shouldn't use AI? I mean, what happens when they get into the environment of like the arena of commerce, and they're like, oh no, no, no, I don't use AI? I mean, like, well then, oh no, no, we don't use you kind of thing. So what is the appropriate approach that education should be taking so that people aren't doing the full outsourcing of my thinking, copy paste from Claude, um, but they are still able to develop the skills of thinking in AI and being able to use the models effectively to enhance whatever job they're going to be having.

SPEAKER_03

So, my opinion, the the number one most important skill to develop is still critical thinking. Like you have to you have to be intelligent to be a good user of AI tools. If you're if you're not there, then like then you're kind of just like a meat body for like AI to tell you what to do. I I don't think that's like a a great way to interact with AI. So I think number one is ensuring critical thinking skills. And I do think like part of this is you know learning the skills, learning to do it without AI, and then learning how you can use AI to be a force multiplier. So so what I the best way I see um schools and universities uh use this is by having no AI policies like early on in the coursework, and then letting students do, for example, like a really ambitious capstone with AI. And the idea is like if you have AI, if you're using AI, you should be able to take on something that's like three or five times as ambitious as as you would have done without AI. Yeah. It's not about doing the same amount of work, but you're less involved because you're using AI. You have to be able to do more with AI.

SPEAKER_01

Are are you guys being approached by um schools and universities and things like that?

SPEAKER_03

Um yeah, all the time. I I think this is um one of our biggest sectors of business. I think I think there's a really big need for this, especially like look, I think there's a lot of places where it's reasonable to use AI. But if we're talking about like a composition class or like an English literature class, it's like don't don't use AI to just like write your essay for you. Yeah, it's it's so obvious to a lot of people, but I think still this is something that uh educators need tools to help them enforce because otherwise you uh a student choosing between no homework or a lot of homework is gonna choose no homework.

SPEAKER_01

Yeah.

SPEAKER_03

You know, there's there's the the the devil on the shoulder that says, but if they're using it to to creatively game the system that they're yeah, the thing is it's it's like not that creative if you just like copy paste in the assignment and then copy paste the the output into a word doc. Uh yeah, I I think there's there's there's ways to be sorry, I almost closed out of my tab. There's ways to be creative about it, but I think overall it I think the best way to um make sure students learn is to give instructors a lot of control over what their exact AI policy is. And I think it's really great that there's some instructors that say, hey, don't use AI at all for this assignment or this class, and some instructors that are like, here is like we're gonna learn AI skills and like we're gonna learn how to use it. And getting getting a sense of both, I think is really the best of both worlds.

SPEAKER_01

Okay, so uh if I'm listening to this podcast and I run a company of 200 to 2,000 people, what are the first three moves that my my leadership team should make this quarter immediately to protect that that reputation of trust that we have with both our internal stakeholders, our employees, and but also our customers?

SPEAKER_03

Sure. So I think the two things there's I I talked a little bit about this like untrusted um layer of communication. Yeah. So anything that you're getting inbound that you you don't know the source, I think you want to make sure that it's human written or not AI generated. Or yeah, if it is AI generated, you want to know about it. So I think like getting Pangram and using it there is really helpful. On the flip side, we have um things that are going outbound to clients and customers. I think if things are AI generated, we don't want it to look like total slop. So I think it's important to disclose our AI generated outputs and especially when we're we're showing them to customers and people we want to impress. We we want to make sure that yeah, we're we're not sending out slop.

SPEAKER_01

So, you know, I I guess I should have probably uh brought this up a lot earlier in this episode. I've just been assuming that the detection is for text content. Yes. Are you guys capable of detecting image and video?

SPEAKER_03

Uh today just text.

SPEAKER_01

Okay. Is there like is there a whole um industry that is about identifying um deep fakes on video or or images?

SPEAKER_03

Yeah, there there's there's a bunch of cus this Bunch of companies. I think one of the bigger ones is Reality Defender. They they do image and video, uh, and they're they're pretty big on deep fakes. Uh, they can like the their product is more like we're gonna help you make sure you're not like hopping on a zoom call with the CEO, but it's actually just like a deep faked uh actor, and then they tell you to wire money somewhere else and lose a bunch of money. So I think that's the like that's the like deep fake angle that they're helping solve, which I think is also like that that's a very important and like financially motivated problem. Sure.

SPEAKER_01

Now so if if I'm you know all things aren't created equal, if I'm getting a high success rate of uh detection with Pangram, I shouldn't assume that that same high detection rate is going to run with image or video detection. Is that correct? Or is it as good?

SPEAKER_03

I don't think there's anything in image or video that has a one in ten thousand false positive rate like Pangram. Uh I you know, I may be wrong. We've we've done some uh benchmarking, but we haven't seen anything really come close. I think Pangram is you know really, really powerful technology. And uh yeah, yeah, I I think it's great, but I think uh I think it's always worth testing every tool for yourself as well. Like don't even rely on the benchmarks, just you know, run a test yourself.

SPEAKER_01

Good point. Okay, let me ask, where do you think detection in general will be in 12 or 24 months? Like what will it catch well? And what do you think it'll just never catch reliably?

SPEAKER_03

Look, so I mean I think a lot of people said like AI like detecting AI content is impossible in 2023. Sure, they they they were actually right, like like there was nothing good months after Chat GPT came out. And so so I think a lot of people assumed that technology wouldn't get better, which I think is a really interesting assumption to make, given that every technology I know is always getting better.

SPEAKER_01

Yeah.

SPEAKER_03

Um detection's interesting because it is a bit more adversarial. But I think what we've seen is that um we've so far been able to keep up with and even outpace a little bit the rapid development of LLMs. So the future that I'm looking at, I'm seeing that uh major model providers are getting a bit more responsible with how they deploy um, especially image and video. They're all now watermarking their image and video, yeah, except Grok.

unknown

Yeah.

SPEAKER_03

Um there uh and I think the text as well. I I think we're I think this is where Pangram will continue to be relevant, is just as agents scale up and are producing more text content, yes, uh, we are going to need to uh find ways to differentiate between uh these these AI agents and real people. And so what I'm the direction I'm seeing the next couple years go is we're gonna have to keep up with these agents that have a bunch of context and they're synthesizing a bunch of other information.

SPEAKER_01

Yeah.

SPEAKER_03

And we're gonna find a way to not only uh catch these high context agents, but also to kind of do better at differentiating between AI-assisted content and content that's fully like de novo AI generated.

SPEAKER_01

Yeah, yeah. Interesting. Well, you know what? This is certainly every time I do one of these episodes, my my intention is that I expand my worldview of how people are using AI and that sort of thing, and do the same obviously for the listener. This has certainly been that because the last time that we had a guest on to talk about uh text detection, it was one of your competitors based in Canada. Um it's been over a year, right? And you and I and and the listeners at this point hopefully know that a year in AI, like ability. I mean, PanGram was a completely different company a year ago than they are today, as far as what they're able to do, right? So it's good that um, because I hadn't really considered the the um a number of things that you brought up, especially the internal angle. I always looked at it as like, oh, how do I make sure that I don't get that I'm not uh ingesting a bunch of spammy AI slop. But the reality is I I also really need to consider what our internal policy is for internal usage. I want my people, and we're an AI company, man. I want my people using the tools, but I also don't want them doing copy-paste ship. I want them to Yeah, yeah.

SPEAKER_03

You want them to be effective without like inundating everyone around them with slots. Yeah, right. There, there's like a great um line in the sand. And and personally, I think like disclosure and honest disclosure is like a really important part of that.

SPEAKER_01

Yeah. Um, now I gotta figure that out for our use policy, but I'm I'm glad we we covered that. So have fun with that. Yeah, I'll let you know how it goes. Um for the listeners who like like they're like, we don't have a solution for this. We haven't even considered this. We have no filtering, we have no mechanisms in place to make sure that we're not uh wasting our time. Because if you think about it, a competitor could could flood you with a bunch of crap and have your customer service team like all of a sudden your net promoter score goes down because AI generated one star reviews on Yelp and Google.

unknown

Yeah.

SPEAKER_03

Could happen to could happen to you.

SPEAKER_01

So if if the the uh audience wants to know more about how they can protect themselves from it, what you guys do, and that sort of thing. And again, I said we're gonna put a link to that um to the uh Pangram Chrome extension and the show notes, but where else should they go to get some more information about how you guys are protecting them?

SPEAKER_03

Yeah, you can find us online at pangram.com, Twitter at Pangram Labs, LinkedIn, Pangram Labs. Uh we're we're all over, but come find us, get in contact with us. We're really happy to talk with you. We're we're uh a mid-sized team. I know you said we were we were 10 uh in February. We're 17 now as of May. Nice. So you know, growing. Yes. But yeah, we're we're happy to talk with you.

SPEAKER_01

Very cool. Yeah, I I and if you're a listener, I wouldn't encourage you to be oh, this we'll get around to it. This is something that if you're not prepared for it, it's only going to like start paying more attention to your LinkedIn uh thread.

SPEAKER_03

And you'll I think you'll start noticing like this is once you start noticing, you can't stop noticing.

SPEAKER_01

Yeah, yeah. So I need to go back through my stuff and see who's who's real and who's not. So well, Max, thank you so much for taking the time and sharing um obviously your expertise on the subject. I've I've learned a lot and I've got a lot to think about on again how we're gonna do it and how we're gonna advise our clients to to prepare for this future. So um very interesting. Awesome. Yeah, thanks, man. And for all the listeners, uh, if you got something out of this episode or any of our episodes, and you've got peers or teammates or coworkers that you think um they're not quite as far along on the journey as they need to be, we'd love it if you would share uh what we're doing here at Using AI at work with them so that they can hear more interesting guests like Max and all of our guests and start to get a better perspective of how this is going to apply to them using AI at work. So thanks everybody. We'll see you next week for another amazing episode of the show. Thanks for tuning in to Using AI at Work. Don't forget to subscribe for more conversations about how to use AI at work. And a special thank you to our sponsor, Chief AI Officer, for empowering businesses with AI education and training. Visit their website for free AI readiness assessment and AI strategy guide to help you get started using AI at work. That's www.chiefaiofficer.com. Follow us on Twitter at the handle usingAI at work.

SPEAKER_00

And visit www.usingai at work.com for free resources to help you harness AI in your role.