DISCARDED: Tales From the Threat Research Trenches
DISCARDED: Tales from the Threat Research Trenches is a podcast for security practitioners, intelligence analysts, and threat hunters looking to learn more about the threat behaviors and attack patterns. Each episode you’ll hear real world insights from our researchers about the latest trends in malware, threat actors, TTPs, and more.
Welcome to DISCARDED
DISCARDED: Tales From the Threat Research Trenches
"Always Intentional": A CISO's Pragmatic Take on the Agentic Era
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
What does it actually look like to bring AI into a threat intelligence program at one of the internet's most iconic companies?
Hello to all our Cyber Pals!
Host Selena Larson is joined by guest host, Sarah Sabotka as they chat with Sean Zadig, Chief Information Security Officer (and "Chief Paranoid") at Yahoo, for a candid conversation about the evolving intersection of AI and cybersecurity.
Sean shares how Yahoo's security team, the Paranoids, is navigating the agentic AI transformation: from running a company-wide "skill-a-thon" to get every team member building Claude-powered tools, to rethinking legacy infrastructure from the ground up. He also opens up about what keeps him up at night; including the looming threat of AI-powered exploit frameworks like Mythos, the growing signal-to-noise problem in threat intel feeds, and the very real risk of analyst burnout as the pace of the industry accelerates.
But Sean's outlook is surprisingly optimistic. He argues that defenders have a home-field advantage, that the best code ever written is just 12–18 months away, and that the goal of AI in security shouldn't be doing more with fewer people–it should be building more resilient teams.
For more information about Proofpoint, check out our website.
Subscribe & Follow:
Stay ahead of emerging threats, and subscribe! Happy hunting!
You're listening to Discarded: Tales from the Threat Research Trenches, a podcast by Proofoint for security practitioners. Each episode you'll hear from security researchers, malware analysts, threat hunters, and more as we dive into what's going on in the world of cyber attacks and how defenders safeguard us from threats. Let's get into the show.
SPEAKER_03Hello to all our cyber paddles, and welcome to the Proof Point Discarded podcast. I'm your host, Selena Larson, on the threat research team here with my colleague, Sarah Sabotka, as the co-host today. And Sarah, who is our very special guest?
SPEAKER_02Well, we are joined by Sean Zadig, the Chief Information Security Officer at Yahoo, and better known as Chief Paranoid. Sean, would you like to introduce yourself?
SPEAKER_00Awesome. Thanks for having me. Yes, so like you said, Sean Zadig, I'm the Chief Paranoid and uh CISO here at Yahoo. I've been here for 12 years, which is kind of crazy. It's a long time. And I've actually been the CISO for six years. So yeah, really excited to be here.
SPEAKER_03I am really excited to have you here. One of the great things about Yahoo and the Paranoids is that they are a great partner, collaborator of us here at Propoint and the Intel team, especially our espionage crew. I know you guys do a lot of really interesting work, and we have collaborated on blogs. We will, of course, share a link to that in the show notes. Um, one of them was one that I worked on 10 Things I Hate About Attribution, rom-com versus transfer loader, which is super fun. But Sean, I don't know if you want to say anything about the paranoids about your amazing threat research and Intel team.
SPEAKER_00Yeah, totally. So I think I don't know, threat research is one of those things that I kind of have a passion for. So my background actually was in law enforcement. And I used to do cybercrime investigations and a lot of focus on attribution when I was with the federal government. I worked for NASA as a special agent for the inspector general. And it was a lot of fun chasing, you know, sort of actors and adversaries all over the world. Wow. But really like to focus on like who's behind, you know, the keyboard. And is it it's not just ones and zeros, there's somebody there who is, you know, kind of directing those and they have motivations and they have, you know, desires and often taskings. And it's important to understand that. And so when I got to Yahoo, a few years after I got here, really started to organize uh what now is our advanced cyber threats team. And that's the group that does the espionage and other related areas like disinformation and such.
SPEAKER_03Before we dive into the questions, I do know that we are going to be talking about LLMs, AI, all that good stuff. But I have to ask, where did the paranoids name come from?
SPEAKER_00Good question. It definitely predates me. It's been, I think it was like 1999, maybe 2000, which incidentally is the age of my Yahoo email address, 1999. But we uh we're like, I think definitely one of the only and definitely the oldest branded security team in industry. And so as far as like where it came from and what the origin is, I don't actually know. I do know that I like to say the I guess my spin on it is that we are paranoid, so our users don't have to be. So we have like hired a bunch of folks who think about the adversaries and the the risks and the threats, but we want our users to be having kind of a carefree and you know, well-informed but also enjoyable experience on our platforms and not having to think about spam and scams and all that bad stuff.
SPEAKER_03For sure. Well, and Yahoo is enormous. The amount of customers that you guys have, the number of people that you are protecting on a daily basis, the number of fantasy sports users. Some of us uh use multiple uh fantasy sports uh apps for every single sport. So yeah, you guys have you guys do a lot. And I think that you're I think that Yahoo is kind of one of those companies that that people it sort of flies under the radar because you just like it's one of those companies that people are just using, right? Like I'm just I'm just emailing people. I'm just I'm just using Yahoo, you know, like is this is just my email, but you have such a broad coverage area, so many, so many customers of it, so many people that are using Yahoo every day.
SPEAKER_00Yeah, and that is exactly the thing that brought me here was because I was so so before between NASA and Yahoo, I was also at Google, also doing cybercrime investigations. And when I was asked to come to Yahoo, I actually jumped at the chance because it's such an iconic brand and name. And I also love the color purple, which helps. But yeah, I mean, like it's you know, the numbers are kind of staggering. It's it's one in three Americans use Yahoo Mail, and we've got hundreds of millions of users all over the world. And as we think about protecting people in their, you know, digital lives and in their physical lives, like it's really harder to, it's hard to imagine having more scale and opportunity to have impact than somewhere like here. And I I also we have a really supportive leadership team who really values a lot of the work that we do, whether it's espionage, disrupting scammers, protecting children, or you know, making sure users are just sort of safe online. Like that's something that's really part of the core, you know, corporate mission here, which I really love.
SPEAKER_03Amazing. And a key part of that, obviously, is the security team and intelligence. Cannot do security without actioning intelligence, right? And so I think that, you know, when when uh we reached out, one of the things that you wanted to talk about was CTI and artificial intelligence and how CTI is changing a lot. So there's been, of course, over the last year, especially, but over the last few years, this emergence of LLMs and trying to figure out how are we incorporating some of these toolings into our security processes? And one of the things I think is is interesting, and there's a lot of different takes on this. I'm curious about your take, but like how are security organizations actually leveraging LLMs and AI tooling in their intelligence reporting and also how they're actioning it, like the whole the intel, the whole intelligence lifecycle and process. Like, is this something that you guys have have already achieved? Is this something that you're working on? Like, and how are you seeing it with like maybe some of your customers or partners that are the consumers of that intelligence?
SPEAKER_00Yeah, I would say I mean, I don't know if you can ever achieve like full, you know, intelligence awareness or whatever, but I would definitely say we are in progress with making our call it agentic transformation. I mean, I guess looking back, like I've sort of been around for some of the birth of what we now consider to be threat intelligence. And so, you know, I was on some really early mailing lists where people were passing around, you know, malware hashes and IP addresses and other IOCs and kind of witnessed the birth of this, you know, multi-billion dollar industry now that is has been built around IOCs and in feeds and things like that. And that's all almost commodity. And so I guess now we we have LLMs and uh, you know, a gentic AI, and we can, you know, use that at sort of raw data and build things that are meaningful and you know really can help our for us help our users and be good sort of part participants in the community and you know share a lot of information back. By the way, one thing that I really am really proud about with the paranoids is we sort of take a like, you know, the rising tide lifts all the boats philosophy. And we don't like to hoard data and we like to like to share data on actors, on TTPs, making sure that the things that we see can then be, you know, absorbed and everybody kind of has a mutual defense approach. Okay. But okay, so when it comes to like AI use in Thread Intel, I would say we're we're definitely kind of mid-journey. And all this stuff is so new. I mean, it's it's really, you know, when when it comes to like actual like, you know, agents and skills and what we can do, the difference in the last three months has been like astounding.
SPEAKER_01Yeah.
SPEAKER_00So, you know, a few months ago, I think my team was likely using it to do things like add context and you know, do enrichments and you know, maybe give the analysts more information. But as we start thinking about, I mean, I apologize for using the word, but like mythos and things like it, I I I guess I use mythos almost as a holding like statement for just that sort of end-to-end vulnerability to exploit lifecycle that AI can do. And, you know, honestly, like the NSA and the Chinese probably have been already able to do, but now we're starting to see work its way more into commodity um models. So I think now we need to like as defenders really need to kick it up a notch. And, you know, just enrichments for our human analysts isn't enough. And so we need to be doing like really autonomous decisioning and moving at, you know, again, using the buzzword machine speed when our adversaries are doing the same. So, you know, it's really like how can we like make the sort of detection to action through mediation lifecycle a lot faster and make it almost like a closed loop? That's I think what we're trying to go for. But and again, like I'm almost more of a qualitative than a quantitative person. And so I think there's a ton of room for human ingenuity and creativity. And frankly, like our goal is to disrupt actors and adversaries. And so, like, really understanding who that actor is and what motivates them and how to stop that individual actor, that's something that might be really hard for AI to do. For sure.
SPEAKER_02You brought up mythos already. And I knew it would only take a couple of minutes to get to that topic. And I in the paranoids, I mean, Yahoo's been kind of a an OG trailblazer on the vulnerability intelligence and research front for a couple decades now. So I I can imagine there's some ideas or thoughts, or maybe even some feelings that you and your team have about mythos and how it could impact the landscape. And I'm just curious how how you are approaching that or how you're potentially preparing for it or what your thoughts are.
SPEAKER_00Yeah, so actually last week I was at an executive offsite where I talked like with a lot of our top executives at Yahoo, and I talked a lot about mythos, but really more like sort of AI, cybersecurity in the AI world. And then tomorrow I'm actually doing a company all hands where I'll be talking in front of the whole company about a similar topic. I mean, basically the same topic. And so as we think about how to best prepare, it's I mean, I guess even zooming out a little more like am I a pessimist or an optimist when it comes to AI? I think I'm an optimist. And it's weird to hear a CISO say that perhaps, but I do feel like we have an opportunity as defenders. We have the home field advantage, and we have an opportunity to use AI to like really not just like modernize, but like rewrite our applications for safety. And to, for example, like, you know, everybody's always struggling with, say, end of life or approaching end of life, like libraries and dependencies. Like, we can we can totally rally around those things and you know, remove end-of-life dependencies. We can do things like use memory-safe languages exclusively. We can use AI to like help us eliminate a ton of tech debt. Because I really do feel like in the next, I don't know, 12 to 18 months, the models will be good enough to write the most secure code that the world has ever seen. And I'm really like excited about that. However, if we're 18 months away, you know, 12 to 18 months away from that, and we're six months away from like the commodity models getting mythos like capabilities, we're gonna have a really rough, you know, year or so in between. And so I'm kind of trying to prepare the company for there could be a lot of third-party. I mean, for us, a lot of our security sort of patching exercises or not incidents, but like sort of rallying points are often around third-party code, supply chain vulnerabilities and things like that. And so we should be expecting that to like really ramp up as bugs just get discovered, you know, in incredible volumes in the things that we all depend on as you know, internet companies, not just Yahoo. So kind of preparing people for that, trying to plant the seeds of as we identify our products and our repos, like let's also let's rebuild it, let's rewrite it so it's you know, the amount of tech debt is basically zero. And it'll make it like not just more secure and safe, but it'll also make it cheaper and more efficient and scalable. And like there's a whole host of benefits that come from that. And we're also just then reducing attack surface as we go. So that's some of the ways.
SPEAKER_03Yeah, that that's super interesting. And I'm curious, are you more concerned about new zero days being identified using agentich tooling or the increase and exploitability of existing vulnerabilities based off of agentich tooling?
SPEAKER_00To use one of my favorite gifts, why not both? I mean, I I honestly think we're gonna see both. It's hard to pick, you know, a favorite there. Yeah, I think I'm definitely concerned about the new zero days because there was that story, again, mythos, and you know, whether or not it's hype. But the story of the was it free or open BSD, you know, 27-year-old vulnerability that had been looked at like that code had been analyzed thousands of times by you know automated fuzzers and human eyes. And that was ended up being like a critical vulnerability that was then exploited and weaponized by by mythos really quickly. So again, whether or not you believe that hype, like that sort of thing is gonna be possible. And so, yeah, like zero days. Now, the caveat, of course, is that mostly it's gonna require the adversary or the researcher to find to have the code. And so, you know, if we're looking at like third-party code, whether it's open source or you know, closed source or it's stuff that we maybe have built at Yahoo, can the actor get that code to then do the review? That's you know, it's a little bit of security through obscurity, uh, not a ton. But when it comes to open source libraries that we're all depending on, that that I'm very concerned there. But one thing that, you know, these new end-to-end exploit frameworks can do is also chain together, you know, low and medium severity vulnerabilities and then generate like an end-to-end, you know, critical exploit. And that is something that a lot of security programs have a have a hard time addressing, is you know, they're really good at the highs and the criticals, but when it comes to the lows and the mediums, they're like, yeah, you know, maybe we'll get to that. And I think as an industry, we need to start prioritizing those two.
SPEAKER_03Yeah, I think that's actually a really great point because it's like the the chaining factor, right? Because if it is just like a low severity vulnerability, but if it's chained with two other low severity vulnerabilities or a medium or a high, then it's like, oh, well, well, this might not be so low anymore once it's actually exploitable. And I I do want to get back to talking about like Intel and AI, but one more question actually about vulnerabilities, because I think it kind of goes in a little bit into reporting. But if we are now having to have higher prioritization for vulnerability management, things like lower severity vulnerabilities, and you're you're looking at reporting that's coming, some of it might be AI generated, how are you addressing that prioritization? And I think in also in in terms of like the people that you're that you're working with, like how are we, how are we as organizations trying to make it so people don't burn themselves out, trying to respond to a lot of these changes while simultaneously doing probably more work because of the amount of vulnerabilities or exploits or flaws that are discovered in our new sort of agentic reality. Like, how do you find that balance while also maintaining security of your organization?
SPEAKER_00Yeah, that is something that is on my mind a lot. I think it's kind of ironic, right? Like we're like, oh, AI is gonna save us so much time and now we're all busier than we've ever been. Yes. Yeah, I certainly feel that way. Um I'm like, I'm always exhausted. I do feel at the same time like I'm invigorated. Like I'm really, I think this is such an exciting time, but like I'm man, when my head hits the pillow, I'm out. So yeah, how do we then, you know, make sure our teams are resilient and are equipped to handle the pace? I think, you know, as much as possible. One one thing that is good about AI and and looking internally at your own organization is, you know, you should be thinking, you as in like the CISO or a security leader, what is the most uh sort of toil that you can reduce? And so if there's work that you can do that will make your employees' lives better, like you should prioritize that. Um, because you know, that's that resource of you know, uh an employee who you've trained and who's got that domain knowledge and expertise, like that is not something that is easy to replaceable. And uh and also I think, you know, in kind of a related note, a lot of industry folks, not just security, but sort of across industry, uh, you know, with when you have uh automation, you can do one of two things. You can do more with those same people, or you can do the same amount with less people. And uh, you know, I think a lot if we see a lot of the news happening across industry, a lot of people, a lot of companies are choosing to do the same amount with less people. But that likely results in burnout and in, you know, in in exhausting the folks who are left. And so I like to think that, you know, the paranoids are gonna take a different approach and are gonna be looking at how do we build a resilient organization, one that is, you know, equipped to handle challenges that that come. And, you know, we're talking about mythos now, but like the pace of innovation continues to increase. And like, what are we gonna see two months from now? It could be something totally different, some insane word that we've not thought of yet. So I don't know. I I I don't want to make decisions for the future based on what we know now and then remove our capacity to react in the future. So I do want to focus on like how can we then make our employees' lives better? And that includes things like making sure they take time off. And one of the things I like to do is like I like to harass paranoids who haven't taken enough time off and be like, hey, like what are you doing? Because it really is uh a marathon and and not a sprint. And like that's the way we need to be thinking of it.
SPEAKER_03Absolutely. I love that mindset. Oh my goodness. I wish a lot more people in this industry had the same uh mindset as you do, Sean. That's great. Our manager also bothers us to take vacation. And I did finally take a couple of days off and I went to New Orleans just this last weekend.
SPEAKER_00So Oh, I was there last week. Exactly. Yeah.
SPEAKER_03Oh my gosh, how fun. Did you get completely rained on too? Or didn't we? No, I missed that. I missed that. Excellent. I uh it was fun doing a second line in a thunderstorm, I have to say.
SPEAKER_00Oh, yeah, we did we did a second line as well, but it was dry. Yeah.
SPEAKER_02Excellent. Party time. I will say it's very, it feels uh very validating to hear somebody in your position, Sean, to say, I feel overwhelmed, but like kind of excited. Because I think that's been something that Celine and I have definitely expressed to each other, like and across our team for the last few weeks. It feels like the landscape is ramping up, but also the kind of the pressure to to incorporate AI into process and workflows is ramped up, and then also to be like innovative innovative, like make up new things. So it is very overwhelming. And I just I mean, you kind of touched on it a little bit in your last answer, but how how does that impact the the day-to-day? Like, how are you changing the way that you're approaching intelligence ingestion and protecting Yahoo, but also discovering new things, creating new things, and and just you know, continuing to be the trailblazing organization that the paranoids has been for years and years. Um, now with this new AI overwhelm, I guess you could say, other than taking PTO off, taking PTO.
SPEAKER_00Right. Yeah, which is important. Don't forget your PTO. I think what I mean, one of the ways to be honest is that we as the paranoids recently uh frankly did an entire org-wide, we called it a skill-a-thon, but like it was a seven-week process to get every single paranoid, regardless of your role, whether you're on the Intel team, whether you're in the SOC doing the vulnerability management and our you know, our security comms team. We wanted everybody to have some ability to do create AI agents and skills. And so we had a you know, seven-week process where we every single paranoid was expected to make a like a for us, it's Claude, make a Claude skill that is relates to your job. And then we had at the end of that a sort of demo and um sort of like award like sort of ceremony where people, you know, in different categories would, you know, can kind of give demos of what they built. And it made like people really excited about the opportunities there and what, you know, not just saving time, but like the ability to sort of combine different types of data. And, you know, a company like Yahoo, we've got a lot of data and we're able to combine that in interesting ways to, you know, sort of understand what you know, actors are doing on our platforms or how they're you know targeting our users. And so as much as possible, kind of like making use of all the data that we have fully, I think is one way that we're trying to kind of like, you know, before we start looking to purchase, you know, expensive feeds or whatever, like what do we have that we're not making the best use of? How can we use that to then, you know, bring that to the Threat and Tel team and then sort of federate that out to our product teams and making sure that they're, you know, they have like an understanding of what bad actors are doing and how to prevent that from happening. And, you know, that doesn't necessarily need AI, that can just happen at any time. Um, but I think with AI, we have the ability to scale it really fast. Yeah, just kind of like helping almost create a baseline across the entire organization of like, hey, you can be you're operating at this speed, but you can be at this speed. Instead, like a much higher speed. And then you don't have to worry. You can almost abstract away some of the cost of like having to go out and acquire data and you know processing stuff.
SPEAKER_03So this brings me to my next question because you're talking about having access to, well, you already have access to a lot of data. What are you not using yet? So you're kind of expanding your visibility a little bit or expanding the amount of information that you're going to be playing with and ingesting and taking action on and you know writing reports on, et cetera. And as you are finding new things, finding new information and incorporating that into reporting, how are you prioritizing or acting on new information? So if you are an organization that is receiving threat intelligence, you might notice that your feeds have increased or the output that you're receiving has increased. If you even if you don't pay for threat intelligence, you might notice that public blogs are becoming a lot more frequently published and there's a lot more information out there. And, you know, as a as a responder, as a defender, you're constantly making triage response and defense decisions. That might mean that some of these things that are are in the public or that they're they're aware of might go unaddressed. So you still have to do some of that prioritization. And I'm curious, you know, one, are you finding as a consumer of intelligence and information externally that the output is getting better because of all of these new tooling? Or is there just more of it? And then second of all, how are you making those decisions about what to prioritize as all of this information flow has increased and we're all kind of like, there's so much more that I can look at and take action on. Like, what do how do I decide where to go first?
SPEAKER_00Yeah. I mean, I think I I'll use kind of to the first part of your question, I'll use almost like an analog to our bug bounty program. And so we have um one of the oldest bug brownie programs in the industry, and we've noticed a lot of, I'll call it AI slop in in submissions recently. And so the sort of signal-to-noise ratio is evolving toward noise. And so, because it's so easy for people to produce, you know, a bunch of high volumes of written material. Frankly, like I my my attention span is shortening, unfortunately, not growing. And so, you know, somebody sends me what's clearly an you know a 14-page AI written missive. Like, I'm like, I'm not gonna read that. I just don't have time. I could use AI to summarize it for me, but like then I don't know, it just feels weird. So I think we're that we are seeing, like you said, an explosion of IOCs, of reports. And I think for us, we try to bring it back to basics. Like, what's our mission? What are we here to do? We're here to not just acquire Intel for Intel's sake, but we're trying to protect our platforms and the people on it. And we want to prioritize in terms of like the highest threats. And and and so, you know, what are the what are the most significant threats we're concerned about? And then how do we prioritize getting Intel about those? And then we can kind of work our way down the list. And so, you know, maybe there are certain countries that target certain segments of our users that we're really concerned about, and we want to make sure that we're focusing on sources that that specialize in those, whether it's, you know, folks that are posting on Blue Sky or our blogs, or maybe there are trusted Slack channels or other places where that intel is getting shared. Again, it's like a lot of like quality over quantity, I think, for us. So yeah, I think that's that's in a way I almost answered both. Like it's it's the the volume, but also then how we prioritize is you know, focusing on what what and obviously we're here to protect people, but we're also here to help a business grow. And and so we uh we do think about, like you mentioned, you know, fantasy sports. So what are the types of threats, usually not from an espionage, but maybe more from a crimeware area like that might affect that type of business? We'll just make sure that we're acquiring those.
SPEAKER_03Hey, uh, you know, I think China would be very interested in my fantasy basketball parlays. Um I think they would.
SPEAKER_00I mean, they could do some really good targeting there, right?
SPEAKER_03Very interesting. They're not very good.
SPEAKER_02Sean, you mentioned AI slop, which is one of Selena's favorite themes um and topics. And of course, we, you know, we I think we expected that there would be some slot polluting the CI CTI ecosystem. I mean, this it's polluting everything. Even threat actors are complaining about the AI slot polluting their, you know, markets and forums and such. And you do have, um, you mentioned that you do have really good sources that you prioritize and so on, but of course, naturally, new sources are popping up every single day and new new voices on LinkedIn and new voices on Blue Sky. And just kind of curious with the volume of it all kind of surfacing, how do you vet these sources? Are you are you able to kind of cross-reference them with what you're seeing in your data? Or it just seems like another point of overwhelm with the amount of humans that have to be in the loop to kind of check things out and and validate and so on. So I'm just curious how you handle that.
SPEAKER_00It it is. And it's actually not really, it's not a new problem with respect to AI. Like there's there's a lot of, there's always been slop, I think, when it comes to IOCs. And that's a specific, that's a particularly concerning problem for a company of our size and scale, because what say, you know, a low fidelity indicator in a, I don't know, a chemical or manufacturing company or whatever might be really like if you see that, oh my God, that's that's red alert, we might see that indicator all the time and it's super normal. And so, like, and especially with the sort of like global reach and lines of business that we're in. And so, like, really first understanding, you know, what is the relevance of that indicator? Is it you know relevant for our environment is one thing. But then, like, when you talk about the sources that are starting to emerge, how do we vet those? In some ways, it's, you know, we like we need that high fidelity assurance. And so there's a lot of behind-the-scenes vetting we do. It's I, you know, I'm I'm in CISO communities. Our threat intel researchers are in, you know, espionage or or crimeware communities. And we're often comparing notes and saying, like, hey, like, have you heard of this person? Where did they get their data from? Oh, yeah, I had that same experience. It wasn't that great. So, you know, there's frankly, the like whisper networks is the wrong word, but I would say the like, you know, the sort of common, like the affinity networks, I guess, of of professionals across industry and you know, academia and even government, there are opportunities for us to say, hey, have you heard of this person? What's what's their deal? Okay, yeah, I don't trust it either. Let's let's exclude it from our collection. And because we're really interested in like, frankly, not just we're interested in quality over quantity. I'd rather focus on sources that might be lower volume but have a much higher fidelity. And especially if it gets us toward like attribution and understanding why an adversary is doing certain things, what they might do next. And then really, like if we're looking at deterrence, like how do we, you know, potentially stop them from doing the bad thing too?
SPEAKER_03Absolutely. And that ties into the another question that I had about sort of measuring success. Because as things increase, as you're getting more and more information, you are taking steps for defense, whether that's improving detections, whether that's improving the knowledge and understanding of the adversary, creating better attribution, um, potentially identifying or naming, naming and shaming, doing, like ultimately even moving to something like a takedown or some sort of disruption. But all throughout that, there are, you know, pieces of the overall puzzle that have to play a role into it. And to achieve like full success, you have to, you know, meet certain like standards, go through different metrics. And, you know, how how do you know if you um and your team are sort of closing the gap and improving defense and not just moving faster on the wrong things? So, like what metrics tell you that your process and actionability and um decision making is really improving and hopefully, you know, scaling with AI and Agentex solutions?
SPEAKER_00Yeah, I love that question. I think I mean one of the traps that a lot of security teams fall in is they like look at metrics as the goal. And like, you know, whether it's like the number of incoming, you know, attacks or what like there's all sorts of crazy metrics that get tossed around. I think for us, we always try to ground ourselves in the outcomes we're looking for for our users. And our by users I mostly mean consumers around the world. And so, for example, like if we are if we ingest some uh some new intelligence feed, and then we are able to detect you know APT activity, and then we're able to notify uh a number of users who then we can measure take a certain action. Like, for example, one of the things that we do as as well as many of our peers in industry is we had a program to notify consumers who are being targeted by government-backed attacks. And not only do we notify them, but we say, hey, here's what you can do based on the security posture of their account to increase their security. And we also say, hey, by the way, you you know, you might be targeted in the real world too. So watch out for that. But we can measure what they do after that. Do they, you know, turn on two-factor authentication? Do they move to a more secure, like they move out of SMS and they move into um, you know, WebAuthN or um FIDO security keys or or you know, pass keys or something? I actually recently moved out of um my old physical security, like physical Yubi key into a into a pass key for my Yahoo and my Google account. But I uh, you know, we can start, we can measure those actual outcomes and say, like, oh, that actually had an impact. And then statistically speaking, the the amount like a fully MFA'd user is much, much less likely to fall prey to, you know, a phishing scam or to have a password reuse issue where their account gets taken over and used to, you know, whether it's intel stealing or um impersonation or whatever. So we can measure those outcomes and say, aha, it's working and we're actually having a meaningful impact in the lives of our users.
SPEAKER_03That's amazing. How often are you doing that? Like, is it with every change you're measuring that? Or how do you like, I don't know, how how how is that part of your overall like intelligence cycle?
SPEAKER_00I mean, we're doing this this notification work every day, all day. And so it's it's sort of a we can look at trends and say, okay, well, here's on on the graph of you know this particular APT group, here is when we ingested this piece of information. Look, look, look at the change here, or look at the decrease and you know, the fidelity there. So I think it's it's just an ongoing process. It's not like a gated thing where, you know, from day in and day out we're doing it. It's more like we made this change. Here's what's happening in the real world in our, you know, in our data. And is that the outcome we wanted? And the outcome we want is always, you know, a net positive impact for our users.
SPEAKER_03Absolutely.
SPEAKER_02I wanted to circle back on the burnout piece and the overwhelm and like still really stuck on that. And you said something incredibly compelling in the beginning about how you're taking this opportunity to rebuild and basically reset and get rid of some tech debt and so on. So I'm very curious if you're comfortable with sharing, kind of like how how you're approaching this in a more practical way on the day-to-day. Like, how are you uh using AI to augment these process this process? And what it how what are you building from the ground up and resetting and just kind of, you know, some examples uh of your approach to this?
SPEAKER_00Yeah, I so we're very early in the process. And I think we're still company-wide really working to get people comfortable with this because it's a big change and it's a it's a cultural change and it's a it's a way of working, it's a future of work change. And so I think we're trying to be very deliberate in how we approach it and recognizing that some people, there's like an adoption curve, and some people are going to be really eager and some people are gonna be hesitant. And we need to make sure that, you know, we're we're sort of meeting people where they're at and trying to get them, you know, to come along on the journey with us, uh, regardless of, you know, of of what their initial preconceived notion of AI is, right? And there's there's plenty of things that, and just to sidebar a little bit, like, you know, there's like there's all sorts of concerns, whether it's environmental or you know, ethical or whatever, that people have with AI. And I think it's there's space to be like, yes, and like we we we recognize that there are problems, but you know, we're also all moving this way. And so like let's figure out the best way to make that work for you. So I think that's that's kind of like part of it is just like understanding how to meet people where they are, go on that journey together. So I think, I mean, as you look at at AI, and I've been using, again, for us it's Claude, I've been I've been using it to build a lot of things myself, which like as CISO, I'm like sort of building like almost like proof of concepts, and then I could go to the team and like, hey, look at this thing I built, and they're probably like rolling their eyes and being like, okay, great, thanks. But um it's it's been it's been kind of fun, but I think there's an opportunity for some things because the cost of like we don't we're not thinking in terms of quarters anymore or you know, second half of the year. We're we're talking in terms of weeks or days in many cases to build a working application from end to end. And so the cost of that labor is really cheaper. It's just in some cases, just token cost. And so it might be easier to replicate the functionality of an application and build it from scratch instead of like sort of you know refactoring it and making it kind of like modern. Like just start over, same inputs, same outputs, get there in a in a modern, scalable way. And, you know, especially with a company that's been around for as long as Yahoo has, there's probably tons of use cases in that code that don't matter anymore and you know, are sort of like it would take a really long time to kind of parse out and say, oh, we don't need this particular function. So, in some ways, starting over a greenfield lets us do things, you know, modern, like maybe totally end of the hood, it's it's not even visible to the user, but it's so much faster, it's so much more efficient. And and then of course there's the security advantages. So we're trying to say, you know, like let's let's simplify the lives of our engineers by making things just so much easier to understand. And the the knowledge, you know, you don't have to like feel like you have a PhD in a certain field to be like, okay, I can finally understand what this code is doing. And so we're trying to pilot and with a few different teams now, like let's let's take this thing and greenfield it and and make it brand and brand new. But it does the same thing as the as the last one did. And I think as more and more people, teams, organizations, you know, move like really into this like agentic future, it'll be easier to build new than to sort of revise old. And then for security people, why we do that, why don't we make it memory safe? And let's, you know, take out entire classes of vulnerabilities and exploits while we're at it and you know make make the the like we have a saying inside the paranoid. So let's make the safe choice the easy choice. And so why don't we make that basically as we build code, like it's automatically by default, has all the knowledge that the paranoids have, and we can just build safe from the beginning and you know, like launch, you know, really quickly because we know it's built the right way.
SPEAKER_02I love that. I love like the the idea of re-strategizing and bringing everything up to speed, but by taking maybe like five steps back and starting from scratch, I think that's just such a fantastic way to approach the use of AI and especially like insecurity, right? So many things go back so so many years. So I love that, Sean. Thank you for sharing that with us.
SPEAKER_00Well, we'll we'll see how it goes. Um still early, and that's that's the pitch, I guess.
SPEAKER_03Wow. It sounds like a pretty good recipe for success, in my opinion. Yes. I am curious just before our last fun question that I have for you. I'm wondering, I am one of those people that is skeptical of AI. I know it's very helpful in technical workflows. It's a great productivity tool. I have come around on becoming a programmer and doing it to build some of my own tooling and scripts and things like that. But I am, of course, concerned about the impact to the environment, the impact to people using it in ways that are not necessarily for productivity and more sort of like this like Gen AI stuff. And certainly we've seen some things that have happened to young people, for example, or other folks that have had pretty negative consequences interacting with it. I'm curious, you know, what do you tell your folks who are suspect of this or who are not quite as enthusiastic about the AI revolution as maybe some other folks are in the tech industry? And how do you kind of help them grow with this new, frankly, like new world that we're in while also recognizing and empathizing with their very real concerns?
SPEAKER_00Totally. Yeah, I think I think the important thing is that empathy. Like you can't just dismiss it. And you have to like really say, okay, like let's first understand what is your concern. Is it is it moral, is it ethical, is it like my oldest kid is uh um an artist? And it's so interesting. Like, oh my god, the kids these days. I'm sorry. But but no, like seriously, if I if I I'm a parent, if I talk to other, you know, parents who work in tech, uh, or just parents in general, like there is a significant uh perception problem when it comes to AI in the sort of like under 20 set, let's say. And you know, even my my seven-year-old will be like, oh, that's AI, and use that as short shorthand for like, I don't think that's real or I don't believe it, or it's fake. So I think we need to, as leaders, we need to, and just compassionate people, like understand, okay, what is your concern? Let's talk about ways that we can mitigate that. As from a corporate perspective, can we, if we're concerned about water usage, like are there things that we can do? Are there, can we ensure that we're only like maybe we have choice over which models we use? Well, what types of data centers are they are they leveraging? Or what's the sort of what's the future plan for how we're gonna grow this technology? Are we moving toward more like, for example, there's a lot of use of the LLMs that are cloud hosted, that are in data centers, that are using tons of GPUs, but like a local LLM could work just as well and have a lot less footprint. Um, and so what I'm really interested in in seeing as the industry evolves is what we've seen in cloud is this the concept of like thin ops and looking at it from dollars and cents, but like, you know, as we all get better at writing prompts and you know, leveraging models and agents and skills, like how do you do that in a way that minimizes the impact, both financial and environmental and power consumption, water usage? So I think there's gonna be a lot of focus in that coming up. And then also like thinking about when do you use AI and when do you not? And so, for example, is that a if you're gonna use, you know, if you're gonna rewrite an app, let's say I'm gonna, my team is gonna rewrite their SIM for handling, you know, threat um incidents and and and detections. Do we use AI to rewrite it and then one and done? Or do you like have an ongoing AI sort of like agent in the background working? And is it a one-time cost in tokens versus like an ongoing cost? And I think a lot of people are like, oh yeah, I'm just gonna have AI built in there all the time. But do you need that? And like really making sure that you're building what your team is asking for, and as opposed to like shoving AI down their throats, which is I think we all sort of feel that when we go online and we're like, I don't want AI here. Like, I want give me this, give me the intel. Or even in the applications we use, like, oh, there's now a chatbot. Fantastic. I didn't ask for that. So, and I think that's honestly not to be a corporate chill, but like I think that's one of the ways that Yahoo is approaching AI is like, let's put it where it makes sense, but not where it doesn't. And because there's a significant, you know, uh amount of the population who's like, I don't want it here. I only want it where it makes sense. Uh-huh. So yeah, I think there's things that we can try to do to meet people where they're at.
SPEAKER_03Well, you almost actually just answered my final question too, because I was gonna ask, you know, everyone always, you know, the joke is as, you know, executives say AI and everything, but you as a CISO, have you ever said we can't use AI for this? No, no, no.
SPEAKER_00It's interesting. So I I have definitely said it, but then like I kind of as things get better, as hallucinations get less frequent, I feel like maybe my my my line continues to shift. I do feel like human in the loop is really, or human on the loop is maybe sometimes people say that instead, is still there's still a place for it. And the thing that I don't want is I want AI to replace the toil and to help us move at maybe, you know, help the program move in machine speed. But like I want creativity, I want the spark, I want people's passions to come out. I want if somebody is really interested in a certain type of threat actor or technique, I want them to be able to pursue that. And whether it's maybe doing more threat hunts that leverage that, and you know, we've we've freed up maybe more of that like kind of like detection response um life cycle. And now we're doing more hunts with that creativity. But I think like you can't replace you know, human ingenuity. I actually used to do a bunch of computer forensics when I was in the government. And one of the things that, and maybe this is how my like, you know, neurodiversity or brain works or whatever. But one of the things that I got a lot of value of was like looking at the data myself because I almost felt like I could find the patterns that the software couldn't. And it took a lot of time, but like I was really able to like make insights and connections that I think had I trusted the tool, I wouldn't have been able to. So I think as security people, healthy skepticism of what the tools are telling you is really important. And you have to have the knowledge to say, okay, that that output doesn't actually look right. Let's dive into it and see see what went wrong. And if you, you know, automate a process and then you know fire your experts, like, well, how are you ever gonna know that you're really, you know, that you're getting the right output? So that's that's my thought.
SPEAKER_02Amazing. I love it. I just want to say I keep I have in my head like everything that you're saying, your approach to AI is so intentional and so like pragmatic. Like it's it's grounded in reality. Your your humanity is still very much there. It's it's a really refreshing approach um to hear from uh somebody in a C, you know, C-suite of you know position. Not to, I don't hope it's that's not offensive to anybody, but I keep thinking like AI, always intentional, like always you're it's like the new kind of AI. Yeah. I love that. I'm gonna steal it. Yeah, you can absolutely. I got it. I was inspired by you, so that's perfect.
SPEAKER_03So amazing. Yeah, I I love that AI, always intentional, and I think that that that sounds like a hundred percent your approach at Yahoo and with the paranoids, and that's fantastic. And I hope folks who tune in to this episode can take some nuggets of your wisdom back to their own teams and maybe rethink or think differently about how they are implementing Agentic Solutions, AI, working with their team members to reduce burnout and come up with creative solutions to problems that they might not have thought of before. So, this was a fantastic conversation. Thank you so much, John, for joining us.
SPEAKER_00This was super fun. Thank you for these wonderful questions.
SPEAKER_03Absolutely. And to all our listeners, thank you so much as always for tuning in. And until next time, happy hunting.
SPEAKER_04You've been listening to Discarded Tales from the Threat Research Trenches, a podcast by Proof Point. Never miss an episode by subscribing to the show in your favorite podcast player. Happy hunting.