AI, Bug Bounties & the Vulnerability "Slopdemic"

Bugcrowd founder Casey Ellis joins me to dig into what AI is actually doing to bug bounties, vulnerability discovery, and open source security. We get into his "slopdemic" framing, the curl bug bounty saga, VDP readiness, the pentest market correction, and where security research policy heads next.

Casey Ellis is the founder of Bugcrowd, co-founder of disclose.io, and a board member of the Security Research Legal Defense Fund. These days he advises and invests through Tall Poppy Group and works at the intersection of security, AI, and policy. His argument is that the vulnpocalypse was already here, and AI has made the cost of both finding and reporting vulnerabilities collapse at the same time.

In this episode:

• Casey's path from building Bugcrowd to advising, investing, and policy work
• Why more practitioners need to get involved in policy, and why law is just code
• The slopdemic vs. the vulnpocalypse, and what actually changed in submissions
• AI lowering the bar for a broader, less predictable pool of threat actors
• Daniel Stenberg, curl, and maintainers below the security poverty line
• The lightning rod vs. rockets distinction between VDPs and bug bounties
• The pentest market correction underway from AI pricing pressure
• Collapsing OODA loops, hack-back, CFAA reform, SRLDF, and disclose.io

Chapters:

0:00 Intro and Casey's background
 2:56 Why practitioners belong in policy
 6:22 The slopdemic vs. the vulnpocalypse
 9:40 AI lowering the bar for threat actors
 11:47 Open source, curl, and the security poverty line
 15:37 VDP vs. bug bounty readiness
 19:20 The pentest market correction
 24:20 What breaks first in vulnerability management
 27:20 Hack-back and non-cooperative defense
 28:43 A near-term playbook for security leaders
 31:40 CFAA, SRLDF, and disclose.io

Connect with Casey:
 LinkedIn: https://www.linkedin.com/in/caseyjohnellis
Blog: https://cje.io
disclose.io: https://disclose.io
Bugcrowd: https://www.bugcrowd.com

Resilient Cyber: https://www.resilientcyber.io
Subscribe for more conversations with security practitioners and leaders.

Transcript

SPEAKER_01 0:00

You've called it a bit of a slopdemic instead, where signal is up but the noise is up with it. You know, walk me through what actually has changed in terms of submissions hitting bug bounty and disclosure program over the last year or so.

SPEAKER_00 0:10

I love how you distinguish vulnerability discovery from vulnerability exploitation, because they're not the same thing. And I do think we're heading towards industrialized exploitation, which is kind of what I think of as the vulnerable apocalypse. That's like dogs and cats living together, mass hysteria, that kind of scenario, right?

SPEAKER_01 0:26

He said there was already a correction underway for AI pricing pressure around pen testing.

SPEAKER_00 0:30

If you've got folks that actually do understand, you know, risk management, they've got this like actual mandate or this actual drive inside the organization to reduce risk, they're the ones who are gonna pay good money for stuff.

SPEAKER_01 0:43

You know, what does uh sustainable vulnerability handling look like for an open source project with no triage team behind it?

SPEAKER_00 0:49

If you've got a bug bounty program, so it's a like a VDP with rewards, you're gonna 10x the amount of attention that you're getting. And that was true before AI.

SPEAKER_01 1:12

Thank you for joining the Resilient Cyber Show. My name is Chris Hughes, and today I'm joined by Casey Ellis. Casey, thanks for being here again. Thanks for having me, Chris. Yeah, I'm excited to have you back on. We've been kind of texting and missing each other, and time zones will do that when you're on the other side of the world. But now we're in the you know, same continent at least, so that helps. And but for we're more or less on track. So it's a good thing. Yeah, yeah, we're getting there. So, but for folks that don't know you, of course, haven't followed your career and what you're up to, please give us a bit of a background real quick.

SPEAKER_00 1:37

Yeah, sure thing. Uh so yeah, uh hello everyone. Um, and yeah, thank you uh for having me again, Chris. It's always fun to catch up. Uh yeah, my name's Casey Ellis. I'm best known for starting the company Bug Crowd. So, you know, Bug Crowd didn't kind of invent volume disclosure or or Bug Bounty, but we did kind of kick off this idea of putting a platform in between all of the you know white hat hackers that are out there and all the people that need their input. That was cool. That definitely escalated quickly. But just in general, been very involved. You know, I've been in cybersecurity like my entire career, really out of high school. More recently, been doing a lot of work on policy and working with with cybersecurity startups. So that's the uh the short version.

SPEAKER_01 2:15

Yeah, you've done uh quite a few different things in your career and you have a pretty broad perspective. And like, like I said, many folks, and and you touched on this, like know you from the bug bounty background, the bug bounty platform category and so on. You kind of help essentially create it from nothing. But these days you're advising, you're investing through Tall Poppy Group, uh, working at the intersection of security, AI, policy. You know, what pulled you from running the company day-to-day to kind of this broader portfolio of work and that has your attention and interest now?

SPEAKER_00 2:40

Yeah, definitely. I mean, uh there was a couple of things. One was that we'd we'd made some changes like over the years. So did the bug crowd thing for 13 years, did the CEO thing there for seven. Um, we made a switch, and then we made a second switch. And the second one was awesome. There was kind of a sense of, okay, I'm now in a position where I can sort of step back from this. We were talking about it a little bit just before the show. Um, I actually had some pretty significant health stuff pop up kind of, you know, out of the uh out of the woodwork about two years ago. So that kind of prompted that thinking along a little bit. But more broadly, honestly, like there's just there's a lot to do. Like, I don't see, you know, Bug Crowd is a company, but I started it really as a as a mission in a lot of ways to to kind of democratize the idea of security feedback as much as possible to try to kind of create a good, safe environment for hackers that operate in good faith to do their thing. And like there's a lot of work still to be done on that stuff. You throw AI in on top of that, got involved in all of that back in 22 or so, and it just accelerates it. So there was a part of me that's like, okay, I need to, you know, free myself up to do more and just really figure out what that looks like. That's ye mostly why.

SPEAKER_01 3:46

That's awesome. So I I I'll definitely get into the the bug bounty, the vompocalypse, as they call it, the slapdemic, as you call it, all the things like the technical topics. But you know, I know you actually spent a good amount of time thinking and talking about the policy side of things too. Yeah. And I wanted to actually uh ask uh why. Like I I mean, I know why. Personally, I think like when you when you look at the field, like we see so many, after you've been doing this for some time, you see so many kind of systemic problems that you know like policy is the kind of the blunt instrument that kind of systemically can address some of these things at a broad, uh uh expansive level. But I'm curious, like what draws you to the policy side? Because I feel like we need more practitioners and and technical folks and so on involved on policy.

SPEAKER_00 4:24

Yeah, no, a hundred percent. And probably that's that's actually a big thing. So I'm super, super involved in like Hackers on the Hill, for example, which connects policymakers with with technologists. Disclosure are really the the kind of the core driver behind the mission of that is to you know create like the infrastructure that powers vulnerability disclosure across the internet, but part of that is to make it safe and legal and and kind of normalized in a lot of ways. And and you know, I I kind of cottoned onto that being important fairly early into doing bug crowd. I think what you just said is exactly right. Like policy becomes a really important way to drive change, but I think the other thing is that it's becomes a really important tool to kind of lock change in once it happens at the at the precedent and the and the at the uh the grassroots level. So like when I started to see the whole bug bounty thing and even just like crowdsourced security, like hackers on the outside, not all being burglars, right? Um, oh, we're locksmiths too. Like we knew that in the space, but everyone else didn't. And the law didn't reflect that. So it's like we need to lock this stuff in because that is a big part of the future. And yeah, that's kind of what got me involved. So it's interesting. I mean, it's you get to do a lot of really fun stuff. You end up talking to people you didn't ever really expect to speak to. And I think on the legal side, like, you know, the law is just kind of like code. So, you know, when you think about like hacking code or or getting code to do a thing that you want it to do that it doesn't currently do, like how to change the state of those different things, it's an intellectually interesting problem as well.

SPEAKER_01 5:53

Yeah, I love that framing actually, like how you put it, like it's uh similar to code or systems thinking, because you know, as you said, uh we have a lot of technical debt, including in our policies and regulatory environments and how we handle certain things. And uh yeah, that's an awesome framing. I never heard it like that.

SPEAKER_00 6:07

I think policy is ultimately a reflection of technical debt. I mean, I'll I'll stop after this, but one part of it is that policy is almost always a trialing indicator of where technology is up to, and I think that's only getting worse. So this idea of like more of us getting involved in this stuff and actually helping, you know, inform where we can, like running for office, if that's a thing that's on your bingo card, like more of us should probably be thinking about that type of thing as well.

SPEAKER_01 6:29

Yeah, I agree completely. I've spent a lot of time in the public sector on my in my career, and it's like it lags technology severely, and like I feel like AI is only exacerbating that. So we need to kind of address it quickly because it is gonna get worse. Yeah. Um, so you know, I used these phrases a moment ago, uh, the volume pocalypse has kind of taken hold as people call it with uh AI industrializing uh vulnerability discovery and exploitation and things like that. Um you've called it a bit of a slopdemic instead, where signal is up but the noise is up with it. You know, walk me through what actually has changed in terms of submissions hitting bug bounty and disclosure programs over the over the last year or so.

SPEAKER_00 7:03

Yeah, and honestly, I think just the internet in general, when you think about actual breaches and what those look like too. I think to me, like when I first heard the term volume pocalypse, yeah, I I love how you distinguish vulnerability discovery from vulnerability exploitation, because they're not the same thing. And I do think we're heading towards industrialized exploitation, which is kind of what I think of as the volume pocalypse. That's like dogs and cats living together, mass hysteria, that kind of scenario.

unknown 7:29

Right.

SPEAKER_00 7:30

You know, in a discovery sense, like based off what I've seen over, you know, now 13, 14 years of hacking the internet at scale, like the Volume Pocalypse was already here. It just sort of wasn't evenly distributed, right? Like we saw we saw it when we first started throwing a crowd at solving this problem compared to having one person as a pen tester. It's like, oh, okay, there's a lot there that's not being found just because the firepower isn't being applied and because writing secure code is hard. So that's like a pre-existing state. But to your question about the slop demic, I think the thing that AI has definitely done is, you know, made the cost of discovery of security vulnerabilities a lot lower and also made the cost of reporting security vulnerabilities a lot lower. So like what happens is this is where you get your slot from, right? It's like I I can just send a report because I think this is maybe real or I'm being opportunistic or whatever it might be. That's always been a thing, but now it's like 10x'd. And then on this side, like there's a lot to be found. That's gotten easier as well. So that's gone up 10x at the same time. And you end up with this like delta between you know the signal and the noise that's actually a lot bigger than it was before. So that that to me is the sloped demic in that sense. It's like, okay, we've got to figure out better ways to understand like risk and triage and all those different things because ultimately it's not the fault of AI or security research. It's the fault of like software being hard. We're just kind of more, you know, the the knowledge of that being true is more evenly distributed at this point. I think is probably the best way to frame it.

SPEAKER_01 9:01

Yeah, I like that framing a lot because like I had written a book on vulnerability management a few years ago, and like it seems like vulnerability management all of a sudden is cool again. Like no one cared about it. There was like this tedious, cumbersome activity that everyone dreaded, and like, you know, it lived in spreadsheets and scanning tools and and and still does a lot of in a lot of ways. But like, you know, now everyone's concerned about technical debt and and findings and disclosures and remediation. And uh, as you said, like, you know, there's also some nuance to it. Like I had read uh uh first, the program that runs like uh CVEs, et cetera. Like they had put out there that yes, we're on track to 66 to 100,000 CVEs this year, but exploitation still has ri remained relatively flat. So there is some nuance there. Doesn't mean it's not going to change in a year or so where maybe we have widespread, you know, uh industrialization of exploitation, which would be a whole different kind of ball game. Uh but i as you're point as you're pointing out, like it's been here for a while. This problem has been something that we've been dealing with for some time.

SPEAKER_00 9:51

Yeah, yeah, you're right. And and I I I think on the the exploitation side of things, it's like it's reduction of the cost of being able to chain vulnerabilities together or to create a campaign that has some sort of impact associated with it. The use of AI to drop that cost has kind of trailed the discovery work that's happened, but not by much, frankly. But the other bit there that's to me kind of a wild card that we're not really ready for is that it does lower the bar to ride. So you end up with a way more diverse and way broader pool of potential threat actors. I do think that, like, you know, as a as a defender industry and and as a you know just general defender conversation, which by the way makes its way into policy as well. We've kind of relied on this fairly predictable set of motivations from the TAs that we expect. And that's in the process, I think, of shifting.

SPEAKER_01 10:41

Yeah. I like the you you use that phrase lowering the bar. And I feel like, you know, you'll hear people say like AI has democratized uh development. Everyone can be a developer now, citizen developer, these kind of phrases. Well, it's also democratized a lot of other things like vulnerability discovery or X maybe maybe uh exploitation activity and so on. Like it's kind of democratized a lot of things, some of them good, some of them bad. Um and I think that's what we're dealing with.

SPEAKER_00 11:04

When you think about it, a criminal is just a business person that is on the other side of the law. So like all of the benefits, yeah. When you think about like, oh, I can vibe code my idea now and and and try it on, there's nothing stopping necessarily someone with criminal intent from doing that. It all kind of plays out in the same way. So yeah, what it brings it back to is this idea of um, you know, from a vulnerability management standpoint, no, we're not going to get around to all of these things. And that was never actually true. And yes, the thing that becomes really important right now is understanding risk and understanding how we prioritize, you know, the the things that are most likely to happen to us and the things that we can do to make ourselves as resilient as is rational. It's like possible's not even necessarily the right way to think about that.

SPEAKER_01 11:44

Yeah, and there's like, you know, the prioritization conversation is one that enterprises have around, you know, C VSS, EPSS, Kev, exploitability, all these kind of things. But I know you spent a lot of time with the open source community as well. And you know, someone uh I've kind of watched this evolution of of this individual, like one of the most kind of uh present and and engaged maintainers in the community, Daniel Stenberg, has kind of I've watched his arc over the last like six months and it's been fascinating because he went from like this is all AI slop, you know, get this shit out of here to like some of these things are pretty legitimate. And now we're actually finding a lot of things. And like, you know, he's it's he's gone on this arc that he's gone on and he talked about drowning in slop. Now he's drowning in legitimate findings. And you've written about like volunteer-run projects can't sustain uh bounty infrastructure on their own. You know, what does uh sustainable vulnerability handling look like for an open source project with no triage team behind it? Because I forget the stat, but I think it's like it's like one out of ten projects is has a single maintainer, and 94% have less than like less than 10 or something. I forget the butt you know, it's it like it we've always heard that phrase under enough eyeballs, all bugs are shallow, and there wasn't enough eyeballs, and they're still not from the maintainer side, but there's definitely a lot more eyeballs from the discovery side of things now.

SPEAKER_00 12:50

Look, I think I mean, firstly, fully agree. Like, I think I think the thing that's been most valuable about like Daniel kind of sharing so much of what's happened with Curl is to kind of draw attention to the fact that like this is really hard. And it was already really hard, and I think like full props to Curl for actually running a vulnerability intake program that did have a reward attached to it for so long. But you know, the thing there is that like if you've got a bug bounty program, so it's a like a VDP with rewards, you're gonna 10x the amount of attention that you're getting. And that was true before AI. So, like all of a sudden you've like you know gone and and and jacked the entire thing up. If you're not ready as an organization to handle that jump in load, you've got to really think about whether or not that's the right way to actually do it. Now, what I do think as well is that like the vulnerability disclosure side of it, so just being ready to receive reports, I think everyone's got to do that. And the reality is that everyone has a VDP, whether they like it or not, they're just maybe not getting the report. So that to me is kind of a primitive. It just becomes a question of like, do you incentivize this stuff or not? And that's, you know, my take on some of the stuff that Daniel's been talking about. But yeah, like the fact that he's drawn so much attention to the fact that like this is a systemically difficult issue to solve for everyone, but especially for open source maintainers that are kind of below the security poverty line when it comes to their ability to respond. I think that's a really important thing to call out. It's gonna be an interesting one to fix because I I think, you know, we've definitely seen, like I've seen a lot of companies and projects just kind of turn everything off and be like, no, too much. That's I can understand that reaction. Like I empathize with that reaction. I'm not saying it's bad or wrong in that sense because you know it's it's a hard thing, like I just said. But at the same time, like that's kind of ostrich risk management at that point in time. And it ultimately doesn't work from a system level security and resilience standpoint, and it'll ultimately kind of end up harming things. Yeah.

SPEAKER_01 14:44

Yeah, the security technical debt doesn't go away just because we kind of pretend we don't see it and put blinders on. Um, you know, I had covered.

SPEAKER_00 14:50

Yeah, we've tried that before and it didn't work, so I don't think it's gonna work this time.

SPEAKER_01 14:53

No, I definitely not, definitely not. And so I had I I caught a conversation of uh you had on a podcast that I was on as well recently called the secure disclosure. And I may be misquoting you, but you had used a line and you said like VDP is like putting up a lightning rod uh while bug bounty is shooting rockets with wire at them into a thundercloud. It's something along those lines. Um but a lot of organizations still conflate VDP and bug bounty. You know, how should a leader decide when they're ready for which one and and how so, you know?

SPEAKER_00 15:19

Yeah, so and just to just to explain that analogy a little bit more, like I think the the version that we were just talking about now, you know, is the idea of like, I don't want you to like I'm I don't have time or it's inconvenient for you to tell me that I've got vulnerabilities in my stuff, like go away. And like in that metaphor, it I'd kind of relate that to walking outside when there's a storm approaching your house and saying, Don't hit me with lightning. Cool, like it's gonna do that whether it wants to or not. And really it's it's more to me an issue of physics and preparedness as opposed to whether or not you feel like it's convenient on that side of things, right? So the idea with the lightning rod is that you anticipate that that could happen, you set something up for it to be able to go towards and then route it around damage and deal with it. For a bounty, you know, you're going one step beyond, like you're taking advantage of all the same physics, but what you're doing is you're actually proactively going out and trying to encourage that. So it's and I'm like a weather nerd, so that's you know just an analogy I love to dig into sometimes. But yeah, like how you know you're ready, I think it's really a matter of being in a position where you can tiger team your your vulnerability intake, your vulnerability triage, you know, your remediation, your downstream, you know, patch deployment, like depending on whether you're a product company or a library, there's going to be all sorts of different things that are going to be different for every org. But to me, the whole idea of like, can you cope as an organization with 10xing all of that overnight? If you're a company or a an entity that says yes to that, then yeah, you're probably good to go. But the reality is that most people kind of can't because they're they're sort of struggling just with the lightning rod version right now. So this idea of like, okay, we're gonna prepare ourselves to deal with security reports reactively because that's not that's not something that we even necessarily control. It's just a part of being on the internet and and writing software. So we're gonna do that and take that on as a part of like core social responsibility of being on the internet. And then we're gonna take whatever advantages we can take from that in terms of what we receive. Then if we're ready, we'll you know add a reward to it and start shooting the rockets out. And in the meantime, it's a it's a funny one because I actually don't I do think that the idea of people paying for bugs, like what is a bug worth, all those different things. I actually see that as a completely separate conversation from a vendor being able to receive a security report and deal with it. Do you know what I mean? Because I think ultimately on the hacker side of things, like this is valuable work. People should get paid for it. And I've been a longtime believer in that. So it's not about devaluing that, it's about on the vendor side, how are you preparing yourself for what's coming? And then how do you decide whether or not you want to crank the handle on that?

SPEAKER_01 17:52

Yeah, I think it's something that organizations should be looking at carefully because once they do start down that path, like as you said, it's kind of like opening the floodgates. You gotta be prepared for what comes with it and have the institutional kind of infrastructure, right, to support it. Um, speaking about paying for things, like you also on that same that same podcast, you had made a uh a comment there around pen testing. You said there was already a correction underway for AI uh pricing pressure around pen testing. Um it's an uncomfortable message, right, for a big chunk of the services industry in particular. I think so. You know, uh walk me through like that correction. How do you think it's gonna play out and who's gonna get squeezed and how?

SPEAKER_00 18:24

Yeah, I think I mean, probably the biggest thing, um, you know, AI is definitely a component to this downwards pressure, because at the very least, it's forcing buyers to ask the question, am I getting value for money? Like, is the thing that I'm paying for this like or is the thing that I'm paying for worth it, worth what I'm paying? So like that question in and of itself is being triggered everywhere right now. But especially I think in in cyber, and especially I think in you know, areas of cyber where you know, frankly, we haven't really had a good answer to that question previously. Yeah, like the idea of like, cool, I've I've paid, you know, whatever for this pen test. Like, would I have gotten a result in terms of like my benefit as a business that's worth the same value to me for less money or not, maybe, who knows? It depends on how you consume that stuff, like the quality of what you're getting, whether it's associated with like actual risk mitigation or if you're just doing it for compliance or something like that. I do have a theory that I throw around a lot that's not super popular, but I think it's true that we've built a lot of the cybersecurity industry on this idea of selling to people that actually don't really care. Right. Like if you've got folks that actually do understand, you know, risk management, they've got this like actual mandate or this actual drive inside the organization to reduce risk. They're the ones who are gonna pay good money for stuff because they understand the value of it and they're gonna go out looking for that value. For everyone else, you know, there's this automatic compression of the margin just because of some of these dynamics that are kicking in. So, yeah, it's a weird one because they sort of feel uncomfortable talking about it sometimes just on account of all of the AI's you know, scary from like a job prospect standpoint. That's true too, I think, in a lot of ways. But you know, if you've got this industry that's sort of pretty inflated to begin with, in terms of its ability to actually defend the value that it's delivering, then it's automatically going to be more vulnerable to that.

SPEAKER_01 20:18

That's suck that that last piece. I was gonna comment on a few things you said there, but that last piece is so true in terms of the struggling to communicate value that security provides. Because like we're often trying to prove a, we're trying to prove a negative in this career field sometimes. Like, you know, it's hard to quantify, it's hard to explain. You know you need it, but you can't explain necessarily, you can't quantify the value a lot of times. We, you know, there's a lot of different efforts to quantify, you know, risk management and things like that in cybersecurity, but the value to the business in particular uh is something that we have struggled with for a long, long time.

SPEAKER_00 20:47

Uh yeah, and it's it's honestly something that that I like. I'm coming back to the bug bounty model, and I'm not talking about the bug bounty model as expressed by Bug Crowd or Hacker One or any of the like platforms, like whatever this core kind of pre-existing model of okay, that's a vulnerability that I'm willing to pay for because I'm now gonna go off and mitigate it. And like the value to me around that mitigation is gonna be proportionate to what I pay. That's how the bad guys do it. Or and like they don't even have to be bad guys. You think about the offensive vulnerability research industry, right, which predates all of this. The reason they can answer that question, I think, a lot more effectively is they're building a product from it. So for them, it's a positive return, not the mitigation of a negative one. And what I wanted to see, and I have, I think, to some degree seen happen with bug band is is causing people to think about vulnerabilities and risk in a way that's more directly tied with economics. But yeah, outside of that, like yeah, we we kind of suck at it just just in general. Because yeah, it it is, you know, it is effectively like cyber defense just in general as an industry, like if we do it right, nothing happens. So cool. Like, would nothing have happened anyway? We're not quite sure. And that's a overly primitive way of kind of putting it. But if you think about the economics that drive the entire market, I think it's a pretty good starting point.

SPEAKER_01 22:00

Yeah, I mean it's just the nature of how it works and like we all know this that like after an incident budgets tend to go up because now there's something to kind of point to, right? Um I wanted to ask you about this too. I I I spent a lot of time on resilient cyber, of course, and and through blogs and interviews and so on talking about the strain on CVEs and MVD and you know vulnerability databases, et cetera. Uh you've written about the window between vulnerability existing and someone noticing it is compressing like to basically instant. Uh if a discovery goes to near instant while remediation stays at you know kind of the human speed that we are used to with the vulnerability backlogs, you know, what do you kind of see breaking first in the vulnerability management pipeline? Because it it it doesn't look promising if that's the case, if that's the way it's headed.

SPEAKER_00 22:38

Yeah, um, I think we're gonna need to rethink a lot of things, honestly. Like uh, and that's something, you know, uh I've talked a lot about the idea of like the the OODA loop, so you know, orient, observe, decide, act. Like that's been from my perspective, fairly predictable in in our industry for the last you know 15 years or so. And now between like AI discovery, but then also lowering the bar to ride, so you've got more people motivated to do attack stuff for more reasons, you throw on top of that the fact that the world's just generally getting spicier and and kind of not really slowing down when it comes to that. This like OODA loop, I think, is small to the point that we can't necessarily count on being able to fit inside it going forward. I I kind of feel like we're already there, but you know, I don't feel like um that's as necessarily well agreed on. So, you know, when it comes to volume management, like how do you think about you know break glass mitigation when you need to do that? How do you think about your risk management, risk assessment, like ongoing threat modeling so that you can be like really as an organization thinking as proactively as possible around how to stay ahead of this stuff? And then honestly, like planning to fail. Um, like how do you do detection, containment, you know, ejection, recovery, like all those different parts of the the process as effectively as possible. If we've just been focused on prevention, then that's gonna that's gonna be rough. Because at some point when that fails, you you've got a blast radius that's kind of uncontained, and you're gonna have to do that reactively, which is not how you should necessarily be doing that. So this is not me saying vulnerability management is dead or or any of that kind of thing. Yeah, I just I I do think that we're gonna need to start to think a lot more creatively about how we're approaching the overall issue of defense. And you know, that includes like how do we think about imposing costs on attackers? Like if we've got you know, this whole WordPress thing that that just happened, um, you know, something that I've been thinking about a lot lately with with um like really broadly deployed systems that kind of create a system level issue. Like, what if we need to patch other people's stuff because all of a sudden there's like a mass event going on? Like, I don't, you know, I think I said it on uh on the other uh podcast as well, but like the idea of like hackback, the idea of non-cooperative defense, I personally don't, those ideas make me uncomfortable because it's messy. But given the current trajectory that we're on, I don't see us not arriving at a place where we need to be doing stuff like that. So, like, what do we do in the meantime, right?

SPEAKER_01 25:07

Yeah, I I I'm gonna ask you another question, but that's such a fascinating thing because I feel like in the last, you know, not to get political, but in the last administration, I did hear some of this stuff about uh particularly like I think you called it non-coroperative defense, where you know you go in and impact someone else's system, but there was a lot of debate about euphemism.

SPEAKER_00 25:23

I love it.

SPEAKER_01 25:23

Yeah, a lot there's a lot of debate about well, what if you take down a commercial system, the government like impacts you know, business operations, et cetera. And then like the hackback has been kind of something that's gotten more steam recently in this administration around uh the going on the offense and imposing costs on attackers. Uh but then these are perfect examples of like technical topics that have broad uh policy and geopolitical implications and business economic implications, you know, all these kind of things. So it's uh it's the intersection of technology and society. So it's uh they're really fascinating. Uh you you made a comment about uh thinking creatively and how and doing things differently. I wanted to bring back to before we wrap up uh VDP and bug bounty. Uh you also have written and spoke about uh kind of giving a cr a concrete playbook that included, you know, kind of auto automo automating most of the slop triage, uh, raising payouts for hard findings instead of cutting rates across the board. Uh for security leaders like listening right now, like who run a VDP or a bug bounty, what are some of the things they should look to change maybe in the next quarter or two?

SPEAKER_00 26:17

Yeah, look, I I think, you know, like we kind of talked about earlier on, like if your inability to deal with incoming vulnerports is causing you to just do these ones, then you know, to me, A, you're not alone. So don't feel terrible about that. I think that's happening right across the board. And it's not just with vulnerability management. This is like simsaw, like all of the different things that are kind of externally triggered are just getting absolutely hosed right now. So there is, I think, in our cyber defense category, this just global need to get better at triage and get better at kind of understanding, you know, what's important, like what can we like definitively definitively rule out as noise and how can we do that as quickly as we possibly can, but also like not screw it up, and that's that's the hard part. Yeah, the early days of spam filters, like that's an example of what that looks like when it goes wrong, right? So yeah, there's definitely a lot of people working on that problem. Like part of what what I've been trying to work on with some of the others around the disclose IO stuff is how do we start to find ways to implement that at a system level on on this whole kind of vulnerability reporting across the internet problem? But again, that's gonna look different for everyone because it's just kind of a global generic issue at this point in time. So yeah. So if you invert that, it's like, all right, well, and this goes back to the the the guide thing that I wrote. How do you put more emphasis on the signal that you do want? Because it's like you can't tell the internet to be quiet. It's not that's not gonna work. So, all right, if you've got a signal-to-noise issue, the other way to solve that is to increase your signal in proportion to the noise. And what that's gonna take as a leader is to figure out what's important to your business, like what are the things that are the greatest risk factors that you can kind of encourage good faith security researchers to go off and demonstrate. How do you reward them for that so they keep coming back to do more of it? And you kind of build that whole thing out and actually build out a relationship with this sort of elastic, you know, team that exists out there on the internet available to help. It's both at the same time. And I think where we get stuck is that a lot of people think of it as one or the other. I don't think that's true.

SPEAKER_01 28:21

Yeah, as I was listening to you say that, I I thought of the phrase like incentives drive behavior, right? Um, and so I think if you want to change the behavior of that elastic team, as you called it, uh, you have to incentivize certain things. And then that will change the way they behave and the way they interact with you and what they maybe bring to your attention and so on. Like that incentive is gonna change the it will drive help drive those behaviors. Um that's a great point. And I was gonna ask last question for you here. Uh you mentioned disclose.io. We've talked about that a couple of times throughout the conversation. I know you're on the board for the Security Research Legal Defense Fund, and of course you co-founded Disclose.io. Uh, you've called CFAA a reform priority. First off, for folks not familiar, what what is that? And then over the next couple of years, as AI generated research pressure keeps building, you know, which of these policy levers matter most for kind of keeping good faith research safe?

SPEAKER_00 29:07

I saw, and you know, for the audience, I got uh a prerun on the on this question, and I was dreading it actually coming up because choosing between the three things is tricky. Um but look, I think um the CFAA, so the Computer Fraud and Abuse Act, it's basically an anti-hacking law, uh federal law in in the US that's um makes hacking illegal. There's been charging rule changes that we managed to get through the DOJ back in 2022 that mean that you have to actually prove intent that it was for a bad reason, which is good because prior to that it's basically if you break a computer, you're automatically a criminal and you have to prove that you aren't.

SPEAKER_01 29:47

Can I ask something real quick there? So you said you have you have to prove intent or you no longer have to prove intent?

SPEAKER_00 29:53

So you have to prove intent to bring it to bring a criminal prosecution.

SPEAKER_01 29:58

So let me ask, okay, so criminal versus I guess a criminal versus civil suit is actually a difference there. So you can double-click into this and it's a whole lot of things. I want to ask about this because um, you know, there obviously the news, the news this week is like a Frontier lab uh, you know, kind of had an incident where where it impacted another organization and it wasn't intentional. Uh and I think as like Gentec AI kind of becomes more mainstream where systems are taking action on their own, right, and being very creative and doing things that we didn't intend to them, we didn't intend for them to do. Uh I can see that getting real messy. Like, you know, we didn't mean to do it per se, but yeah.

SPEAKER_00 30:30

So the reality of of that one is that like intent is really hard. Like it's the the reason that the charging rule changes are good as progress, but imperfect, is that intent is really difficult to discern at wire speed. Um and that's that's always been true. So like when when this uh charging rule stuff came through in 22, it's like great, this is progress. Um still not a perfect solution. And by the way, all of the states have their own laws too in the US, and then you've got like international laws and all sorts of other stuff. So that's the challenge with it. Um I do think that you know between like Frontier Labs having sandbox escape issues, and even frankly, you know, good faith security researchers having their hack bots go off the rail, which is a thing sometimes, um, there are all these different examples of computers doing things to other computers that look like exceeding authorized access, that may have been in good faith but look like they weren't. And then like the decision tree around like, okay, was that a good thing or a bad thing? Do we try to bring, you know, a civil or a criminal suit? That gets real messy. So like at some point those laws are going to need to be rewritten to kind of acknowledge how the internet works, I think, in a lot of ways. Um, in the meantime, for researchers, it's like documenting your work and making sure that if there's ever a misunderstanding, you can actually talk people through that. And I think going back to what you asked around, you know, disclosure and and SRLDF, like SRLDF exists for when basically companies are just literally trying to, you know, bring a lawsuit because they're annoyed. Right. So this was clearly good faith research, but the recipients, like, I don't like that, that's pissing me off. Here's a suit. Like, I'm trying to suppress and censor that research and and ultimately like intimidate you out of doing more of it. So that exists to basically defend that. And it's a bunch of people that have been doing this stuff for a really long time. So, like, part of what we've signed up for is try to discern this wire speed intent problem that you just called out, which is fun sometimes, but someone's got to do it. Um, I think the other side of it with disclose IO, like the goal there, aside from putting out like infrastructure to make this as simple and straightforward as possible, is to kind of normalize it. So it's like, okay, here are the, you know, here are the rules of the road, more or less. Here's what you do as an organization to say, hey, we're, you know, willing to receive security reports from the outside world and we won't prosecute that. But by the way, if you're a bad guy, we're still gonna cloud up and rain. Um, and it doesn't mean that people on the internet can do whatever they like. That's a tricky balancing act to write. So there's a lot of tooling in there to try to make that easier, is really what it comes down to. So, yeah, there's a lot. I mean, that's a that's a like an internet scale.

SPEAKER_01 33:17

Yeah, I was gonna say I don't think we're gonna I don't think we're gonna stop this in a uh you know 60-minute conversation, but you know, just uh there's a great pausing point because as you point out, like there's so much to this that's there's technical, there's societal, there's economics, there's legal. Um, you know, and I appreciate you jumping on and diving into it with me. And I always enjoy uh kind of le following your insights and your perspective. So thanks so much for jumping on again, man.

SPEAKER_00 33:38

I appreciate that.

SPEAKER_01 33:38

Yeah, thanks so much for joining me and hope you have a good afternoon.

SPEAKER_00 33:41

Thank you. Likewise. Cheers. Cheers.