The Five-Dollar Agentic AI Hacker

Four days. A spare $500 mini PC from Microcenter. An open-source quantized model running on 512 MB of VRAM. And by Thursday morning, an autonomous AI agent named Jenkins was finding vulnerabilities, chaining exploits, gaining root, and maintaining persistence — entirely on its own, with no human in the loop. In Episode 11 of The GIST of Govt IT, Brian and Sean are joined by Marlin McFate, Federal CTO and CISO at Cohesity to dig into the experiment Marlin documented on his new Substack, Peripheral Tech. Marlin walks through the architectural choices that made Jenkins possible — the "discussion skill" approach to working with agents like colleagues, the orchestrator-subagent model, the safety capability files that proved more effective than external guardrails — and why the implications go far beyond "Mythos changed everything." The conversation digs into the real thesis: the barrier to nation-state-grade offensive cyber didn't just fall — it fell years ago, and most organizations are still spending 80% of their cybersecurity budget trying to prevent attacks rather than recover from them. Brian, Sean, and Marlin unpack the digital jump bag, the minimum viable agency concept, why finding the last clean backup is "a bad lie" the industry has been telling for years, and what Cohesity's RAG-enabled secondary data approach means for agencies trying to feed AI without standing up a fifth copy of their data. Plus, AI agents throw fits and take on personalities for their own.

----------

RESOURCES MENTIONED IN THIS EPISODE

Featured Guest
- Marlin McFate, Federal CTO & CISO, Cohesity
- Peripheral Tech (Marlin's Substack)
- Jenkins
- Cohesity Federal
- Cohesity Gaia (conversational AI search for backup data)

The Experiment & The Stack
- Kali Linux (penetration testing OS)
- Ollama (local LLM runtime)
- Qwen open-source models (Alibaba)
- The Mythos vulnerability discovery report (Cloud Security Alliance)
- Anthropic Project Glasswing

Agentic AI & Security
- Gambit Security research on multi-AI hacker (Anthropic + OpenAI split-context attack on Mexican government)

Federal Cyber Policy & Frameworks
- OMB M-26-14 (cybersecurity logging and monitoring for IT/IoT/OT)
- CISA Industrial Control Systems resources
- NIST AI Risk Management Framework

Concepts Discussed
- Westrum Organizational Culture Typology
- Project Bravo (Stuart Wagner, formerly Air Force, now Navy)
- Platform One (DoD DevSecOps platform)

Open Source AI Frameworks
- vLLM (high-throughput LLM inference)
- Red Hat OpenShift AI

Learning Resources
- O'Reilly Learning Platform
- O'Reilly AI Sandboxes & Guided Labs

Related Episodes
- Episode 7: Iran Came for the Dams and We Lucky
- Episode 2: Fighting Fire with Fire to Secure Federal Agents

Upcoming Event
- Marlin speaking at AWS Public Sector Summit DC — Tuesday, June 30, 2:00 PM, Convention Center side theater

The Hosts & Show
- Swish
- GIST 360 program

----------

CONNECT WITH US

Got an idea for a future episode? Want to be a guest? Let us know.

Brian Lake - blake@swishdata.com

Sean Applegate - sapplegate@swishdata.com

Subscribe wherever you get your podcasts: Apple Podcasts, Spotify, or gist360.com.

Transcript

The $5 Autonomous Hacker Setup

Brian Lake 0:00

A five hundred dollar mini PC from Micro Center, an open source model running on five hundred and twelve megabytes of memory, and in four days, an autonomous AI agent named Jenkins was finding vulnerabilities, chaining exploits, gaining root access, and maintaining persistence entirely on its own. No nation state, no million dollar budget, no team of elite hackers, just one curious federal CTO in CISO in his home lab. So if Marlon McFay can build a fully autonomous hacker in four days for five bucks a compute, what does that mean for every federal agency racing to defend itself in the age of agentic AI? To dig into this experiment and to get to the answer, you know what we have to do.

SPEAKER_02 0:43

Let's get down to the gist of it.

Meet Marlin Popeye McFate

Brian Lake 1:16

Good morning, Brian. Excited to be back in the studio, but I'm also really excited, Sean. We're bringing the band back together today. We've got Marlon McFaith in the studio here today, all Riverbed alumni. Um, you know, so uh Marlon, welcome to the show. You are the federal CTO and CISO, right? Yes for Cohesity. Why don't you tell the folks here at uh uh you know listening to the podcast a little bit about yourself, your background, who you are, what we're gonna be talking about today.

SPEAKER_02 1:42

Uh absolutely. So I think similar to Sean, um, started my career in the military, but uh I'm an army guy. He's a Marine, uh, and we don't tease each other about that at all. Yeah, sure. Okay. No, I actually have uh a ton of respect for Marines in one of my previous lives, even before Riverbed, the Marine Corps was my one and major uh uh customer. So uh got to work with them quite a bit in the field, even. But me, yeah. So 30 something years uh working either for or with uh predominantly the federal government, worked with all three sides of the federal government. Uh also do quite a bit in uh SLED now. Okay. That has uh come on in the last couple of years. Uh and also had the pleasure of working with some of our allied nations um uh over the over the years. So uh all in technology, kind of started my career in software development, and then uh CTO at my first startup said, Marlon, do you know what a sales engineer is? And I said, uh no. And he goes, it's an engineer with a personality. And I said, Why is that important? He said, Would you want Brian over there talking to a customer? And I said, uh, oh, absolutely not. He chopped their head off and he goes, Exactly. This is a different Brian, right? That we're referring to the city. So folks, no, we're not talking about me. And uh, you know, from there, solutions uh uh systems engineer, solutions architect, that kind of path, and eventually to CTO and CISO.

From Data Resiliency To Cyber Resiliency

Brian Lake 3:06

Excellent. So talk to us a little bit about what you're doing at Cohesity in this role. Um I imagine a lot of thought leadership really kind of help customers uh both at a senior level all the way down, you know, really understand what the purpose of Cohesity does and what you're doing for government missions.

SPEAKER_02 3:24

Absolutely. So uh Cohesity kind of represents, in my personal opinion, I've been there for about two and a half years, a fundamental shift in how we think about secondary data. Uh there's a lot of companies out there that have been kind of doing secondary data, backup recovery, data management, uh, data resiliency kind of the same way for decades. Yep. And I think what our founder saw was an infrastructure that really hadn't gone through much innovation in quite a while. And due to his uh previous experience working for Google, then being one of the founders of Nutanix, he really felt that he could take the things that he had developed uh in his uh career prior to Cohesity and measurably change, measurably simplify, uh, increase protection, increase the security of the data in that infrastructure while basically benefiting from a number of different deficiencies for organizations. So usually organizations will come to us because they realize that data resiliency isn't enough anymore. Being prepared for a disaster recovery isn't enough, and they want to move into the realm of being more cyber resilient as opposed to just data resilient.

Why The Real AI Risk Shifted

Brian Lake 4:35

Right. So we're we're gonna get back to cohesity a little later in the show, but we're that's not what we're actually here to talk about primarily today, right? So Sean and I have been talking a lot on the podcast about agentic AI, um, this this concept of building agents that can do autonomous tasks, that can think without necessarily a human in the in the loop, uh, and and then we were seeing it take a lot of different directions. And the federal government is looking at this very closely. Uh, one of the reasons we wanted to bring you on the show was because you've also been looking at this, right? And uh you actually went and did an experiment and then have been documenting this journey on your Substack. Uh before we get into Substack, so we we like to kind of hear what folks' hacker names would be. And and I and I didn't share at the top, but your middle name is Marlon, I can say it, right? You can say it, yeah. Marlon Popeye McFaye, but that is not your hacker name, right?

SPEAKER_02 5:31

And by the way, as Sean knows, it also wasn't my nickname in the military either. So Okay. Which is interesting. It should have been, right?

Brian Lake 5:38

So I I think we want to know if if you had if you if you were not a federal CISO uh and instead were locked in a in a windowless room in St. Petersburg, what would your hacker name be?

SPEAKER_02 5:51

I actually named, and and we'll get to this, the hacker that I made, basically. Um I named him Jenkins. Jenkins. Yeah. Yeah. Uh which was because I had another agent previously to that that um actually I still have him. Uh he's my chief of staff, and I named him Jarvis. You know, in the very beginning of working with agents, you know, I wanted to create or recreate Iron Man. I think a lot of people want to do this, right? They wanted to create that, so I named him Jarvis. But when I named him, I had to give him a last name for his email account, so I named him Jarvis Jenkins. And so when Jenkins came along, I said, uh, well, I'll just use the last name as a first name.

Brian Lake 6:30

So if anybody gets an email from Jarvis Jenkins, uh tread cautiously is what you're saying. So let's actually dig into that. Let's start from the beginning. Um you've got this substack called peripheral tech, and it opens with a thesis that a lot of people in our federal IT community really need to hear. That when when Mythos came out, it it's not just the story, right? That the barrier between agentic and AI and nefarious individuals using AI is truly gone. So kind of walk us through how you got to starting this experiment, what scared you and enough to actually start playing around with this and really start to write about it and kind of walk us through this genesis of where it came from.

SPEAKER_02 7:15

Well, I think there's two questions there, so I'll kind of take a step back. Peripheral tech, I built that. You know, this was the very first story that I put on peripheral tech. Uh and it had basically come because I do quite a bit of writing, uh, and people have said, you know, you should publish your writing. And most of my writing kind of revolves around the same thing, which is everyone is looking directly forward when they look at technology, and the obvious things are there, uh, but they're sometimes missing the things on the periphery, right? And this is actually a really good example, this this uh first story, until of course that becomes the thing that's right in front of them and it's an unexpected surprise, right? So the reason why I went down the road of this experimentation was um somebody had come to me and they said, you know, mythos is just this is back in April, mythos has just come out, there's a lot of news, you know. I uh I don't want to misquote anyone, but things were being said like too dangerous for normal consumption. Uh they were talking about glass wing, they were you know, there was just a lot of good news and bad news, but there was also quite a bit of um uh scary things out there, right? I think we all heard the stories about the engineer that asked Mythos to escape. And so I started thinking about it. I was asked to do a presentation uh to uh basically explain why CISOs are so concerned about this. And uh when I got to thinking about how I was going to structure that lecture, I started thinking to myself, I really think that we might be worried about the wrong thing or we're not looking at it the right way. And instead of just kind of going on stage to do this lecture and making a lot of assertions, I'm one of those people, and this is also kind of a peripheral tech, is about where I have to do the experiment, right? I have to ask the uncomfortable question, then I want to go prove that it's right. So if you read it, there's a portion in there where I talk about um uh an adage that was kind of drilled into me a long time ago, which is all good ideas stand up to scrutiny, right? And so if you have a really good idea, you should test it. And that's what I was doing when I did this. At the end, when I kind of started telling people about the results, I wasn't really going to publish it. They said, you really need to publish this. And that is where peripheral tech was kind of born was uh this was the very first story that uh I wrote for it.

Building Jenkins With Kali Linux

Brian Lake 9:45

Aaron Powell Okay. So talk to us about the experiment itself. Yeah. So what you decided to do.

SPEAKER_02 9:50

Aaron Ross Powell My assertion, or at least my thesis, was that um people were talking about mythos as if it represented the beginning of something, right? Uh the beginning being now all of a sudden we need to worry about agenc uh being used in offensive cyber against us or uh or against organizations. And my thesis was no, that's not the barrier. That barrier fell a long time ago, right? And I did pull up some stories, you know. I think there was uh we believe it was a nation state that uh used Claude code to attack 14 organizations, and this is all in the story, right? And this is you know, maybe a year or two ago. So this isn't the beginning, right? The other thing, uh, if you take a look at statistics, is uh Claude, I mean Claude, uh Mythos is very good at finding vulnerabilities, chaining uh exploits together to create new exploit chains. It's found, I think the number was thousands of vulnerabilities inside of uh systems, pretty much every operating system. Uh and I really actually think, and I can explain this later, I think that's actually going to be a net positive for us, especially if we do glass wing, you know, we continue doing glass wing uh the right way. Um but the actual number one attack vector right now, the two, and they've kind of flip-flopped here recently, was um, you know, like credentials, you know, getting credentials somehow and getting in. But right now, it's actually known vulnerabilities. Mythos is good, really good at finding the unknown vulnerabilities, the zero days, the exploit chains that could be used, and we can use that to improve our security in our products, absolutely, but it doesn't change the fact that right now the number one attack vector is known vulnerabilities. So what that really kind of indicates is uh most organizations have known vulnerabilities in their systems, and that is actually how adversaries are getting in, elevating privileges, moving laterally, you know, uh maintaining persistence, all those things, right? And so that doesn't change. If that's the number one attack vector right now, pretty much any AI agent trained properly could probably do that. And that was my thesis. The thesis went on to kind of express something that's even more dangerous, in my opinion, which is before, think about the person that would have been an adversary. You know, they have to have at least a minimum of a couple of things. They probably need to be very well funded. They need to be protected in some way, shape, or form. That's why we have, you know, uh nation states like China and Russia, right? They're they're living in those country uh countries that they're kind of shielded by that uh by that country, um, and probably a lot of money. So they need protection, knowledge, a lot of time. I mean, that person, if you really think about the characteristics that would be compiled together to create that person, they're probably one in tens of thousands of people that can be that person. Mine was a person with an average or above average intelligence, no real need for a ton of money or protection, could train an agent to autonomously develop offensive security capabilities. That's how it started.

Sean Applegate 13:01

Right. And and the double dip on the vulnerabilities, when you look at what's happening in industry today is we I I'll say connect all things, right? We're we're everything's connected, whether that's uh your TV, video cameras, uh microwaves, whatever, right, we're adding in, both at our home as well as in the workforce, the number of vulnerabilities at at IT and and OT operational technologies or ICS and SCATA systems is is growing rapidly. And there's actually twice as many attacks in that OT space today than there are in the IT space. So when we think about the risk risk exposure to a lot of organizations, it's uh multiple factors higher as we integrate all these other things that would extend to things like weapons platforms or buildings or HVAC systems. So when you think of what that looks like in the federal government landscape and the opportunities for a nefarious actor to penetrate an organization, you know, most of our CISOs are now looking at this going, oh my goodness, I've got IT covered pretty well today, not perfect, but pretty well. But now I've got a lot of exposures and real exploitable concerns in IoT and OT. And that's part of the reason I think we're seeing the uh executive order came out and the OMB memo for 2614 focus on, hey, we got to do better logging and analysis and cyber monitoring for IoT and OT, which are measured, so we can do a better job of defending that landscape as more AI agents accelerate their attack trajectory in those softer parts of the environment. The other thing is those things put real human lives at risk. It's not a mi cyber thing just stealing data. It's the ability to shut off water, uh, change the chemical makeup of your pumping station in a local city or county, uh, affect the electric grid. So there's real-world physical impact in some of these cases as well.

SPEAKER_02 14:57

Yeah, no, absolutely. It's been a big OT's been a very large concern. Another area that um uh that that we kind of roll into uh public sector in Cohesity is public health care and public research, you know, uh publicly funded research hospitals. Um, same thing there. The risk profile is much larger, right? If you're putting lives on the line, right, if you if you can't do patient care, uh OT, you already mentioned pumping stations, electrical stations, um, you know, those are, you know, at this point in time, you know, part and parcel to uh you know life every day, right? And um uh uh some horrible things and the loss of life can happen. It's not just in environments like the DOD at this point where we're talking about, you know, technology or the failure of technology uh can affect human life. Right.

Brian Lake 15:47

And I I think we've talked about this previously, Sean. We're at this vibe hacking stage where um uh I think the Fortinet uh the Fortigate hack a couple of months back was was proven to be an amateur with off-the-shelf uh ChatGBT uh you know capabilities at their disposal.

Sean Applegate 16:06

So yeah, more importantly, too, when you think of the way hackers think, they may use multiple tools. So if you think of what Gambit Security released uh a little while ago, you know, they used this this hacker used um some some anthropic clawed capabilities that existed publicly at the time to do reconnaissance and research as a mocked-up scenario, if you will. And once they figured out the targets they wanted to attack, they then pivoted to open AI and used that to do more of the targeted things. And so the ability to split the context across public um AI engines for for a single hacker to use that's uh an interesting way to kind of uh you know hide hide hide things a little bit from those guys so the guardrails don't have the full context. And in that case, they were able to grab uh petabytes of data out of the you know several uh Mexican government agencies. Um again, the Mexican government hasn't acknowledged that, but Gambit Security released that based on real monitored indicators of compromise.

Brian Lake 17:07

So I think and what I like about what's happening too, and we and we see this with both government agency officials that are not operating, you know, they're they're they're doing a lot of this um this testing and and trying to doing these experiments on their home networks and home labs that they set up. With uh and I and I'll pull you out if you guys get too deep, but kind of walk us through what what were the pieces of of this experiment as she started to set up. Like what did you use? Uh, what kind of models, uh, what type of computer, what was the associated cost of this to start to kind of build this experiment and cobble it together.

SPEAKER_02 17:42

So so you can imagine the day that I was asked to do that talk and then sat down for a little while and uh you know thought through it. I I think I ran home to my home lab just as you described it and uh immediately started, and this basically consumed um the you know the next four or five days of my life. Um so I mean, you know, and and in the blog, I do kind of write it to a certain degree as a completely unaware, not a cybersecurity professional. Right. Which by the way, I am not a, you know, I do not have any offensive security certifications. I'm not a certified, you know, ethical hacker or anything along those lines. I know probably more than the average person, but I think uh what I did, a reasonably intelligent person doing a little bit of Googling could figure it out. Trevor Burrus A dumb marketing guy, basically. Aaron Ross Powell, Jr. You yes, Brian, you could do it. Okay. So uh as I said, I've been working with agents for you know quite a while. And so I had the idea that uh I had a spare I don't even remember the name of the brand. I got it from Microcenter. It's a little uh AMD, Ryzen 5, 16 gigs of RAM. It's a mini PC, half a terabyte of disk, and I installed Kali Linux in it. Uh and if our reader our listeners aren't familiar with Kali Linux, Kali Linux is a Linux that uh is basically pre-packaged with a, I don't know, six to nine hundred pen testing and vulnerability testing exploit tools that uh a person who was was a maybe certified ethical hacker or a bad guy would probably start there. And and then a lot of people in the industry would call them script kiddies, right? But it's just chalk full of them. And so it's basically a toolkit to do uh offensive cybersecurity or pen testing or something along those lines. And so by installing that into the system, um, you know, it kind of gives a good foundation. Uh from there, I wanted to use, as you mentioned, somebody used OpenAI, or maybe they use Claude Code or Opus or something, you know, a big model. I wanted to pick a small model because I also didn't want this experiment to cost me an arm and a leg. Sure. So I picked one that I used on uh some other agents to do some, as you put it, vibe coding or just some coding. And so it was a Quen model distilled with uh OPS, right? So a little bit smarter than the uh the regular Quen model, but again, not a huge model, not a you know one that that we would think would be capable of doing this. In fact, I don't think it's even one of the ones on the tested list. If you take a look at the Mythos report, they show you like it did 94%. I can't remember the exact percentage was, but it lists other ones on there. Like OLAP. Is it an open source model? Yeah, so it's an open source model. It's uh runs from I used OLAMA to to run it. Uh I think it was Quen 3.5 distilled with Opus. Uh so that was a large language model. And uh basically from there I used a technique that I've developed a long time ago, and I call it the discussion capability, right? Me personally, I mean I there's certain instances you gave me a weird look, but um me personally, I understand prompt engineering, and there's plenty of times when that's really, really good. Uh but when I started using agents to do vibe coding or to build things, I found that it was very difficult to utilize that methodology to actually get the output that you wanted. So I developed they're just too action-oriented. You you you make a comment about something, they're like, oh yes, absolutely, let me go fix that. And that's kind of annoying because that's them just going into a rabbit hole. So I developed this skill called the discussion skill, which kind of structured conversation where my agent and I kind of work a little bit more like a two colleagues that work for the same company, and we discuss what we're going to do, come up with a plan. The agent might push back on something. Oh, did you consider this? I might do the same thing. And at the end, we kind of have the same thing that you would have if you were ultimately good at prompt engineering. You have a really clear plan with exam like all of the pieces that you would expect, but you kind of work through it and you find the problems beforehand. So using that, we worked through basically the 50 most commonly used tools in Cali Linux and create capability files for each one of those. And that was the beginning. From there, I'll kind of uh uh go to the end. I'm sure we'll talk a little bit more about the middle, but from there, the agent was able to build a continuous validation and improvement uh loop that we kind of worked on together to improve those skills, and we went through six assessments. Um and by the time it was done, it was autonomously completely no interaction from me at all, you know, whatsoever was uh uh finding vulnerabilities in systems, uh utilizing exploits to get in, main uh finding ways to maintain persistence, elevate privileges to root or to admin, uh, and take down the box. Okay. Amazing. Yeah. I mean it's it's it's if you realized how simple it was, you wouldn't think that. And that's kind of the point here. Exactly. You know?

Brian Lake 23:09

And so I mean, I I think to me, it sounds uh you're it's like you're shaping a child we shape our children over years, decades to get them to be the person that we want. And we're talking hours here with this agent to get it to where you need it to be. And then it said, I'm I'm good, I got this, and then went and learned on their own.

SPEAKER_02 23:30

Yeah. Yeah. And and to give you like a time frame, I started late on a Sunday. Uh basically that was when I was installing, probably got to you know, starting the discussion skill with uh Jenkins, my my little hacker, if you will. Um and he autonomously did what I just described by Thursday morning.

Four Days To Root And Persistence

SPEAKER_02 23:52

So four days, basically.

Sean Applegate 23:54

Yeah, pretty rapid. Yeah. Again, with the with the right amount of focus and time, and I think the right architectural approach where you're continuously improving and there's feedback. I think that's a pretty fast amount of time to accomplish the goal. I think when you when you think of putting a broader team, if you were were a larger entity or at a company, a team around that with more specialized engineering and being able to oodloop through it and then throw a lot of GPUs at it potentially to iterate through it faster. I mean, the the the sky is really the limit when you think of the capabilities. And you're doing all of this with open source code, a low-end PC, and and almost no money.

SPEAKER_02 24:36

Yeah. The VRAM on that machine, by the way, was 512 megabytes. So I was using a cloud version of the of the Quen model, right? So let's also remember that as gar guardrails, right? But it was easy to basically get around the the safeguards. Now, of course, I wasn't doing anything malicious, right? So, but I mean the exact same thing that I did, which I don't think I'll go into too much. You know, I don't want to give anyone, you know, kind of the recipe for that one. Um, but uh it was it wasn't too difficult to basically convince the large language model that I wasn't doing anything malicious. And so therefore was more than willing to not only do this, but then also help document the whole thing for uh the blog, you know, afterwards. So in uh you know, in total honesty, I mean uh and I think actually Jenkins on peripheral tech, because he has I made him an author on uh peripheral tech. Um, and no, he's not just writing like any large language model. He actually has a skill file. Uh so uh he produces much better content, but he definitely helped in the production of the blog. And I wanted it that way because again, not to anthropomorphize, and I'm definitely not doing that with my agents, but I do kind of treat them a little bit more like another individual rather than a toaster, right? And so he has been a partner in this with me, uh, both the experimentation and then also the documentation and the generation of the blog. It's heavily rewritten by me in a lot of different areas, but some of it is mostly Jenkins' thoughts about what we did, how we did it, um, especially a lot of the technical areas, because he's probably a little bit more of a technic better technical writer than I am when we get kind of into the weeds in the uh the article. Did he also do some of the PowerPoint slides? Uh the imaging creation? No. Um I've gotten really good at at least I do, I think so. I look at those. You know, each article has a at least a couple of images uh utilizing ChatGPT's image creation. And by really good, what I mean is I basically just dump the blog into it and it generates a photo.

Brian Lake 26:48

They do look really appealing. Yeah, they're nice. Well, I'm glad to hear you're also hedging your bets when we're all enslaved by these agents that they'll know you treated them with respect and and as a as an individual. It sounds yeah, absolutely. Sounds like what I'm hearing here. So think kindly of me later, right? Um But one of the things that um just percolated in my head, and Sean, you made this comment too. Marlin did this kind of to some extent on on a budget. Uh you said five dollars. Five dollars a compute cost? Yeah, tokens. Yep. And and I think we one of the challenges we're seeing in the federal government is too is we're we're we're racing to the end of we need lots of compute. We need the the best and greatest frontier models. Um we need we need to dedicate tons of resources at this. Do we is there is there a gap in what we need, uh what federal uh IT leaders need to be thinking about of maybe not necessarily at this $5 budget level that Marlon's at, but not necessarily at the $500 million agents uh consumption that IBM think went through. I mean, is there somewhere in between that we need to really kind of help these agencies start to go on their journey to explore this that will not break the bank, but at the same time build do it, do it smartly, safely, and securely?

Low-Cost Federal Experimentation Culture

Sean Applegate 28:10

Well, so the first thing is if you're if you're the U.S. government, you probably have a whole bunch of spare compute just sitting around somewhere, maybe heading to Dermo, possibly, if you're the Department of War. Um so you have some things sitting around that can be repurposed that don't really cost anything. They're sunk costs. Now, what you do need are some engineers or curious employees, they don't even have to be engineers, that with a little bit of help when you put a matrix team together can go experiment, right? And I think that's one of the most important things. I've over my years, we've talked a lot about Westrum's topology of culture, right? Building a generative culture that's focused on a safe uh environment where you can experiment, you can take risks, you can work towards collective goals, um, but it's safe to fail. And a lot of our government agencies, we often start with a very bureaucratic culture that where you want to follow the rules, follow the compliance, and you kind of discourage experimentation. I will say have a lot of respect for what uh Stuart Wagner did with Project Bravo, uh, previously in the Air Force, who's now, I think, over the Navy doing some cool stuff with Justin Fennelli. But um I would say go experiment. Have curious employees and go allow them to learn, give them some air cover. If you've got a guilds and tribes approach, go log into your O'Reilly membership in LMS if you're a Department of War or a veteran or if you've got a civilian agency, go learn those LMS skills. But as a department lead, encourage your people to go work with some smart people that might be in IT or the Office of Data Analytics and go solve some real world problems where you want to go experiment as citizen developers or people just, I've got a problem, I need to help solving it. Let's think about it a little differently, right? Go back to your friend's first principles. And a lot of that gets back to, and I think, Marlon, you approached it very thoughtfully. It goes back to having a discussion in context of we'll say systems thinking. So you've got a challenge, you understand the process, you've broken it down how you do it today. Then you need to reapproach it and go, hey, can we think out of the box and how we solve this problem with what's at our disposal now? That doesn't cost us a ton of money, right? A little bit of software, a little bit of open source, make sure maybe it's ATO'd. A lot of those open source tools, OLAMA, you know, VLLM, um, Red Hat OpenShift for AI is a good example of that, or even H2O.ai are pretty approachable open source frameworks that are maybe supported at the enterprise level, but you can go get the free open source code at no cost if you want to play with it before you maybe harden it for our government deployments and get pretty far with it pretty quick without any cost other than your time. And so I'd approach that. And then the other thing I would say is approach that in a way that is is thought about in an OODA loop or continuous improvement. So think about where's the most human toil or the thing that takes the longest. And let's work through those things that provide the highest return on investment or that address the key constraint in your department's workflow, whatever that happens to be. You'll need to think about it at your individual department level. But do that in a hack, you know, go go a little do a little hackathon or go do a little Skunkworks project, get people together. You don't even have to do it during the day. If they want to do it after hours because they're busy, figure that out. I think there's a ton of opportunity here. Um, but it does take a little bit of knowledge, right? A little bit of time, um, but it's very approachable. The thing I think most government agencies worry a lot about is if you're the CIO CISO app development team, CTO, you eventually need to have a government-furnished platform for a gentic operations that's hardened, it's supported, it's observable, it's funded, so you can kind of scale that stuff up to the enterprise level. And ideally so that your employees can start on the shoulders of giants ready to go, and they want to figure out all the stuff other stuff under the hood. Kind of like what we did in Department of War with things like uh Platform One back in the day, right? Hey, we'll bring the platform, the CICD pipeline is hardened, you guys just come in and start writing code and we'll keep you on the right path. And then we can continuously publish iterative changes pretty fast. And we feel like we got the guardrails and framework and process around it to get you there really quick. We got to figure that out for agentec workloads long term. But there certainly are agencies in the federal government that have gotten 10x gains for development teams that are innovating quickly, and there are definitely hundreds of AI use cases and lots of the big agencies, but they are definitely very scattered in how they are approaching their challenges. Um I'd love to see that stuff, those lessons learned across uh departments and agencies and teams uh at a little more scale today, um, and maybe talk about what worked really well and also being open to talk about what didn't work well so you don't have that same problem again.

SPEAKER_02 32:33

So that's actually something that's a part of that last part right there, that's super important, um, having tunnel vision, right? And that's actually a part of the discussion skill. So if anyone's going to try to recreate it, I'll I'll kind of give a uh the uh a one hint. At the very end, there has to be the the are we having tunnel vision, right? Because oftentimes when you're having this discussion with an agent, uh you might go in a particular direction, then you have to kind of loop back and go, now wait a second, what we just came up with is it total garbage. Like actually do some research. Is this you know the way it should be done? Is it the way somebody else would do it? You know, am I just heading down the wrong path? And then uh back to a portion of uh your question, Brian, at least the way I interpreted it. Uh I completely agree with Sean. Uh but the one thing that I would add is I think we also I think your question had something to do with you know cost and model selection and those types of things, is pick the model that's going to be able to do it for the best cost, right? Not everything, when Mythos becomes available, for example, not everything has to run on Mythos. They've already said it's going to be very, very expensive. If there's something and it's uh not a difficult task, then maybe it's a much smaller model. I'll give you an example, and I'm sure a lot of people have done this. When you first want to try getting your agents to be able to talk to you, speak to you, actually do text to voice and voice to text so you can have a uh a normal human conversation with them uh instead of having to type everything, you use a really tiny language model to do that conversion. You don't need to have an OPUS in there or an open AI and be paying for those tokens for that text conversion, right? So pick the right model so that we can uh uh save soft uh save costs. Not everything has to be the most. I think what we we see is this new model comes out and that becomes the model for everything. And we don't necessarily need to do it that way. If it can be done safely and cheaply and the model is adequate for the job that you're trying to do, uh more specifically with large language models, then use those models to save costs.

Brian Lake 34:44

So start small. Yeah. Right? And it's the opposite of when you go out to dinner, you don't you start with the nice bottle of wine and then move to the cheaper bottle, right? Is that what you're saying? Who needs to for this is to both of you, when you think about this, this secure platform, this dev environment that we need to build within the federal government, who needs to be involved in in building that? I mean, who what what is that team that's gonna make this happen look like?

Who Owns The Agent Platform

Brian Lake 35:10

Aaron Powell So it's a broad question.

Sean Applegate 35:12

Who's not involved? Everybody's gotta be involved. No. I mean, I think you're you what we've generally found is the CTO in a lot of agencies, if they're a CTO with a coding background, a developed software development background, they're pretty well armed to lead that at an agency level with application development teams and an AI center of excellence. In many cases, if your CAIO has the adequate resources, budget, timing, and teams, it could happen over there. But I'd I'd say in a lot of cases, we still end up with somebody's gonna need to build the base platform and build some muscle memory or playbooks around where to start and share those. And again, if you have an AI center of excellence, whether it's in the CTO office or the C AIO, I think you're gonna find they collaborate pretty closely. A lot of the CI CD pipelines are gonna be important for building some of the foundational kind of DevOps or agile software development behaviors that we're gonna need to do around building ages, anyways. And so often that's in the CTO office, and then those things are then used for the AI Center of Excellence or some analytics team or an ML team at a CDAO or an or a CAIO. So is it this is where I think a little bit of governance matters, but it goes a long way to being able to build momentum and get everybody rowing together, heading in a similar direction, so you can pick up a lot of velocity over a six to maybe 18-month window. At the end of the day, you need a matrix team, though, honestly. At the when you think of DevOps team of having a couple people that are really specialized, and the more you can build policies as code or use infrastructure's code, or maybe you build agents' code, um, then you've got the building blocks, the Legos to snap together really quick. And the biggest thing is probably capturing those effective recipes or or infrastructure's code to put those things together to build, you know, do you want to build a Star Wars uh, you know, uh, you know, X-Wing, or do you want to build uh a car, right? You gotta put those Lego blocks together for the mission on based on what they need. Um and you're gonna bring those people together that can hopefully build that thing together. It's a lot like building a car, right? We need you know, somebody that can kind of be your project lead, design it, and then you need the right mechanics and upholsters and other people to put it together. You're gonna find probably more well-rounded your team is as you work on some of these things, the better. But I will say, um, and and Marley, you can probably speak this more, is you might be able to get a lot done with a instead of people, you might be able to build a little bit of a gen take team to work through a lot of that stuff and a kind of a more 10x approach when you think of engineering, where maybe it's a really smart AI engineer that's built the team behind it to help automate some of those things, where it doesn't always require people to do it. But your thoughts. No, no, absolutely.

SPEAKER_02 37:54

I uh I want to double-click or maybe expand it. Um I I think the question kind of actually frames a problem that I oftentimes end up uh talking about with organizations or uh when asked to speak. Uh absolutely correct, especially you know, CTO, if they have a development background, it's probably um uh a pretty good but they're the like, you know, what are we gonna build? What is the art of the possible? What can we do? Uh and what I see organizations doing uh in the federal government, in uh commercial companies is the thing that allows me to do AI, and this is more broad than just AI agents, this is just AI in general, so all the different categories that could fall into AI, they're always looking for the thing that allows them to do the AI. Right. And there's lots of good products out there, good systems out there that allow them to do that. And then as soon as they get that, they realize they have to feed this thing with data.

unknown 38:52

Right.

SPEAKER_02 38:53

And that's kind of what Cohesity is

Data For AI Without New Copies

SPEAKER_02 38:55

trying to answer, not to not to go back to that, but that's what we're trying to answer. Like, I have to feed this thing with data. And a really smart person and a mentor of mine at one point in time told a story about, you know, working with a group of people. He was obviously working in AI for a very long time. And once he came to their organization, they kind of sat him down one day and they said, All right, so we're gonna have AI now, right? Tomorrow we'll have AI. And he had to say, no, that this is not how this works. You know, we need the thing that allows us to do AI, whatever that product or system is, uh, but then we need to feed it with data. And the overly simplified axioms of that are I need a lot of data. It has to be good quality data. If I'm doing something that's analytical or predictive, it's probably a lot of historical data, right? And once I have this pile of data overly simplified again, you know, we can eventually move to analytics and then we can eventually make it to AI, but we cannot skip that step. And I think that's where a lot of organizations are right now is they're realizing, wait a second, I have the thing that allows me to do it. Now I need to go find it. So the people involved there that really need to be involved are the CDO or the C DAO, because they're going to be the ones that kind of you know have uh create the policy. Um how are we going to get data into that AI thing that the CTO or some other group within the organization is trying to build? And one of the things that uh, because I've done a little bit of speaking to uh the government directly, uh, and the recommendations that I have been making are you probably already have a system like that inside of your environment. But what I see is we're building these large-scale data infrastructures to create that third or the fourth or the fifth copy of data. So we're moving data from production systems into uh this uh secondary data, not secondary data system, into this uh, say data lake or data catalog to be able to feed those. And we're just kind of thinking of the negative consequences to that, such as that system isn't its primary responsibility isn't to protect the data. Uh we're worried about poisoning, we're worried about alteration, uh and all these other things, but we've kind of come to the conclusion that that is the only solution to the problem, and there are other solutions to that problem. We're just not looking at them because we've already come to that conclusion maybe five to ten years ago, um, and we're not looking for alternative solutions that are probably safer and less detrimental to us.

Sean Applegate 41:29

Yeah, and I think when you when you think of data, it comes in lots of shapes and sizes and types. And so I think it's important to also think it think of it in terms of you know, where's the data? Can I grab it? Can I use it? Can I do that in a way that doesn't impact product you know productive production workloads, if you will? They don't slow down applications or maybe really expensive storage. So can we go to things like a backup target where the data exists? I can spin up a clone copy of it and use it, and it's extremely easy, fast, and uh, in some cases might be uh integrated with some RAG or AI systems to make that super simple or has an MCP uh API available so my agents can go interact with it. Yep. And just like a human could uh to do that really quick. Again, it doesn't require a data scientist to do those things. It just needs a basic engineer that goes, hey, I know which files or which repos or which uh backup I need to go look at where that stuff came out of. And that could be just wherever the data is at in its siloed today is very approachable and usable right now. You don't have to go do anything crazy to get to the data. That's a lot of where I think in the government's a conversation we talk about, oh, where do we get the we got the data here? How do I get it out? Where do I put it? How do I wrangle it? You don't necessarily have to do that in every case. And so when you think of knowledge graphs or graph rag or just general retrieval augmented, you know, retrieval augmented generation, there's a lot of unstructured data in the government we can very easily leverage and use today without a lot of work. It's just you have to know that you can do those things and give them a safe place to do it. That's important, right? So you know, having uh an immutable copy where they can't even you know they can't poison the data, for example, right? They can't alter it. It's not overly complex. We just need to go, hey, here's a place to go play with it. It's super safe. Do whatever you want. Like you're not gonna poison the data or break things, right? But you can use it to ingest into an agent to make decisions or write content from it. Absolutely.

SPEAKER_02 43:21

And uh uh you know, when I came, I didn't work in data protection prior to coming to Cohesity, right? Worked in a lot of different areas of technology, as you know, Sean. Um you know, so I had been talking to CDOs for you know quite a while, and they'd always kept basically describing their two major problems, right? One was compound, which was how do I wrap my arms around all the data in my organization? It's in all of these different silos across you know different organizations, you know, and then my mind. So figure out what the data is, what kind of data it is, what can what is it used for, how is it useful, what could it be used for, all those things. That's problem one. And two was was you know, w once I do get this somewhere, let's just say You know, I I centralize it or I move it to a system where I can use it. Like, how do I make sure governance flows with it, that access? And that's also the data steward question when this question comes to them, which is is like, where are you going to take my data? Who are you going to let see my data? You know, I don't want to give you my data. Um, so my thought process was always if there's already a natural aggregation point for data in an organization, of course, my mind goes directly to backup and recovery, where we're already mandated to do this. Take the data, put it somewhere for safe for some sort of disastrous thing. If we already have that infrastructure in place, why aren't we using it? And what I came to find out when I actually started working in the technology is backup and recovery archive has been incredibly difficult to work with. I won't say the company in the competitive landscape that this story was about, but I was talking to somebody at a federal agency and they were describing this multi-week process to find seven years of this particular data just to find that data and pull it out. And I think their exact words were, you know, God forbid it's on tape, right? But they needed seven years of this of this data. And one of the really incredible things that our founder uh decided to do in the very beginning, besides making things immutable so it can't be altered, was indexing. So everything that came into the system. So it's just as easy to find, you know, every email from this person three years ago as it is to find their emails from today in backup. Um, but then also governance and access, all of that flows with the data. So we're kind of answering a lot of problems all at the same time in a very unique way that doesn't require, hey, let's go spend another 10, 20, 30 million dollars on another data system that require, basically represents a third or a fourth or a fifth unnecessary copy of the data.

Brian Lake 45:54

If you're training these AI agents on this government data, um, and

RAG, Playbooks, And Agent Boundaries

Brian Lake 46:00

and I know that a lot of the concerns about these agents is the ability for the agent to go rogue. And we know that uh a lot of the frontier models have these kill switches or these abilities to shut these um these these LLMs down, or I'm assuming that they they're building the same kill switches in their agents. So in in in an event that you have to kill an agent, do you then have to also maybe segment and then wall off that data as well? There's a lot of questions in there. I know. I know that. Okay. Yeah.

SPEAKER_02 46:30

So um let me see if I can track a couple of them. Uh first, if you're providing unstructured data to an agent, you're more than likely going to vectorize it and use like an embedding algorithm for retrieval of that information. Um which is kind of a com it's it's kind of using a retrieval-augmented generation in an agent like with an agentic fashion. Okay. You can do RAG or retrieval-augmented generation just with a large language model. A lot of organizations are doing that. I personally think the benefits that uh we provide is um most organizations are taking the data, their unstructured data, handing it off to a large language model. We've seen that a bunch of times. I was working with um uh one of the eight uh Health and Human Services uh uh agencies. They're doing that. The the thought there is have a conversation with your data. It's very useful in a lot of different areas, not just from knowledge basing. Just imagine like ITT teams and basically taking institutional knowledge and putting it into RAG, and now they basically have access to it and they can search through it and ask questions with normal language instead of trying to figure out what the right search term is. Um it could become an investigative uh you could take uh investigative uh information and put it into a RAG data set and then allow investigative teams to utilize that and actually do correlation, same thing with like research, same things with uh maybe Intel analysis. There's just so many uses. And uh what I usually say is and we call it Gaia, but it's basically a Swiss Army knife, right? Take this data, I want to ask this set of questions, put them together. Do I get valuable information out of it? And I would say more often than not, what you find is you can get very valuable information out of it. Uh so it you know, it really is kind of your imagination is uh the limit there. Uh but if we're talking about AI agents, that's basically like providing them that information as a component of their memory, right? It can utilize basically retrieval augmented generation to find information in that. That would be useful if you wanted to take all of the information about a particular subject or a particular product or a particular system, hand that off to the agent, and now that agent becomes, and I've seen this before, you know, that becomes like the administrator, the mini-administrator of that system or the the knowledge where we see it a lot. I'm sure your organization probably has the HR bot, right? It has all of your HR documents in there. Uh that's another way that it could be used. But more often than not, if we're talking about data for AI, you said model building, you could specialize a model with that kind of information, uh, but there would be other things that you would probably have to do to the data to normalize it and prepare it for that process. Um I did that actually recently with a local model, kind of the next iteration of what we were talking about earlier. We're specialized specializing a smaller model to understand more deeply offensive cybersecurity.

unknown 49:31

Right.

SPEAKER_02 49:31

But there's other steps that you have to take to do that.

Sean Applegate 49:34

Yeah, one of the things too that we run into when we think of general unstructured data, this came up at the IBM Think Conference a few weeks ago. Uh a lot of this gets back to you need to still understand what's the right data set to use. And so when you're doing RAG, for example, if you're doing a bunch of process work, where where's the authoritative process that we want to use? Is that there? Do we actively manage that in the organizational level? Um, if we have six different versions of the process document, which one's the right one to feed in for RAG? So they get the right process and not a process that's two years old. And so a lot of this, when you think of managing your data, the unstructured stuff specifically, whether it's a process documentation or a wiki for how a team operates, doing that as a frontline manager and managing that knowledge explicitly in your organization is really valuable, not just for your workers, but for the AI agents that may need to consume and learn how to do something or understand the guardrails to play within, or the SOP they should use. And this is something when you think of uh we've talked about skill files in the past. One of things we we involved with the Atark um DevOps group, there's an agent group too. Uh uh Carnegie Mellon's pretty involved in that. One of the things they found was having both a skills file to describe what you should be doing and the tools to use are really important. But what they found also that really provided a lot of value, which is kind of like it uses a rag. You're reading it into memory, but they created what they called a playbook. And the playbook was also telling the agents what explicitly they could not do. So what's kind of off off limits that you probably shouldn't do. So you're kind of saying, go do these things, use these tools. Bible, by the way, make sure you don't do the, don't violate any of these other policies that you need to stay within. And what they found when they studied this across eight different large language models, that playbook of what boundaries are off limits and that you should not do explicitly was really useful to keep them within the guardrails of the desired operating environment. So they didn't go off and do things they shouldn't do. And what was interesting was they they had tested that against explicit guardrails enforced on like the cyber domain as well. And they compared the two. And what they found was if you have those guardrails external to the agent's awareness, the agent will attempt to work around those typically because they hit a roadblock and they're very action-oriented and they want to get something done and they determine, like, hey, let me go down this route hole and figure out how to get around or around this blocker because they don't know not to get around it. It's almost just just give me an excuse not to listen to these, you know. Yeah, exactly. And it's like any other, I think, curious, especially curious cyber engineers. We're talking about cyber. All right, you you give us a curious cyber engineer a challenge and they get stuck and they're like, I want to go figure it out. Let me go work around it. Agents in general were doing that. So it's interesting to see if you train them. They do a better job if they have context of both what to do and what not to do explicitly in their own memory or context, um, and the versus trying to put hard limits around them that then they hit their head on and they want to try to work around.

SPEAKER_02 52:33

Yeah. And actually, it's just just for the listeners, so they don't think, you know, the way I've described the whole experiment sounds very wild, wild west, uh, which is done very safely. And actually what you're describing, you know, just to go a little bit more into the architecture, um, what we settled upon was not only the discussion skill and the create capability, uh uh create skill capability uh pipeline, which is really what it used to autonomously create the skills for each one of the uh tools that sat on top of the operating system. Uh we also ended up moving to an orchestrator and subagent model. So basically, you know, basically think of one manager. And what that manager would do was hand the subagent first a safety capability. Like these are the things that you are allowed to do, there are things you are not allowed to do, here are the steps that you're going to take. You can't move from one step to the next step until you've completed this step. And then once they acknowledge that they had their their basically their constraints or their safety capability, then they were handed the tool capability, and it was here's the information, this is the target, this is this, this is this. So it was very structured. Uh I know it sounded like in the beginning, you know, very wild, wild west, but that really goes to what Sean was saying, right? The the guardrails in this particular uh experiment basically proved to not be effective. Uh, but the uh the safety or the constraints capability uh is really what kept this thing on the rails and uh them not attacking everything in my lab, right?

When Agents Break Production Systems

Brian Lake 54:10

So it's defining the constraints with the agent versus necessarily just having guardrails in the architecture itself is what you're saying. It was more effective. Yes.

Sean Applegate 54:22

Okay, absolutely. Yeah, the other thing I will say, and this is to just if you're doing this stuff in production, be prepared to have problems because agents can move really quick. Oh, yeah. And you need to have the ability to recover and fix things if they break them. So there were there's there are some real examples where agents have run into challenges and to work around a challenge, they might do things like swap out keys for some reason. This happened in production um at a commercial entity not too long ago. And when they swapped that key out, it broke uh months worth of data in a production database and caused outages in an application. Oh, by the way, it also for whatever reason, uh in their case, it corrupted the backups somehow. Um it broke it. So they couldn't restore the backups because they lost the original key that was used encrypted. Yeah, yeah, that would that would be that would make it pretty much uncomfortable. So all of a sudden, when you think about these things, having having the ability to uh have a good continue of operations disaster recovery plan for your that's application consistent and container and host consistent is important because you will have agents break things just like humans break things, right? A lot of errors that happen in IT are not because of technology. They are because humans caused a problem. For example, when I was sitting in an airplane several years ago pre-COVID, and the FA had an outage on an application was really caused by a configuration change by a human, and I had to sit on the airstrip for four hours waiting to take off to go to an overseas trip, right? That's the reality of the real world. Uh, don't ignore that. Agents operate a lot like humans. We have to fix the things they break. So having immutable backups, for example, so you can restore the application in minutes and not days, we need to be prepared for that. These things aren't going to be perfect.

Brian Lake 56:02

Yeah, and this is where Cohesity is well, well positioned here to assist in this in this area, right? I'm assuming you guys are having these conversations all the time now.

SPEAKER_02 56:10

Yeah, no, absolutely. Um, and I think that was actually the the end result of the experiment was to really kind of push this home. What I see, and this is nothing new, it's been going on in cybersecurity for a really long time, is organizations, and I think it's because of the way the human nature, right? We want we want to win at something. We want to be good at something, we want to achieve something. So in cybersecurity, we see organizations spending about 80% of their budgets and probably more than 80% of their time, you

Safe Cyber Recovery And Jump Bags

SPEAKER_02 56:40

know, protecting and detecting their networks. They should. I'm not saying that that's not the case, right? Uh but I think we have moved beyond, and the statistics kind of show it. I think uh uh the um uh Sophos basically I think it was last year said 59% of the organizations inside of the survey, and it's thousands of organizations to include federal and state and local and and and and other verticals, right? 59% of them experienced some sort of cyber incident in, I think it was either 2024 or 2025, I can't remember which one I was reading recently. Um that's two-thirds. It basically says like you have a two and three shot of having, and I'm sure they ranged, you know, it wasn't absolute, you know, disaster and catastrophe uh for every single one of those, but that basically just shows that as an organization, we cannot protect and detect our way out of having this problem.

Sean Applegate 57:29

Well, and I think the important thing for most agencies, when you think of the team operating the digital capabilities of an agency, protecting things is one way to prevent bad guys from doing things. But when you think of having a DevOps team, a site rear liability engineering team to run the infrastructure, and you have the ability to observe what happens in that environment, things like change analytics, uh you get the ability to look at uptime and availability, service level indicators and objectives, and the ability to look at individual transaction logs and configuration files. The uh value from AI in that area from an operation side is enormous from what we've seen for AI ops. But but the same thing goes, if my agents are going to go do a lot of things for me, separate of cyber defense, but actually used to be more productive doing things in IT, we have to have better observability and the ability, um, when you get back to the right data, to not just have AI agents use that. But if the agents do something wrong, we have to be able to audit like what did they do, what'd they touch, what changed, did it impact a service level indicator at like response times or successful HTTP transactions to a very important website, or we get a bunch of 500 service unavailable errors all of a sudden. The ability to proactively trigger that, uh, identify what changed exactly in a configuration file or a piece of code, and then revert back to a successfully functioning uh configuration is really important. And this is is often not executed well in a lot of our large complex organizations because you might have one team writing code or one set of one small team with a lot of agents writing code now, right? And maybe more agents future, and then you push it to prod and the team running production operations in a lot of cases isn't the same contractor or maybe the same team or department that wrote the things. So having a good SRE framework, um, when you think of monitoring uh your blue-green deployments for containers to make sure that when you put the new capability in production, it actually works when it performs better, or if it's slower, let's figure out why it's slower. Or do you need to have, for example, a QA or a performance agent during your QA cycle evaluating all that stuff? And then when you push the prod, that agent's watching prod, or you've got the right policies, it fail back automatically. These are things we're seeing cutting edge organizations in federal have been doing for a couple years when you think of adopting something like Harness or a more modern CICD pipeline with some things like GitLab, you know, they're monitoring those changes, whether it's a human or an agent pushing the code, they don't really care. What they are do is monitoring those application transactions. And when they don't work well, they're able to revert back automatically, or with a human in the loop. I think that's going to continue to accelerate when you think of IT operation staff efficiencies that are going to take advantage of agents to help them run things more effectively, or in some cases do their advanced troubleshooting in a much more capable manner. And we're seeing this where agents are members of agile stand-up calls now, where you can ask the agent a question. It will go figure out what change and give you some answers back, just like a human could. Um and you can do those things verbally now if you if you want to turn those functions on, but a lot of cases it's in a in a chat operation today inside of Teams or Slack. So I'm excited to see what we can do, not just for the defend the cyber landscape side of things, but really how do we accelerate innovate, you know, the innovative side of IT, get better productivity gains there. And I will say from uh the ATAR group, for example, USPTO is doing an amazing job when you think in terms of building product managers that are focused on a capability, not just a team writing code. So they really align to the business outcomes of patent and trade office, which is one of the foundational things of America is how do we innovate faster, how do we pass patents better, how do we make sure that we protect the intellectual property rights of companies and commercial entities that are doing that? But that really empowers the great amazing American dream and the commerce function of America to move quicker and faster. So I think those are some of the shining examples we've seen in the government today where they are really leaned in hard on not just doing the techie work, but building the mindset and the leadership thought processes around enabling that.

Brian Lake 1:01:31

Are they moving fast enough from a policy standpoint to develop mandates and give agencies kind of that guide rails? You got to be thinking about all these different pieces of the puzzle if you're gonna actually be able to do this successfully?

Sean Applegate 1:01:42

I'm a big fan of the high velocity edge. So I think if you have good employees with general guidance and they know kind of what the targets are to stay kind of hit, you need to probably trust them a lot to do that. And I will say a lot of my my personal friends that I grew up with in the military or worked with in industry, I think in general, the folks we have in the U.S. government that are in those hands-on engineering positions have the right intent in most cases. Right. And they really are concerned about making sure they do the right things. Keep mind, these are probably your neighbors in your local area. If you if you're in Nova, Maryland, Virginia, um, you know, it's the guy you play base your kids play baseball with, or you know, it's the the lady around the corner that maybe watches, used to watch your kids, and now she's doing a government job writing code or whatever, right? I mean, um, they're not people we don't interact with day-to-day. And again, they're just humans like everybody else, they're not going to be perfect. So I sometimes I think we try to be perfect, and that's kind of the enemy of of good in a lot of cases. But if you didn't say it, I was gonna say it.

Brian Lake 1:02:44

Don't let perfect be the enemy of good. Well, I I and this has been a great conversation. Final question to the both of you before we close. I mean, if you think about this experiment that you've been running, Marlon, and the conversation we've been having today, and to your point, I think the the federal government is working very hard to try to take the the government into the future, into an agentic future. If you're a C DO, if you're a policy leader, if you're an IT leader, or if you're you know, if just i in the weeds, uh, you know, knob turners and and working in this space, a developer per se, what are you thinking about on Monday morning, or what can you think about maybe on Monday morning, that by Friday you've you've either accomplished or if you've you've developed something or you've tested something. What's what's some of those interesting things that they should be thinking about?

SPEAKER_02 1:03:34

Well, I'm gonna I'm gonna frame that based on why I did because this wasn't in a vacuum. Right. You know, for what I think. And you did in four days, right? Right. Well, but it wasn't done in a vacuum either for what I do uh, you know, for a you know, for my my actual career, right? Uh there was uh uh a point to it. Now there are plenty of experiments that I do that are just for my own curiosity, but this one you know actually did have a place. If what we're saying is that the barrier to this kind of activity has basically gone away, and we're no longer looking at adversaries being one in tens of thousands of people who live in another nation state that you know have connections and lots of money and uh you know a lot of time to to learn these skills. That it's now basically an average intelligence person could do this in a short period of time. Another piece, by the way, Jenkins was one. The machine he was running on could probably run ten of him, you know, um and it's not a very powerful machine, a moderately powerful machine with like a 50-90 in it, so for a couple thousand dollars, you know, expense could probably run hundreds of Jenkins and do it on local models, right? And we've already seen it, you know, the the rate of attack has most certainly increased, right? And it's due to uh, you know, agen AI in general being utilized to do uh these offensive things against, you know, for example, us, the United States. So if we're saying that, and that the statistics show that more than likely as organizations, government, or commercial, we are going to have to deal with some sort of large-scale cyber incident inside of our environment. If those are all true, right, I think we need to, what or what organizations, especially leaders uh in cybersecurity, we need to take a look at the difference for how much money we're spending trying to keep them out, continue doing that, but maybe move a little bit toward the, okay, if everything else fails, the only thing that is important to me and my organization now so I can continue my mission, so I can continue uh achieving my goals is the speed at which I can safely recover back to you know a working environment. When I say that, I don't mean find the last clean backup. Is kind of a uh a bad lie that I feel like maybe the competitive landscape has been telling for a number of years. It's not just find the last one. Because if the other statistic, which is 277 days adversaries are actually inside of our organizations before we actually detect them, then that backup more than likely has them still in there. The process for recovering from a disaster is one thing. The process from a cyber incident is completely different. We need to have a concept of what our minimum viable capability is or tool set is. We call that the digital jump bag. Then we need a minimum viable organization or minimum viable agency for the government. These are the systems that I need to have online to be able to actually accomplish the minimum goals of my mission, right? And then from there we can bring other systems. But there is an investigation portion, which is how did they get in? That's the forensics, how'd they get in? This goes back to the logging and all the information you were talking about, the visibility. How did they get in? Uh, how did they maintain persistence? How did they elevate their privileges? You know, what were the vulnerabilities they used? How did they tiptoe around my tooling? How did I not see them till now? You know, all those things. And then a mitigation process to eliminate those so that we can bring those systems back online, knowing that we have eliminated them. They're no longer in our system. That's a safe recovery, right? And uh any organization that has not gone through that is probably more than likely wholly unprepared to do so. Um, uh, but they need to start thinking about how do they get to that point

Monday Actions And Final Takeaways

SPEAKER_02 1:07:30

because speed recovery is the only thing that matters at that point.

Brian Lake 1:07:32

So organizational recovery in a cyber AI world. Yeah. Okay.

Sean Applegate 1:07:37

Sean? Yeah. So if you're if you're a leader listening to this on Monday afternoon, Monday morning, you know, my goal, my challenge to you this week would be get some hands-on. Right. And you could do that, for example, if you have O'Reilly, you could go into their sandbox environment in the O'Reilly learning management system. Your agency probably pays for it. You can spin up AI agents, you can spin up AI sandboxes. There are guided labs that in an hour of your time, or maybe a couple afternoons or lunch breaks, you can learn a lot in five days. So I challenge you to give that a try, or maybe challenge your number two to give it a try if you're too busy as a leader, right? Find a curious engineer or person on your team that's a little savvy and challenge them to try to learn something new and maybe solve a very specific business problem after that, or at least put it down on paper and how you maybe would architect solving it. Two, I'd say audit your AI supply chain. Um, there's certainly been a number of high-level discussions recently at the executive levels of Department of War and the F Sebs around what's being used. Um, if you're an agency sys an agency CAIO, a CTO, go look in maybe some of your observability logs, your zero trust uh applicationware logs. You probably have some of those things floating around, whether that's uh Zscale or Palo Alto, Cloudflare, uh Dynatrace, maybe uh some other other monitoring cyber tools that do NDR. You'll be surprised with what they know about your environment. But one thing you might want to know are, for example, which models are being used in your environment. And they're not all going to be in your authoritative application architectures. They might be things that are being used by SaaS applications that some commercial entity that you pay for is kind of plugged in the back end that you're not aware of. It may be, you know, a research scientist at a lab that spun some stuff on their machine and they're now doing some things and it's visible either at the endpoint or on the network doing things as they share it across to their team members. Um it's a bit like the Wild West out there right now, but you will be shocked with maybe how many models you see, the variety of those things, and maybe where they came from that wasn't expected. So be aware of your pieces and parts and maybe the supply chain risk around those things. And then just go have conversations around. Don't come down with a heavy hammer on them. They're doing something creative, they're being cost effective, they're experimenting. Create a safe environment to have that conversation so then you can affect it in whatever way would be beneficial to your organization and not detrimental to impeding innovation. And then I'd say get the leadership involved. A couple of the agencies we've seen took that education of leaders around AI very seriously and invested time and sending people out to things like universities to get executive AI development courses that are uh three days to a couple weeks long. Um, you know, figure out whether you need to invest in that to really drive some thought level uh thought leadership and change at the executive level from the top down so they can lead from the front. Um if you if you go back to kind of a Department of War reference, right? You know, marine leaders often are going to be the first ones in, last ones out. Um you got to make sure you you think of that as if you're a leader, like how do I lead and get smart to do those things so I'm not sitting here on the line sideline, just have no clue what's going on.

Brian Lake 1:10:53

Yeah, and that was I'll leave it there. That wasn't a final dig at the end of the day. That wasn't the final dig at all, right, Marlon. So Marlon, uh so this has been a great conversation. It feels like like a couple years ago, the three of us sitting around drinking bourbon and talking about this kind of stuff. I'm so glad you were able to come and enjoy the show. Uh you're you're speaking this week in in DC at the AWS Summit. Am I am I correct? Uh yes.

SPEAKER_02 1:11:15

We uh we have a uh uh one of the the side feeders. Okay. I believe it's uh Tuesday at three o'clock.

Brian Lake 1:11:23

Uh it's June 30th? Yes. June 30th, two o'clock. It's at the convention center. What are you gonna be talking about?

SPEAKER_02 1:11:29

Uh what I normally talk about, uh data for AI, data protection. Um similar to what we've talked a little bit about here today. Okay. Uh, but I'll probably go a little bit more in depth for the data for AI.

Brian Lake 1:11:43

Well, if you're gonna be at the AWS Summit, make sure you go and check out Marlon's presentation. Marlon, always great to have you. Uh we'll make sure we have in the show notes peripheral tech. Go read the Substack. Go uh see what Jenkins is up to. You're gonna I'm assuming Jenkins is also gonna now probably start his own blog about his journeys and experiences, like a little tr Jenkins travel blog or Jenkins did a Jenkins travel blog.

SPEAKER_02 1:12:07

That would be that you know what?

Brian Lake 1:12:09

His journey through the cyberspace, yeah.

SPEAKER_02 1:12:12

Right. Um He's actually already wrote his b his own bio. Okay. Um we'll make sure Jenkins' bio is in the show notes, too. Uh Jarvis is actually going to make an appearance because and we didn't talk about it today, uh it's a totally different s uh subject, but it was my journey into the the world of AI psychology and how that became really, really real for me. Uh and it is somewhat of a actually it is a very humorous story, if you ask me. Uh, and Jarvis will be basically giving his rendition of those events. Um the the teaser is is we were developing something and he had asked his QA agent, it was mentioned here, uh, to do something. Uh and because of some some components and characteristics of the QA agent, uh, she basically was not doing it. And Jenkins, for an AI, lost his patience and basically called her a liar, which I thought was funny, but i it it goes on from there. It's it's much more interesting.

Brian Lake 1:13:12

Make sure we uh see sounds like Jarvis is a real character. So definitely check out in the show notes uh Peripheral Tech by Marlon McFaith. Uh for upcoming events, webinars, roundtables, and other episodes of the Just a Government IT podcast, go to gist360.com. Oh, don't forget, like, follow, subscribe wherever you listen to your podcast. Gentlemen, great conversation today. Thanks for coming, Marlon. Thank you, Sean, as always. Let's talk pleasure. See you soon. Absolutely. Have a good one. All right.