.png)
Detection Dispatch (Alex's Version)
Detection Dispatch (Alex's Version) is an independent detection engineering & threat hunting podcast. Rebuilt. Community-first. Featuring a lineup of the real and active projects pushing the limits of detection engineering, threat hunting, and everything in between.
Detection Dispatch (Alex's Version)
There's No D3FEND for AI (So He Built One) feat. Edward Lee
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Quick count: MITRE ATLAS, OWASP's Top 10s, NIST AML, Cisco's framework, MAESTRO, Databricks' AI security framework..Every org says "go do AI security," startups are popping out to sel lit to you…points you at a pile of frameworks that all describe the same handful of problems in slightly different words, and leaves you to cross-reference them yourself at 11pm.
Edward Lee joins Dispatch to talk about AIDEFEND: the open-source knowledge base he built to be the thing MITRE ATT&CK has and AI security doesn't: a D3FEND. One place to search across AIDEFEND and ATLAS techniques, MAESTRO threats, every OWASP Top 10, Cisco, NIST AML, type in a keyword and see how it maps everywhere at once, instead of holding nine tabs open and doing the translation in your head.
In this episode we get into:
- "Detect" isn't one thing — it's 16+ techniques, and only three signals actually matter: prompt-level telemetry, tool call graphs, and retrieval provenance
- Why the retrieval layer is ground zero — if you can't say what got pulled and from where, you're not reconstructing that incident, you're guessing
- Rogue agents behave less like malware and more like an insider threat — legit permissions, illegitimate intent
- "Goal drift": when a tool call sequence takes a left turn that only makes sense if something got injected midstream
- The case for giving agents their own identities instead of borrowed ones — and why token usage tells you almost nothing useful
- Where to start logging with zero visibility today: proxy-level tool calls and API calls, before you even look at content
Follow Edward's work:
- AIDEFEND — aidefend.net
- AIDEFEND on GitHub — aidefend.dev
- AI Defense Matrix (more executively friendly matrix) featured in the episode — https://aidefensematrix.com/
Detection Dispatch (Alex's Version) is an independent detection engineering & threat hunting podcast. Rebuilt. Community-first. Featuring a lineup of the real and active projects pushing the limits of detection engineering, threat hunting, and everything in between.
Welcome back to Detection Dispatch, a show where we go beyond the alert and into the craft of the engineers, uh building the detections that actually hold up under fire. Every team I talk to right now, I feel like is either building an agent, buying a stag of agent swarms, talking about their frontier model comparisons with the release of Kimmy K. But the threat frameworks, it seems like they caught up pretty fast, right? The miter, I guess OWASP now has its own uh agentic top 10 and then Maestro as well. There's almost so many that we as detection engineers now have to keep track of. We're going to get into today something that can help us navigate it all, quite literally. Uh joining me today is Edward Lee. Um, Edward, welcome to the show. I'm so excited to have you on. I've been after you for a while, chasing the time zones, and we're finally on the same ish time zone.
SPEAKER_00Yes, yes. Thank you very much, Alex. Um, hi everyone. My name is Edward Lee. I'm currently the uh CEO and founder of AI BFEN Labs. And yeah, I like what Alex introduced. I've been working in AI security field for some time, and I hope uh my work can contribute a little bit back to the community. And my eventual goal is to democratize the AI security so that everyone, including big companies, including small companies, any organizations, can adopt the best in-class AI security concepts and put them into practice.
SPEAKER_01Your work is already creating impact and making its rounds around our industry. I've just saw last night, it seems like there's a bit of iteration, a similesque type of defense framework, which I want to get into. But of course, we always want to know uh how did you get into this space? What was your journey into the cybersecurity industry like?
SPEAKER_00Sure thing, sure thing. Yes. So I've been in cybersecurity for about 14, 15 years from now. At the very beginning, uh, I was a security operations engineer and working on the actual tuning of the firewall rules, uh, setting up how to setting up locks, uh, how to how they get consumed into the systems, setting up the detection, the detection systems for enterprises, enterprise networks, things like that.
SPEAKER_01And then detection engineer yourself. Yes. Because this is a space for detection engineers. Did you do it across like what kind of environments? Mid-size environments, 100, 500 employees, or like the large enterprise?
SPEAKER_00Yeah, it was a mid-sized enterprise at the time. That's where my uh first job was. I was a security operations engineer at Box. Yeah, yeah, for a couple of months, which I learned a lot. And then, yeah, and then I moved on to the security architecture place into my next company, which is Akamai Technologies, where I served as a security solutions architect as well as enterprise security architect for almost five years. And before I move on to another space, which is cloud security, which is more interesting in a sense, but uh more things to look at uh at the same time. So the next job that I went to was the security and trust advisory at Google, uh, where I served as the post sales security advisory person. Um the GCP, the G Suite. Back then it was G Suite, now it's workspace, but back then G Suite, uh cloud identity, cloud armor, security consultations. Yeah. And that that was a good experience when you can get into the space where uh cloud security was uh was hot. Yeah. Oh, it is still hot. It is still hot.
SPEAKER_01Oh, 100%. Hot hotter now than ever. Uh but it seems like you made uh sort of this rotational wave around all the the different parts of security, like you did from the firewall, from the cloud. Have you ever touched endpoint at all?
SPEAKER_00And point, not in my career. That's uh something that maybe worth exploring as the as the next step. Maybe we'll we'll we'll see what happens. Well, my philosophy or my approach to my career is that at uh from the beginning, there are a lot of things that you need to understand. So you want to exactly, exactly. And to be a good expert or a good leader in this field, you need to get your hands dirty into different uh fields before you can, you know, fully understand that and then you know race up to the next level to uh lead the different fields of security. So that that's that's my thought and approach.
SPEAKER_01Absolutely. I've always gotten along better with leaders and CISOs that are more on the technical side of things, that truly understand things. They've walked the walk before. Former practitioners live by the trade of it all, like the trade craft. I have found them to be the most pleasurable people to work with.
SPEAKER_00At a certain level, you will realize that only if you know something in detail, yeah, that you will understand the good side of it, the best side of it, the challenge of it. And then you can you can relate to the engineers and the actual practitioners and then tell them the right thing to do. Otherwise, you will, you know, if you don't have the practical experience, it's from time to time you will give them some guidance that don't work as expected, which we don't really want. So that's true.
SPEAKER_01And I also find that your team, the team that you're leading, uh just also just respects you more. I don't know if respect is the right word, but just sees you have a better working relationship, I guess. Uh, but also the caveat, the opposite side of it, is that being too in the weeds sometimes can lead through can lead to rabbit holes. So just still keeping that touch of reality with the bigger picture, too, is like this this fine line that we tend to play like push and pull with.
SPEAKER_00Exactly, exactly. One of the best leaders that I work with, uh actually in my previous row at JP Morgan, I respected him a lot in the sense that uh whenever we went through uh projects altogether, right? There are things that I could probably properly check, tracked. Uh, there are things that I lost track, right? And when I reviewed that with the leader, uh he will be like, okay, you did not track this. Let me show you these are the things, these are the systems that we can check into. Hmm, let me take a look with you at this right now. Hmm, this, this, and that. Okay, now you can find A, you can find B, you can find C to get the progress of the next step. I love this style that uh no blaming and uh giving clear instructions with the solid knowledge and understanding about how system how system works within one organization and use that to lead people with by example, not by authority.
SPEAKER_01That's by example. That's by example hits it right on the money. And funny you mentioned that you also came come from JP Morgan because you're now I can see it perfectly. You're cut from the same cloth as a lot of people I know that come out of JP Morgan, they have built incredible careers. Everyone I know that has done security there, I feel like they are incredibly talented. And also just it's a great place to build your cybersecurity career because JP Morgan has a lot of budget to work. You could try so many things. They're so open to trying all of these different products and systems. And I don't know, I feel like it's almost like unlimited budget to do what truly needs to be done. Or maybe I'm wrong. That's the perspective I get from the outside and who I've met.
SPEAKER_00Yeah, yeah. I mean, that that's mostly correct. That uh yeah, JP Morgan did give me and my team, uh, and a lot of the cybersecurity folks within JP Morgan, a lot of rooms and a lot of budget to do a good cybersecurity. We had very good leaders that meets the uh cybersecurity and the technology space. So, you know, I I like that environment, I like that workplace a lot.
SPEAKER_01Okay, love that. So, all of this journey and your path into the space now has led you to create all of these different facets and angles to uh this knowledge base you've basically built. So, tell us a little bit about what you've been working on.
SPEAKER_00Sure thing, sure thing. So uh AIDFend, uh, you can find it on aidfend.net, which is the website that I show the whole framework. And you can also find it at AIDFend.dev, which is the GitHub repository for AIDFend. It's an open source AI security defense, it's an open source AI security defense knowledge phase for practitioners, for engineers to start with, to understand how exactly one can do AI security and one should do AI security. So the AI Defense itself is a framework that consists of different tactics and um 300 plus techniques, the uh practical defense techniques for AI security, and it cross map to nine frameworks at this moment, including uh OWASP, yeah, uh My Cherry Atlas, NIST Advisory Advisorial Machine Learning, yeah, NIST uh Cisco, Google, Scife, uh, Data Bricks AI security framework. These are the top and most mentioned uh AI security frameworks in the industry today. So it is cross mapped to those, so that I try to find a balance between the the practitioner's perspective versus what management level wants to see. So that's why uh that's how the cross map happened.
SPEAKER_01I think that you you solved the the particular problem of there isn't like a single place to map every you know AI defense that is possible. And when there's literally so much, like you just mentioned more than five AI frameworks. So I think that that's literally the problem solving aspect of it here. When was the moment that you realized was there a story behind that or an incident, or was it just literally like death by a thousand disconnected frameworks?
SPEAKER_00Yes, so that's exactly the point, right? Uh so as a security person, right? We are always under very tight timeline limitation and that budget issue, right? So we can only focus on the things that we want to focus the most. However, what happens is that back in my previous roles within financial services companies, people say, yes, we need to do AI security. How to do that? Let's find there are some good resources or frameworks, and they go ahead and find that. And then they realize, okay, great, we need to do uh prompt injection protection, we need to do data poisoning protection, we need to do intent drifting protection. Those are the things that exist in those frameworks that tells you that, hey, you need to do this so that you can secure your AI environment. And as a security person, I was happy to see that. And then the next question, Tom, okay, great. I need to do these 10 things. How? How do I do this? Right? And that's the next question that as practitioners we have to figure out, right? So we either have to go ahead to find if some frameworks mention their solutions about it, or we go online to see if there are commercial solutions for it, or if there are uh open source resources for this, if there are guidances provided by some institutions or community that help us to do that. Now, all of these are, but I myself, I find it extremely difficult to keep all of these in one place so that I it's easier for me to do my job, right? I, as a practitioner, I want to know okay, things that I want to know about prompt injection. Here's a goal. Here are the technical guidance, the technical steps that I can do or I should be doing. And if I don't have time to do it, here's the list of open source resources. And if I have more budget, here are the commercial solutions that can solve this problem, right? And yeah, at the end, what compliance or regulatory or framework or framework problems that this uh this uh technique solve. That's that's the thing that I want to know. And that's the thing probably I want to show to my management. So AI defense is the attempt to collect all of these together into one single source of truth that shows that AI security uh could be separated into tactics and techniques, and practitioners can use it to actually implement these with this information given. And the management can use this information as hey, now by implementing this, it fulfills this from the uh from the frame from the other frameworks. So that's the goal of the of the AIDF and framework.
SPEAKER_01As you were building it out, did you notice or were you surprised? Um, because I'm sure there is a lot of overlap between a lot of these frameworks. How did you deal with the overlap from it all?
SPEAKER_00Sure thing, sure thing. Yes, there are certainly some overlaps between different uh frameworks that we're seeing. I would say that the personally, and I liked them a lot, data bricks, their AI security framework, and Cisco's AI security and safety framework, they are they are narrower in scope, but they are closer in granularity to AI defense implementation first orientation. So I like that a lot. What happens is that from implementation perspective, these frameworks tell you what you need to do, right? But so far, at least from what I can see, uh, there's no single source of knowledge base that tells you exactly what, that tells you what you need to do, and in addition, how you can do it. So AIDFEN, if you think about many security frameworks, they are the backbone or skeleton of a good AI security practice. And uh AIDFEN fills in the muscle of those in those skeletons to show you how exactly you should do good AI security.
SPEAKER_01Yeah, no, this is a lot more. We were just talking about this, it's a lot more practical information across the tactics. Like it truly tells me what I need to go build detections for, right? Like you were mentioning prompt injection, the drift detection, the monitoring of the policy enforcement of it all. Whereas I've I've seen some iterations of literally almost like the same thing, though the same way you you called this the AI defense framework, probably more for the C level, C level friendly, executive friendly view of kind of cross-reference against the NIST framework. And by the way, no, I will be sharing. I just permissions don't let me share right now, but I will be sharing like both of them. Um and uh and yeah, like like what am I supposed to do with AI security posture management? Like whereas in the AI defend framework, it's a lot more robust, a lot more concrete and specific of a behavior that you can actually build for.
SPEAKER_00Yep, yep, exactly, exactly. And as you uh as what Alex you described, right? Yeah, do you think it's not trying to replace any of these frameworks, right? It's just trying to tell you, uh fill in the how of what's missing in these uh in these frameworks. So yeah.
SPEAKER_01Yeah. Well, I gotta ask. I mean, we obviously love all and any community open source work, but it's not always the easiest to be a maintainer of an open source initiative or repo. So how are you doing this? Is it a personal, unaffiliated initiative? What's it been like maintaining this solo? Are you getting any kind of help or support, maybe miter-backed efforts?
SPEAKER_00Yeah, for now it's it's my personal effort mostly, right? I do get the uh pull requests from some of the uh community uh contributors, which I appreciate a lot. Yeah. And including those latest information on the defensive side into the AI different framework. That's something that I uh that I'm passionate about. That I'm doing almost every day, doing the research and track on the latest AI security trends and see if any of the new AI security defense techniques could be included into the AI defense framework. That's something that I am passionate about and happy to do when I wake up every morning.
SPEAKER_01So that's fantastic. Same pattern. I mean, this is this is what the the community thrives off of is in MITRETAC did this also too in the early days where one person or maybe it was even a small group, but they just decided that the industry needed a shared dictionary or shared language, shared vocabulary, and you just build it and share it and just not you know not wait for permission.
SPEAKER_00Actually, exactly. Actually, if you look at the uh the structure of AIDFEN framework, uh it contains tactics, right?
SPEAKER_01Yep, yep. Model, harden, detect, yep.
SPEAKER_00Yes. So the concept of those seven tactics. Wait, yes, as this is model, harden, detect, isolate, deceive, evict, restore, those tactics. I didn't reinvent the wheel. Those are inspired by another popular framework created by Mitre. It's called Defense Framework. And it's the defense, they spelled it as D and number three. D three. Yeah. Yeah, I and D, uh, which is a uh a defense framework that mapped to its own mitre attack framework. I I love both a lot, right? I mean, both of them have good, there are good indications of how secure in the security world, how attacks and how defense should be done uh from different perspectives. So, and one of the starting points of me creating AI Defense is that Mitre also created this amazing Atlas framework, which focuses on AI security, right? Attack is the uh attack uh techniques and uh mechanisms on the on the general uh world. And mitra Atlas is the the AI version of the attack framework, right? And so far, I haven't seen a defense version for the AI security. Atlas, yeah. Exactly, for Atlas. So uh that's where I, you know, that's when I thought about that, which is in about around June and last year, June 2025. And I think, hey, there's there's a gap here. Uh in my organization, uh, we really wanted to understand how defense technicism, how defense techniques and mechanisms should be done on our end. So, you know, we figure out, hey, why don't we make it a collection of a knowledge base or database so that we know what to do? And that's how I started this project. And then I feel that hey, this is something that I could contribute back to the uh security community. So that thing uh got started. Yes.
SPEAKER_01Well, you're right. The uh miter attack has become such a household name now. We get less love for defend, let alone miter atlas. Right. Hi, yeah. Uh-huh.
SPEAKER_00Yeah, I think there are a lot of reasons behind it, right? Uh, and I do think that as a security practitioner, right, I am happy to learn about the attacking mechanisms, how bad guys attack a system. That's great. Uh, but more importantly, at the same time, my management will ask about okay, great. Now you have all of these knowledge about how our systems could be attacked. Tell me how you can protect against these attacks. Then I'm like, oh yeah, we can use this tool, we can use this solution, I can create a script to detect this and that, things like that. They're all good, but I think it will be the best if we can collect all of them into one single place, and um you can keep track on all different kinds of defensive techniques, and then uh and and then you can decide um which one you should use to put into practice so that you can protect your organization better. No, no, that's the the thought of the uh that's the thought of uh AI defense framework behind the scenes.
SPEAKER_01So yeah, no, it absolutely has a place, and you got on top of it pretty fast. Have you gotten into translating this defense uh detection work and operationalizing and implementing it at companies? I want to get into that because that's something that I think that is on a lot of our mind as an actual detection engineer. I think we would probably live, if I'm looking at your framework, on the detect side of the pillar. Sure, sure, sure thing.
SPEAKER_00Now, if you look at uh the framework, right? Detection, yeah, detect is a tactic that consists of uh 17, if I'm not mistaken, 16 or 17 techniques. So there are definitely in AI security world, there are definitely uh detection engineering things that uh will be included, at least in the AI defense framework. I can share some of the things that I put into the framework, which I think that from detection engineering perspective is super important. Prompt level telemetry, right? You will need um structural features like injected content length related to system prompt, uh delimiter row confusion patterns, and sudden shift in prompt entropy. Those are the top things you want to look at at prompt level. And in agency system world, right, tool call graphs is super important. Uh, which tools got involved, uh, in what sequence, with what parameters, whether the per the sequence deviated from the from the agent's expected task. Those are the things that you need to focus you need to focus on when it comes to detection engineering in the agentic security world, right? And reg, right? And RAG. Exactly. Retrieval provenance is super important for RAG, right? Which document chunks actually got pulled into the context and from where, right? This is the uh these these are the things that are included in the techniques under the AI deep and detects tactics that are super important to look at in the from the context of AI AI security.
SPEAKER_01Yeah, as well as the framework, the view by framework, it literally has every single defenses by every single technique.
SPEAKER_00Yes, exactly. Exactly. Yes, yep. Oh, yeah.
SPEAKER_01I I had not gone to that part to that tab.
SPEAKER_00No worries, no worries, no worries. You will love the different things that I love about the about AI defense framework is that I try to make it as operationalized as possible so that in addition to tactics view, I also provide several views if you want to do good AI security. So, for example, in Pillar's view, you can see that the defensive mechanisms, the defensive techniques, they are split into four different pillars. They have security, infrastructure security, model, algorithm security, and application security. Right? So Pillar Vie is for is for people in different roles, right? Let's say I am in this company and I focus on infrastructure security. Okay, what do I need to do? Now, if I come to the pillars view, organize, yeah, that's that that's a list of things that I should be aware of or I should be doing. Now in addition, the the phases view, right? It is using the uh the concept of separating um separating security defense techniques into different phases when it comes to AI system deployment. So when we uh when we're deploying AI an AI system, uh we have different phases from system design scoping to model training, uh training and building, pre-deployment validation, pre-deployment validation, production operation, incident containment, and system restoration improvement. There are different phases, right? And in some companies, we have people that that are in charge of uh different phases of the AI system deployment. That's number one. And number two is that uh we can now clearly see that okay, for this system, now it's under the phase of model training and building and hardening. Okay, so here's the line that we of defense techniques that we should be looking at. And when we move to the next phase, then it's the next line of defensive techniques that we um that we should be looking at. So that's that that's a concept there.
SPEAKER_01Yeah, I also I keep thinking about what you said about um you you're you're offering various options, like from free all the way to commercial, because that story will be different, right? Depending on depending on who you are. For example, if we take this like prompt injection, uh what a detection engineer needs to look for, sometimes it's gonna have to be, you know, versus uh signature versus behavioral baseline versus maybe an LLM judging another one. And that that that that means different the the cost of doing each one is gonna be very much different.
SPEAKER_00Exactly. Yep.
SPEAKER_01So so they can either go with more of an open source route of detection or they can fully now go buy a tool that helps with maybe something like this. Yeah.
SPEAKER_00Exactly. That's that's the uh the concept behind it, right? The the goal of this framework is that organizations or teams in different size in with different budgets can do good AI security, right? So and it also depends on how uh critical that specific defense related to your environment. Maybe to your environment, a certain defense is super critical. Then my recommendation in that framework is go ahead and buy the best in class, super expensive solution if that works well for you, and I list out those uh commercial solutions for your reference. If it's not as critical as a functionality device to you, then I provide the example code as well as the open source resources that you can refer to as well as building it, building one for yourself to protect your system for to protect that uh that field of um security risk at certain basic level. So that's a thought behind uh the different information given in this AI defense framework.
SPEAKER_01I want to touch back on the RAG part of it all. You said there's a lot of a potential opportunity detection engineering opportunities for the RAG pipeline specifically. Where would you tell a detection engineer to start instrumenting first? Is it the retrieval layer, the prompt lead construction, the model output? Do we really just have to wait on the until the downstream side of it? Like that's you're too late at that point.
SPEAKER_00Yeah, that's that's uh that's a great question. I I personally have a strong opinion about it. So uh feel free to disagree with me if you would like to. And obviously, in security, we want to do layered security, right? There's no single silver bullet to do good security, right? We want to do defense in depth. And but I do have this strong opinion that if you can only pick one, the retrieval layer is super, super important. Almost every serious rec compromise incidents that I've seen, they enter at retrieval layer, right? So if you're not logging what retrieved and you know from from where, uh you have no way to reconstruct an incident after the fact, even though, even though you have contained the uh the impact that it is causing. But if you there if you have no way to trace back to how it happened, how it got happened, then uh it could still happen again next time, right?
SPEAKER_01Yeah.
SPEAKER_00Yeah, so uh, you know, definitely for rack and pipeline, you want to do good security at uh retrieval layer.
SPEAKER_01At the retrieval layer.
SPEAKER_00At retrieval layer, yes. Prompt construction is also is also very super important. You want to have the visibility into how retrieve content got merged with the system prompt, and because that's where the injected uh instructions actually take effect. So uh that's I would say that's the second thing to look at. But retrieval layer is super important that uh you need to know and you need to understand where uh things were retrieved from at a very at a very basic level. So yes.
SPEAKER_01Yes, and of course, just by keyword searching retrieval rag.
SPEAKER_00Yep, yep.
SPEAKER_01We'll we'll get we'll we'll we'll get populate populated some things. I'm already seeing some harden and some detect things, permission aware retrieval, of course, always always at the identity part as well, like defense and depth. For for OWASP specifically, for so for the OWASP top 10, it seems like now Agent AI it's getting its own OWASP. Uh and so what do you think is fundamentally different about detecting an agent doing something malicious versus an LLM just generating bad text?
SPEAKER_00Yeah, yeah. So uh I think it's uh it's much closer to insider threat detection than malware detection. That's my thought, right? An agent is it is closer to a legitimate credentialed user that can be socially engineered by prompt, right? By malicious prompt. And when they got socially engineered, they are led into the field of they can do malicious things within or outside of their original intention, uh, but with legitimate permission. That's the worst part of it, because they have in that in that scenario, they have the legitimate permission. Yeah. And personally, I think that actually reframes the detection engineering posture that you're looking for behavior that's technically authorized, right? It's authorized by the uh correct source. Yep, but they are contextually wrong, right? Yeah, the agent using a tool it has every right to use in a way that it doesn't match the task that it was given, right? So my working point of view is that you look for the goal drift, which does the agent's tool call sequence stay coherent with the um original stated task, or does it branch off in a way that only makes sense if instructions were injected mixed stream from a external source? So yeah, that's my thought on how this should be done. Yes.
SPEAKER_01Yeah, I think this is that's the conversation I feel like that I keep hearing across podcasts is how do you distinguish a true, you were talking about insider, a true human prompted request versus a malicious one. It's the line is getting blurry between what is what. I I've been I've been watching my own Claude Code request. They touch a bunch of my credential stores. And sometimes looking back, like I can't tell if it's just the tool doing the job or if it's sometimes getting abused because they look almost the exact same.
SPEAKER_00Exactly, exactly. And uh it's super hard to detect. Yeah. So it's a hard job for detection engineering people that uh to distinguish between between the a legitimate prompt versus uh what's being maliciously injected into the system. So yeah, at different level, right? Uh you want to be able to detect the malicious or the not so malicious, but the prompts that could lead to goal drifting later on. Yeah, you want to be able to detect that. And at goal level, you want to check the agents or the uh MCP service that you tell that you you authorize uh um doing your work are actually doing the things that you're expecting.
unknownYeah.
SPEAKER_00So these are things that uh uh you know, a huge amount but but valuable job for detection engineering people.
SPEAKER_01Oh uh do you think they should have their own set of credentials or like its own entity, its own employee ID, so that maybe on the governance side you you can keep track of them as like users and which one's an agent, which yeah.
SPEAKER_00I mean should they have their own? Very similar to 10 or 15 years ago when we introduced cloud, right? That the the the cloud environment. We used to have human being account, now we have the service account. The service account. Which is used for the uh the the the system services and APIs. So I agree with that approach that uh from tracking perspective, it will be much easier and clearer if we have different if we have categorized identity for the agents and for different AI services. I agree with that approach.
SPEAKER_01Yeah. No, if we think about the data feed of it all, I guess that maybe now it's getting better. I haven't, to be honest, looked at it in a while now, and I keep hearing that it's much better, that the Claude, specifically the OTEL telemetry. But before it used to just be like token usage and session duration. It's not really that uh helpful information to build a detection per se, a rule. What would you say is probably the best visibility if someone had let's say zero visibility into their company's LLM usage? Where would you get started with logging? What hotel you do would you have them stand up? Uh what kind of variables would you be looking for?
SPEAKER_00Sure thing. Yeah. Uh if I have to choose, right? Uh I would say start with the tool call logs as well as the uh uh the API call login, right? And uh this can be done at the uh at the proxy and gateway level, right? Uh whatever sits between you and your users and the model provider, you know, if you have the uh gateway and proxy setup, then you will have visibility into the tool call history, uh the tool call locks and the API call logs. Yeah, I mean uh from there it gives you the uh the request volume, which models are used, yeah, which tools got involved, you know, without touching the content at all. So I think that's a good thing that you could use that you track the tool call and API call at the proxy level. And I think that the layer in retrieval provenance login, if you're running around, that is also super important, right? And then probably tool call parameter login if you're running uh agents, right? Uh that's that's the next things that I I'll be looking at.
SPEAKER_01Yeah, much practical than the token usage, token maxing of it all.
SPEAKER_00That doesn't tell you anything meaningful from security perspective. So, well, it is a well, the indicator, right? You can you do see that uh if there's a search on the usage of token, that that means something. But you know, models uh behave in ways that people cannot fully uh expect, right? So there are some legit tasks that uh cause a lot of usage in tokens. You think that's weird, but if you look at it, you know, it's normal, it's just how that works. So it's an indicator, but it's not the uh most useful indicator in the security world. That's what that's with.
SPEAKER_01Yeah, that I've seen some good research from Trail of Bits. Uh, this would only apply if you if you have an MCP, since MCPs has now become more of the default way that agents are or talking to each other, to tools. And a trail of bits has like a like a like an MCP wrapper that will track all of the input-output requests into the MCP. Yeah, which but again, not that's not the only way agents can be called.
SPEAKER_00That's true. That's true. So I do think that there are several good contributors in the security world and in the security field that there are several tools that try to that try to collect as as many logs in a meaningful way in detail at different levels. So I think there are several very problem and promising open source projects for that. Okay, yeah. I love seeing that because you know detection engineering is so important that it's almost the very first step. Well, second step after uh after log collection, right? You have to collect all the signals, but everything that you need to operationalize that comes after the actual detection happens. So yeah, that's something that I myself, I'm, you know, I look forward to happening. I look forward to see more tools get into this field that people can leverage to do detection engineering and the follow-up operationalization.
SPEAKER_01So yeah, no, totally like the same way, like what Sysmon was for Windows. That that's what I'm I'm looking for. Trell of Bits was like the closest thing. This is this is also a step in that direction. Uh, there doesn't seem to be like a an open source telemetry enrichment type of thing for this yet. Uh, I think everyone is still building kind of bespoke logging for vendor uh or the tool calls, like you said. Yeah, have have a lot more to explore there. Uh we're getting to the end of the episode. I want to kind of close with what how exactly our detection engineers, who's never touched security, maybe has a backlog, you know, a mile long before they even start to get into some of the some of the techniques. Uh, what would you tell them to implement first, would you say, to get their best return for their efforts?
SPEAKER_00Sure thing, sure thing. So I would say that depending on what your role is, depending on which phase that you're at when it comes to the AI deployment at your in your organization, right? And depending on what your management cares about in terms of uh in terms of framework or in terms of best practices guide, right? You can always refer to the uh the detect tactics under under AI defense and cross-map that with your with your role responsibility as well as the phase that your organization or the application that you is at. And and you know, look in look at the the those techniques that you think that will to your work when it comes to implementation or operationalization, and then exploring the tools that I mentioned in those in those under those techniques, which I think that uh it will be most beneficial for you to understand the tool first, and then you can decide how important that is to our organization, and then you can decide, okay, I want to build it myself, uh, or I want to build it within my team, or oh, that's a lot of things to do, and we have the budget, so let's you know, buy a good solution for that. So, you know, just remember that the the goal of this framework is not selling you things, right? It's not it's not another framework that tells you that tells you, hey, you need to do this, but without telling you how to do that, right? The goal of this is depending on your the reality that you're at, yeah, uh it tells you the different options that you can do based on the resources, based on the budget, based on the uh the the criticality of that specific technique, uh the that specific AI security uh defense perspective in your organization. So that's a goal. So if this uh the if if the tool or if the framework could help you a little bit in doing better AI security, then I am so happy about that. That's the goal of this.
SPEAKER_01Spoke as like a true engine uh detection engineer, is starting with what you have, what you prioritize, like the good key, what your business landscape looks like before trying anything out. See, I I am I'm immediately drawn to the to the rogue agent discovery.
SPEAKER_00Yep, yep, yep. You should uh yeah, that that's a good part. I uh I think I spent almost two days researching on that topic and then and then worked on that that part. That's uh that's a good part that I have that I I have pretty strong memory around uh the creation of the part. So yeah.
SPEAKER_01Do you do you think the role of an um an AI detection engineer will will c will start to pop up in like LinkedIn?
SPEAKER_00A lot. A lot, yes. So we are now talking about uh you can see a lot of job jobs around uh AI governance or even AI security governance that are open up today, right? Yeah, which which is great, right? We uh as more and more companies and organizations are deploying AI systems and no governance is the next thing that people start looking at, right? Yeah, okay, what do we need to govern? Right? That's the next question, right? So follow up on that question, the next thing we would need to understand what to govern before we can do good governance.
SPEAKER_01Yeah.
SPEAKER_00Where do we get the data of understanding what to govern from detection engineering, right? So I think the role, I think the role of the the the detection engineering will be uh will be thriving a lot, simply because of the need of more clarity and more governance are needed in the AI and both both AI and the uh agentic AI world. So uh that's that's uh that's something that I I would put a lot of effort into my uh my framework as well to help the uh detection engineering people getting more resources and getting more better guidance on how to do good security.
SPEAKER_01Yeah. For for my for my more senior detection engineers, do you think that their access to the way they consume this framework is it mcp friendly a ball? So like if they have a whole, let's say, you know, codified CI CD pipeline as a part of their coverage assessment and everything is API driven and a part of this like agentic framework. Is is this front is this MCP friendly where they can query back into their system and their setup where they don't have to, let's say, you know, not go to the UI. They could do it in a headless way.
SPEAKER_00Yeah, yes. So this framework, AI Defense, it does have uh come with the uh the MCP service, which I built. So if you go to AIDFend.net, you will see that there's a uh on the top, there's a UMCP server and service of AIDFEN, which you can integrate with your CI CD or operation operations pipeline uh to make better use of the knowledge in the uh that that is built in in AIDFAN. Uh also on the AIDFEN.net, the the website, uh I integrate it with the the web mcp, which means that if you use your your browser's AI tool and ask your browser's AI tool on the AIDFAN.net saying, hey, I want to do this, this, and that watch in, what's in here in this in this in this AIDFEN.net website, it acts like an MCP service. So it will give you the answer back with a more structured and accurate way, uh, as how we build it in the AIDFEN.NET uh framework structure. So yeah, it has the AIDFEN has both uh the MCP service provided, which you can integrate directly, or on the website it has it is integrated with the web mcp. Fabulous, fabulous.
SPEAKER_01Yeah, and it's all linked to your GitHub here, I see. Yeah, yes. Well, thank you so much, Edward. I think we've covered quite a bit in terms of your repo, your contribution, your initiative to the community. I'm so glad we were finally able to catch up on the same time zone. Actually, are you in town for Black Hat?
SPEAKER_00Not this year, so uh this year. Yes, I will be traveling in the uh Asia Pacific region, speaking in different conferences in August and in September. But I surely will keep my eye on the latest AI defense security trend and uh keep working on refining my the the AI defense framework. So that that's for sure.
SPEAKER_01What's your plans for V2 of this? Just continuing refinement? Are you gonna productize this maybe?
SPEAKER_00That's in the plan, but I think either I productize it or not. Uh the goal is to make it more operationalized, right? Okay. So then I think that let me put it that that way. That if we if I if there's a way that I could make aitfant.net more operation operationalizable. Is that a is that a word? Operationalized. Yeah, yeah. Operationalizable. Yeah, then I'll do it. Either open source it or making it a product. I'm happy to do it. Uh, in fact, there are some projects underway that one of them is that I am trying to grouping, trying to group different use cases into um, you know, trying to put different uh AI different techniques into groups so that you could, based on your use case, you could understand faster and easier what to do if you want to implement security force in in specific use case. Let's say you're doing uh rack, right? Then one of the projects that I'm working on is okay, if you're doing rack, out of these 200 techniques, these 30 techniques are the top ones that you want to look at. So it's easier for you. It categorize things for for people in different roles or working on different projects so that you can utilize and leverage the knowledge in the uh AI Defense framework better.
SPEAKER_01Fantastic. That's I'm so happy to hear that. Well, uh, that has been AI Defend. Remember not to be confused by AI defense.
SPEAKER_00There are there are several names from different companies, from different people around that, which I like a lot, right? I mean, that the more that we have the awareness that AI security is important, the more people that we have people, the more people that pour in and uh counsel people into this field, uh the more the the better, the better future that we have for knowledge and the technology in this field, which is something that we would love to see.
SPEAKER_01And that's the spirit of community, which is what this is about. So thank you so much for the reminder that the fastest growing attack surface in most companies right now literally has a place where it could it can all live and have a shared language that we can yeah, that we can enforce and harden and build our detections against. So thank you so much for building this out in the open and for walking us through what it actually takes to operationalize it. And I look forward to continuing to see how you continue to operationalize it or make it more operationalizable.
SPEAKER_00Yeah, sounds good. Sounds good. I really appreciate your comment, Alex. And yeah, I had a great time talking with you and the uh the audience here today.
SPEAKER_01Likewise, to our listeners, if you're a detection engineer and you haven't looked at your own AI or agentic tooling as an attack service yet. I think this is a really fantastic place to start and you're prompt to go do that. I'll link all of the resources in our show notes. So go actually check out what you're logging and you know, not just bookmark it on and just save it for later. Literally actually tune in. So thank you all for joining us on Dispatch today. We'll see you next time.