EP1030: AI agents and the risks they pose in financial services

Chandra Gnanasambandam, Chief Technology Officer & EVP of Product, SailPoint

In the background, data that shows 80% of organisations reporting that AI agents are taking unintended actions left to their own devices. Indeed, agents are mutating irrespective of being denied access to systems. Agents need intent-based authorisation, not policy-based authorisation. Robin Amlôt of IBS Intelligence discusses the evolving challenges of managing AI with Chandra Gnanasambandam, CTO of SailPoint.

Transcript

Copy Transcript Copied

SPEAKER_01 0:00

I'm Robin Emler of IBS Intelligence. You're listening to the IBSI Views podcast. With me is Chandra Narnisan Bandam, Chief Technology Officer and EVP of Product at Sailpoint. We're talking about artificial intelligence. Isn't everybody talking about artificial intelligence? Specifically, AI agents, agentic AI. There seems to be a new announcement about a new AI agent every day, if not more than once a day. Is this headlong rush something we should be encouraging, Chandra?

SPEAKER_00 0:33

Robin, this is um inevitable. This uh AI and agentic, you know, technology, it's not a fad. It's here to stay. It's one of the technologies that's going to fundamentally drive productivity and economic growth, right? And so the pace of innovation is just, you know, we have never seen none of the none of the prior technology waves, all the way going back to the internet, you know, has seen the pace of innovation now that that we are seeing with AI. I think we will stabilize and consolidate, and this will the market leadership will emerge. But until that happens, I think this pace of innovation is going to continue.

SPEAKER_01 1:13

The problem with it is, though, that there are risks that people don't entirely, I think, understand with what they're doing as well.

SPEAKER_00 1:24

That's correct. And uh, Robin, I would even say, you know, people are not even beginning to understand the risks. There are sort of three kinds of risks here at a meta-level, Robin, that people are in quite beginning to understand. One is you cannot manage and secure something that you don't know you have. So, first is just the visibility risk. You know, there is not a single corporation that we know of, and we have been in this protecting this uh AI and agents now for more than a year, that have full visibility into all of the agents and non-human identities that work inside the corporation. That's risk number one. Risk number two is even when they know, they have very poor mechanisms or no mechanisms to really control who has access to what, which human has access to what agent, and what those agents have access to. Do they have access to sensitive data or no? Those kinds of basic things, even, not even sophisticated things, right? That's really risk number two. Risk number three is even if they know, oh my gosh, right, this person has access to the wrong agent, that agent has access to sensitive data, it's not supposed to. They don't have controls to stop that from happening right away. So you have those are, in our view, the three meta risks that we see today.

SPEAKER_01 2:53

I'm slightly worried by your risk number two, because I do know that AI is starting to change. I wouldn't necessarily say improve itself, but it's certainly starting to mutate. And if you're telling me, first of all, that an organization doesn't know what it's got, doesn't know what the uh AI it's got knows, and then the AI itself is changing. Correct. How do you control this?

SPEAKER_00 3:25

So, you know, you need to get a few principles in place here because it's not just technology. You need to get some principles and practices and methods, you know, right. One is you get near complete visibility. And and the good news, Robin, is there are technologies available today. I can say that with confidence, where, you know, including us, but this is not about our product today. So there are technologies available, solutions available in the market, which will get you almost near visibility, number one. Number two is to manage risk number two, which is, you know, how do you control what this agent has access to? And what do you do when it mutates, which is a question you're asking. Again, there are solutions available. It's called the security parlance's access model, is the concept, is the term that's used, which is really simply defining which human has access to what agents, what data an application that agent has access to, right? That's sort of what in security language we call an access model. You can define and enforce an access model today. Now, your question is okay, you define it, but these things go wrong, they mutate, then what? That's why this concept of you have to monitor them in real time. Think of it as kids playing in the playground. You always have an adult watching them because you know you don't know what they're going to do and you want them to be safe. It's the exactly the same way. You want an quote unquote an adult here, which means another agent, practically Robin, another agent, which is an adult agent watching what all these agents are doing and what they are accessing. When they find something wrong and they need to be told what is quote unquote wrong, they should act on it and stop the misbehaving, the rogue agent, the mutated agent. There are concepts like the kill switch, disable switch, immediately stop it, or if you cannot, at least alert a human that can do something about it. That's how you manage those risks. You know, but the good news is there are solutions available today to do all of these things, Robin.

SPEAKER_01 5:46

Okay. This is probably not as apocalyptic as I sometimes think it might be, but it's obviously going to change the way we do business utterly.

SPEAKER_00 5:58

Completely. It is, it is already changing, you know, Robin, because, you know, this is I always think of uh, you know, the best way to think about this is, you know, in any corporation, how a business process is getting executed. Because that is always, you know, you can think of corporations as a combination of business processes they run. And a lot of them are getting either completely automated with a fleet of agents that autonomously are working to execute the same business process that humans were doing, or humans and AI or humans and agents are jointly executing those business processes because you can be more efficient, you can get them done faster, and so on and so forth, right? So that's why, you know, so the short answer, yes, it's gonna profoundly change business. And Robin, we haven't seen anything, quite frankly. You know, it's going to be, if you really, if you really step back and think about the invention of the steam engine, electricity, internal combustion engine, the railroads, I think this is going to be even fundamentally bigger than any of those in terms of what it's going to change or what it's going to do to change the way humans work. So it's going to be a profound change. And we are in, we are very, very early in this. We are very early in this journey, Robin, the whole world.

SPEAKER_01 7:21

Well, in terms of managing it, there's intent-based authorization or policy-based authorization. Talk to me a little bit about what these are and the differences.

SPEAKER_00 7:31

Sure. So humans today, right? So let me step back, right? So I'm gonna I'm gonna use some technical judgment, but I'll simplify it. There are three ways of getting authorized today. You can do it either role-based, meaning Robin has this role inside this company, and that role can perform these tasks or access this data. That's role-based. Policy-based is independent of what Robin, Robin's role allows him to have permission to do, we're gonna have some policies because he can't access the data at all times. He can only access it during this time or only during the time he's working on this project and so on. That's policy-based. The third kind, Robin, is actually intent-based. Intent is even if an agent is acting based on the policy and accessing a piece of data or being authorized to act on a piece of data based on policy, its intent could be wrong. So, and because agents are prompting, you know, the way they do this, Robin, is really, you know, literally plain English. This whole fancy term of intent is all about analyzing plain English, plain text, or natural language, be it be it, and be it in any language. You have to sort of apply machine learning to say, let me understand the intent of this prompt. Oh, that person has the the is authorized to access this piece of data. The way they are prompting it, I suspect foul play here. I suspect somebody else has taken over this person's role. And so I don't really trust this anymore. You have to do that for agents because you have millions of agents prompting other tools to do something for them, because they are like a digital human. And you can't trust that these things are acting with the best intent at all times. So this whole fancy concept of intent-based authorization is where every prompt that the agent is typing, asking something to do or some tool to do something, you have to analyze the natural language in plain text, understand the intent, and then decide whether, independent of what the role and policies are, whether it should be authorized to go conduct the task. Hopefully, hopefully that sheds some light into that question.

SPEAKER_01 9:58

Up to a point, this is what SalePoint does. You work with major companies, I believe nearly half the Fortune 500 to protect against the vagaries, shall I say, of artificial intelligence implementation. That's exactly right, Robin.

SPEAKER_00 10:18

So we do, if I were to grossly simplify how what SalePoint does in agent security, we do it grossly simplify for me, please. Yes, I will. We do three things, Robin, and they address the three risks I talked about. Number one, we have the best visibility or discovery and visibility solution in the market. So you can have any type of agent, any platform in the world, we can discover and we can provide visibility. Number one. Number two is we can define the access model. We can specify, we have solutions to say which human should have access to, which of these agents are non-human, these non-human identities. What application or data should those non-human and agents have access to? So we do both of those. We do the access model, and we can also provide auditing of the access model, Robin, because it's not just you defining the access model. You have to periodically certify whether the accesses are correct or whether you have to go course correct them. So there are multiple frameworks in the world. In fact, the UK has some of its own frameworks they are now debating. So that's they want all the companies in the UK to follow that. So how do you go audit the access model? That's the second thing we do. The third thing we do is all around intent-based authorization and prompt, what we call prompt security, which is all of us, you know, use some, you know, you can go to just if you, you know, if you go to Google today and, you know, and do a search, you are essentially, if you're in the AI mode, you are essentially prompting Google's underlying Gemini agent. That, you know, you are using an agent without even knowing you are using it. If you're in the AI mode in the Chrome browser doing a search in Google. And so how do you analyze the intent and make sure that what you're prompting is actually legitimate and not you've not been hijacked, you know, and the uh intent behind the prompt is wrong, right? That's the third thing we do. Authorization and intent-based security. So discover, govern, protect. Those are the three things we do.

SPEAKER_01 12:52

And I think you're going to be busy doing them for quite some time to come as the world of business evolves. Chandra Nanasan Bandan, Chief Technology Officer at Sailpoint. Thank you very much.

SPEAKER_00 13:04

Thank you so much, Robin. Yeah, I really enjoyed this.