IA talks AI
Targeted at UK investment management firms and IA Engine innovators, 'IA talks AI' covers the implementation of artificial intelligence within the sector, from the perspective of both general business efficiencies and with investment-specific use cases in mind. Beginning in May 2023, the series contains short, informal interviews with industry leaders, policymakers, regulators and innovators, covering policy, regulatory and practical considerations.
IA talks AI
04.01. Adam Grainger of Agentic Risks discusses the rise of AI agents
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode of IA Talks AI, Lawrence Baker speaks with Adam Granger, founder of Accomplish and Agentic Risks, about the rise of AI agents and what they mean for investment management. Together they explore how agentic AI differs from traditional software and generative AI, the opportunities it creates for research, operations and productivity, and the new governance challenges it introduces. From the “working dog” analogy for AI oversight to the emergence of the “human–agent organisation”, the conversation offers practical insights on managing agentic risk, maintaining accountability, and preparing organisations for a future where humans and AI agents increasingly work side by side.
Hello and welcome listeners to the latest episode of The IA Talks AI, our podcast in which we talk all things artificial intelligence in the investment management and wider financial services sector. Today I'm here with a very special guest, Adam Granger, the founder of uh two companies, Accomplish and your nonprofit Agentic Risks. So why don't we start off with a bit of background about who you are, how you came into the industry and the companies you founded and what you do?
SPEAKER_01Well, Lawrence, uh it's great to be here. Thank you very much for having me. Um I've been in the investment industry my entire career, other than a small misspent youth uh for a few years in the military. Uh so I've now been in the uh investment industry for over 30 years. Um, 25 of which I was in-house before uh set up with the same team that we have running Agentic Risks. I set up Accomplish to manage the industry's global client experience benchmark. And the and the short story that links that uh with agentic risks is when AI agents arrived on the scene, we thought, let's make our um client experience benchmarking business uh more efficient behind the scenes. So we took it through an agentic transformation. Uh the kind of people who make lists, lists of things that could go wrong and lists of things to make sure they don't go wrong. And uh by uh the summer of last year we realized we had something that we needed to share with the rest of the world to make sure people who also get excited about agentic AI do so in a safe way. So the first thing that we did was set up a not-for-profit, uh, as you rightly say, where we share our risk and control and agentic AI governance frameworks for free. Um but nobody can do everything for free. So we also have a um you know a perfectly normal risk management consulting business as well, you know, with just slightly different suffixes uh on the end of agentic the on the end of the agentic risk brand.
SPEAKER_00So agentic risk then uh because you had something you wanted to share with the world. And um so uh let's start by setting the stage for what that is. So when we talk about an AI agent, um you have a very a specific definition for what you mean by an agent, uh, and that's very important to understand why it is that you founded agentic risks to deal with the very specific nature that this technology represents. Um so perhaps we can start there with um what an agent is under your understanding.
SPEAKER_01Okay, so if our listeners take away one thing from this session, from this uh from this podcast, uh I would like it to be that AI agents are not just another AI model, and that we uh perhaps need to stop referring to just AI as an overall uh phenomenon. Um we have uh we know already know that we have predictive AI and assistive AI. A few years ago there was a big leap forward, um, and generative AI uh came on the scene. Um they are generative AI is the most powerful, it is self-directed, you can set it a task uh and it will um figure out how to uh achieve that, but it is passive. And this is a really important point. It will create an output for me to review. What a genetic AI will do is it will, again, plan, it will pull in the tools it needs, the ones that you've let it use, uh, it will take actions, it will interact with its environment, it'll interact with other agents, it'll interact across systems, uh, both internal and external systems, again, if you allow it. But crucially, it will then, rather than creating an output for my review, it'll create an outcome in my name. Done. Gone. And then the last thing um before I pause for breath, uh this is really important to remember the final feature of AI agents, and that is that they learn from their they can learn from their experiences uh um so they can adapt for the future for the next time you ask it to perform the same task.
SPEAKER_00So I think y agents, the way you've described them, uh are a more versatile technology than we've seen in the past, as you say. We you use terms that we would tend to associate with uh human beings, like the capacity to plan um how to address a task um and then to do it itself, as you say, to call on the right tools to do so. So I think listeners perhaps would like um and invoking um the host Truti Deb's question, her challenge to participants to describe things in ways that their mother would understand. And I think that um it disciplines you, doesn't it, to uh to describe things in in simple terms, be it quantum or any other thing. So when we talk about AI agents and the the kinds of things that we're going to want them to do that we've not been able to do in the past with older technology, what examples have you seen in the work that you've done with with clients uh where they say we want our agents to do to do this?
SPEAKER_01So for an investment audience, perhaps the most uh common example is uh perform investment research. So rather than just getting a you know a machine learning model uh to perform that, you can actually give a whole series of context and and creating the task is a mini project in itself. Uh you you know you you you can give it the knowledge that you already have, you can give it the context, you can give it the um uh the reason why you need to make a choice, and the reason why you might not want to make a choice, uh, and you can um you know uh direct it uh to achieve uh certain outcomes. Uh you can tell it what it out what its output should be, uh, and then it will figure out how it can achieve that output. But it could also be uh you know in the operations area, you could be looking at making processes that currently uh have a human, you know, taking information from multiple different systems. Um you could get an agent to do that for you because it can go in and and and read information from multiple systems as well. I can just see those as invoking tools. Uh so you can be uh you can use this autonomy uh to um to increase uh the amount of automation in in your organization uh as well as to um to elevate uh yeah, humans uh so that they can be looking at the findings of that research in that example rather than performing it themselves to higher judgment tasks.
SPEAKER_00I think the um the investment research examples are a really clear and good example because um when we think about what that entails, it's usually the the processing of lots of unstructured data. And by unstructured data, mum, um what we tend to mean is data that doesn't fit a familiar format. So we've lots of experience in financial services of technology that can say, okay, I see a field there, that information needs to go in that slot in that system, and it just kind of robotically moves that information around. Whereas agents have the ability to navigate unstructured data, you know, a company publishes their annual report, the agent's able to read that to find relevant info and put it where it needs to go or to summarize it appropriately.
SPEAKER_01Um can I add one uh extra point as well? For our fund management audience, it's easier to buy than it is to sell. Because to sell is in some way, you know, potentially to say that perhaps our buy decision might not have been right in the first place or uh, you know, needs to come to an end. When we perform the research ourselves, the person that decided that we should buy something in the first place, we are bringing a bias into that. Now, agents have biases too that we can give them, but they don't have that bias. Um uh so getting their advice as to what our actions should be is uh you know is a very useful second opinion.
SPEAKER_00I think that's that's a really good point. Uh that one speaks to one of the advantages we might see in getting an agent to do something that in the past a human being has been responsible for. So another is of course just capacity. So an agent might tirelessly work through hundreds of annual reports in a day in a way that a human analyst um would find that very difficult. But what a human analyst can do, of course, is to um read and to make judgments based on lots of summaries that um an AI agent has has performed for them. So I think um our sortness that's right.
SPEAKER_01We need we'll face a new type of exhausting. We'll be uh exhausted less by doing the doing. Um if we get this right, we'll be exhausted um uh by keeping up with all with all of the outputs um and uh you know and and and keeping up with these incredibly capable agents. Yes. But sorry, I I cut you off. You are moving to another question.
SPEAKER_00Hopefully, uh hopefully we're we may be exhausted in in a good way, like a when you have a good workout, in as much as we're focusing on the tasks that require human judgment. You know, the ones that where we really add value rather than just summarising. And that's that's not to um that's not to to say that there aren't some challenges that we'll get on to later about um you know the extent to which you know what can agents be reliably um tasked with, what is the role of junior employees in this new structure when when agents perhaps do some of the grunt work? Um but the the value is clear. So uh so for agents then to be able to navigate some of these um unstructured uh data and to uh to engage in a more versatile way, that's down partly to their probabilistic nature, isn't it? Yes. Um so what um what questions did that throw up for you that you thought were particularly important to ad to address and to think about through agentic risks, your non-profit?
SPEAKER_01Well, I think the probabilistic nature, which for our mums, just means that they're very good at sounding plausible. And uh that's that's their faulty. Uh they're really, really, really good at plausibility. But they're not magic. There is no magic, it's just a it's just a plausibility machine. Uh so how do you get the best out of a plausibility machine? Well, it's inputs and outputs again, right? Um and in the investment industry, if we've heard it once, we've heard it a thousand times, the importance of data. Uh but the new shift, the uh the new twist for agenda KI is that we need to have our data structured in such a way that an autonomous consumer can use it, can retrieve it and uh and operate it with it. So we need to prepare ourselves beforehand. Um we obviously need to train our agents on uh on data. Uh so if you haven't got sufficient training data, then think about a different use case, because otherwise you're gonna have quite an uneducated agent um performing a task and it may not go as well as you you had hoped.
SPEAKER_00Yes. Uh uh perhaps another way to to put that would be that you don't get perfect reliability out of the box in a way that we would with traditional software as we're increasingly referring to it, which means that it requires extra thought, doesn't it, into how exactly we condition these agents to to do what we want them to do.
SPEAKER_01Yeah. Uh it's it's like we're being given the parts to from which we can build agents rather than being given the finished software. We now need to create the capabilities ourselves to uh to put the parts together uh and to and to make sure they're properly informed to perform the tasks that we want them to.
SPEAKER_00And with my sort of um advocacy hat on, you know, we we have a role at the IA in in in creating rules and structures that that um that enable this innovation to work. And one of the questions I think that regulators have and are grappling with, that we're grappling with the new, of course, with the clients you work with, is how to reassure regulators, policymakers, that this new probabilistic technology is still going to deliver good outcomes for the end consumer. Um and that's that's a challenge we have to address in a new way, isn't it, to to pass software.
SPEAKER_01Yes. I mean the end consumer should always be at the heart of this either by uh using new technology to to increase our investment performance or to um to manage our our you know our own internal economics.
SPEAKER_00So as regards the ultimate goal then of this technology, which is kind of good outcomes for the consumer, we're we're kind of starting to imagine then a new world in which we've got humans who are you know understanding of the rules, what the good outcomes they want to achieve are. Increasingly we've got agents that can help them deliver those outcomes, um and they're working together. Now, your uh framework, your non-profit framework, which is uh freely available, you describe in detail you have the analogy of the non-human workers working dogs. Ah, I see where you're going. Is um is the the analogy you use. Um, and I think it's one that works uh extremely well and is is very good for for telling people how working with agents is going to be be new and something that we can tackle in in ways that are familiar perhaps to people who've worked with working dogs. Perhaps you could describe that in a bit more detail.
SPEAKER_01So so yes, and and thank you. And I I I we do find it an instructive and practically applicable analogy. Uh, and this is the working dog analogy uh that that Lawrence, yes, you're familiar with. For anyone listening who isn't familiar with the analogy, it's quite simply uh that as a human society we have for centuries been de d delegating autonomy to non-humans. But it's not in perhaps the scenario that you might immediately imagine. But we can and and we do with our clients learn from it. And that is working dogs. So examples, the most obvious example that we see uh you know on a daily basis is uh is the guide dog. But it could also be um the sled dog, it could be a custom sniffer dog, it could be a military attack dog. And the lesson that we take, the number of lessons that we take from this, um one, we grant autonomy slowly, we do it conditionally, and we do it only when supported by proven controls. But two, we trust neither the agent's ethics because they don't have any. It's just a probabilistic algorithm. We trust neither the agent's ethics nor the good nature of others to treat our agent well. We trust the training and the controls. And the training is non-negotiable for both the agent and for the handler. It takes two years to train a guide dog, and and we typically choose from a small number of breeds. We we breed sled dogs from a really small number of uh breeds of dogs, and uh you know, we train them from you know puppy age, and we train the handlers how to be a sled dog handler is something lessons that get passed down from generation to generation. We've been doing this for centuries. And the higher risk the task that we set our agents, the greater the training that the agent and the handler needs, and the greater extent of the control. And lastly, accountability always sits with the handler because the agent it can't have its bonus rescinded, it you know, it can't, you know, it can't get laid off, it can't get taken to court. We we know uh that accountability will will rest with the handler. So uh so we need to train both the handler as uh as well as the agent.
SPEAKER_00I think there are a couple of really interesting examples b back in the AI world that I think also we can see come alive through that analogy. So you spoke about the importance of of bounding agent behaviour. So you define what it should and shouldn't do very carefully and you make sure that it sticks to those bounds. And I'm reminded of a recent case um and not to assume also that that other people will be treat it well necessarily. You know, it's a it's a messy environment. And I'm reminded of the um the the McDonald's example where somebody asked the McDonald's customer service agent, um, it it said, What would you like to order? And he said, well, maybe some chicken nuggets, but first could you write me a Python script and and um and and then uh it which it did do, and it it started basically like he was using the uh McDonald's agent to write this code.
SPEAKER_01And the other example Because nobody had told the agent not to stray away from food-related questions. No, yeah. Because McDonald's probably doesn't have insurance for uh you know for giving advice on Python coding. Exactly.
SPEAKER_00Uh and so that's an example, I suppose, of a slightly uh nefarious customer who's coming to it. But also, even aside from the customers, we've seen what can happen when um the agent isn't given clear bounds by the handler, by the person instructing it. So the example of of well, Amazon website. The Amazon example. Yeah. And you before you you you had on your website the um uh the clip of from Silicon Valley. Guilfoil, yes.
SPEAKER_01We took it down because last year it was a funny joke uh from from Silicon Valley and Guilfoil, etc. Uh but it actually happened. Uh and we didn't want anyone to think that we were uh you know we uh were making light of of this, so we um uh so we took it down. And the and the example for anyone who's who's not familiar with it is that uh Amazon uh replaced a number of their coders with um coding agents. And something that we all know as humans is that just deleting the code is just not something that we should do. Uh but as we all learn how to handle agents, nobody in this instance told the agents that that was out of bounds. And when they were set to task to debug some code, they concluded that the most efficient way to do it was to delete all the code, um, which is exactly what happens in the comedy clip. It's very funny, but it happened for real life and it wasn't funny. AWS's website went down for six hours, and you know, they were in the Financial Times, I think it was February, March, and a quote, so so we're not telling tales out of school, it's you know, it's it's in the public domain. And and the quote was that they had deployed these agents before their safeguards uh were ready. Um and that's why uh you know that takes us on to the point around how moving from generative AI up into a genic often unintentionally, unconsciously through vibe coding, exposes us to a new class of risk, and and that is agentic risk.
SPEAKER_00Yeah. It's I remember how crisp it was in Silicon Valley where um Anton was the name of the AI, and and he says, Oh, uh Anton's deleted all the code. Why has he done that? And he's like, Well, I asked Anton to get rid of all the bugs in our code. And Anton obviously worked out that the most efficient way to do this was to delete all our code. And um but it's very true, it illuminates the something alien about the nature of agents. We speak to them and they speak back to us like, you know, in very, as you say, persuasive, coherent ways. And that can obscure the ways in which they can actually also be rather stupid. They're not magic as you put. They don't know unless we tell them. Yeah. And that that boomerangs back to us. My my colleague Paul gave another great example where he showed this video that was um it was a father with his kids, and it was he was saying, Daddy needs you to tell him exactly how to make your peanut butter and jelly sandwich. And they say, Okay, well, you start by putting the peanut butter on the bread. So he basically just picks up the whole pack of butter and just plonks it on his bread. And he's like, Well, there, I've done it. You know. But and it was to illustrate the point that you have to properly define what the behaviour is, because otherwise you just don't know how they're going to behave.
SPEAKER_01That's right. That's right. You have to really spell things out and break things down and and prove the capabilities and start small. But I think we're gonna get onto that.
SPEAKER_00Yeah. Yeah. So um that starts to speak to um the fact that we need the right controls. So if we if we rather than simply trusting that agents will perfectly infer our intentions and and and deliver what we want, we have to go through a rigorous process of defining that behaviour and also implementing the right controls so that that behaviour uh is. Consistently applied. Perhaps you can describe a bit more about how you've seen that play out in some of the use cases you've worked with with your clients.
SPEAKER_01Yeah, when we're working with clients, we always advise them to begin their control assessments really early in the process because the controls that you need for your agent will influence the platform that you select. Are they available to you? They will influence the um what you need your engineer to encode uh into the agent. So they will influence the build and they will influence the kind of data that you need. And if the platform can't give you the controls that you need and the uh and the engineer uh can't encode the controls that you need, and you don't have sufficient training data for the controls that you need, then you haven't got a very good use case. Um so keep looking for another use case. So it's very good to look at these things early. Um and a good example that is a novel example and something for every firm that is subject to the likes of Dora regulation is uh is is a kill switch. Because we've got these autonomous agents and they can do things like getting stuck in a loop. Now, you might have an agent stuck in a sort of a um, let's call it a non-damaging loop, where the worst it's doing is just incurring token costs. But it could also get stuck in a loop where it's turning and turning and actually causing damage, and you you need to have a proven kill switch, and the link to Dora is that that needs to be in your incident management procedure. How many firms have an incident management procedure that will stop a runaway agent? I'll bet there's you know a really small number. But this lists this comes to our point around how you need to make sure that you can govern these things, these incredibly powerful tools uh before you uh before you start using them.
SPEAKER_00And I suppose that's that's something that you have to look at throughout the life cycle, isn't it? You you mentioned the beginning part, so it's like understanding that the data has to be presented to the agent in the right format, otherwise it's going to navigate it with more difficulty, you know, or saying we need the agent to have these tools available to it, um, but you don't just say, well, we'll give all our agents access to all the tools in case they think you're you're shaking your head there. That's it.
SPEAKER_01Think of the least privilege process, the pro the principle of least privilege it's called uh in information security, where we only have access to sensitive information on a need-to-know basis. I I encourage everyone listening to translate that into the agentic era as the need to do. You might want to give your agent access to a system, but does it need to create data in that system? Does it need to be able to edit? Does it need to be able to delete? Or does it just need to be able to read? Give it the least privileged behaviors that that it needs. Why? Because uh you're just you by that by doing that you're you're you're de-risking uh your deployment and when you're de-risking you're increasing your chance of success.
SPEAKER_00Yeah. I suppose to to r return to the dog analogy, um we don't we don't train our we we can have guide dogs and we can have police dogs or military dogs. We don't train our guide dogs to defend their owners against attack. We but we do train them to have the ability, for instance, to say, okay, we're not going to cross the road just yet, don't we? Because that they can override certain commands to keep their owner safe, but within certain contexts that we clearly define and train them to do. That's right.
SPEAKER_01The working dog analogy is you can you can take it quite far because they they are a sort of a general purpose agent as well as um general purpose agents. And uh the um the point that you're making there is about what a a phenomenon called intelligent disobedience. Uh, and there are lots of examples, but two spring to mind uh the guide dog that refuses to cross the road when uh its its handler encourages it to do because it can see what the handler can't see, which is a silent bicycle coming at speed uh that could cause damage to all three of them. Uh in uh in in a sled dog analogy, uh you know, you might have um you you you you might have your lead dog get receives an instruction to go straight ahead. But with its paws and its eyes much closer to the snow, it might judge from experience that the snow straight ahead is not safe, and it doesn't want anyone to break through and go through into the cold water below. So it will be intelligently disobedient and it will go around the unsafe area and then it will get back on track. And we need to train our agents to do that, and that boom rams back to us as uh as handlers. These are lessons that have been learned over centuries.
SPEAKER_00Yes. I I it's clear how we can see that will enable agents to navigate difficult contexts like, for instance, where you have lots of users in the general public, not all of whom are inclined to treat the agents sympathetically. Some of them will be there to to kind of um well indeed, red teaming is is something that um that we can do to help manage that risk by properly subjecting these agents to the kinds of uh stresses and tests that they might be subject to in the in the wild, as it were.
SPEAKER_01Yes. And and for our mum who uh might not know what red teaming is, it's it's basically adversarial testing. So if we think about a deterministic software, we don't deploy it until somebody completes what we might call confirmatory testing. They say, I've proven that it does everything that we want it to. Adversarial testing does. Also, not saying you do one or the other, you need to do both, um, is finds out how the the AI will fail. Because uh then you can put the controls in place and the monitoring in place to one, ensure it doesn't, or two, catch it fast.
SPEAKER_00Yes. And it it's tricky terrain for an agent or indeed a working dog to navigate because uh because we don't want them to have complete autonomy because then we we no longer have control over their behaviour, but we do in certain circumstances want them to, as you say, to r n to refuse a command because the safest thing is for it to refuse a command rather than to let it its user walk it walk in front of a a speeding bicycle or that's right.
SPEAKER_01So I think that touches on a couple of different points. One, um, there are agents and there are agents and there are agents, and we encourage our clients to take a risk-based approach. Uh so uh start with uh low-risk agents, then uh move to medium risk agents. Um uh and uh you know uh only when you're really ready and you've got a full set of governance and controls in place if you need them and move to higher risk uh capabilities.
SPEAKER_00And you um in your framework, you kind of you articulate a bit more about what that low, medium, and high risk looks like. Perhaps you'd like to run us through you know, low risk, presumably the a human is very closely monitoring it at every stage. Medium risk is is gives the system more autonomy and high risks still more autonomy. How do how do you classify those?
SPEAKER_01So just to give some uh perhaps to answer uh through uh through some examples, a low-risk agent is might just be a um productivity assistant, uh helping you, you know, manage your calendar, respond to emails, um something like that. It's accessing public data and and it has limited uh autonomy. And it's helpful to look at it this way because then for uh uh for an agent like that, a company could say, well, perhaps we just need a register and a test uh you know, governance system. A medium risk agent, let's say it's accessing business data as opposed to just calendars. Let's say it's reasoning independently and uh uh you know um but but pulling together data from across different systems. Uh that that needs that needs governance, it needs controls, it needs to make sure that uh when it goes into those systems um the agent can only read if that's all that it needs to do, that it can't delete, can't cause any damage. Uh and then you've got your high-risk agents that can act independently. Uh you know, they they're probably dealing with confidential information or they may be um dealing with external systems. Uh. If they are interacting with uh with other agents through uh concept called orchestration, then that's probably a sign that you're looking at um high-risk agents as well. Um and we would also say if they have external exposure, then uh you know you you need to put a full set of governance and controls around that.
SPEAKER_00I suppose if we consider the Amazon Anton coding example, uh this is with the benefits of hindsight, of course, but you might go back and say, well, the problem there was that we we should have first asked it to to scan our code, find the errors, and then tell us what it's intended to change and how, rather than simply giving it full autonomy from the outset to say, find and fix all bugs, you know.
SPEAKER_01That's right. You would you would take a much more step-by-step approach if you were doing that a second time, and you would give the agent more knowledge beforehand about do's and don'ts.
SPEAKER_00And that that um that kind of step-by-step is one of the processes that you've learned through working with clients directly on agents, is that that's a really important way in which to work towards an agent that can do a process is to really think carefully about what steps you should break it down into.
SPEAKER_01Yes. I don't want to sound hackneyed, but uh uh you know you'll uh you'll find US Navy SEALs talking about slow is smooth, smooth is fast. So uh so yeah, take it, start small um and uh you know, build your capabilities small, prove to yourself that you can do it and and then combine them.
SPEAKER_00Are there um are there other risks then we we should consider in the context of agents? We've we've spoken a bit about some of the um setup agents, uh setup risks, the tool risks, but there are also risks once you've deployed your agent.
SPEAKER_01Yes. I mean drift is the obvious one that we should uh mentioned at the start when we were talking about what is an agent that it can learn from its experiences so that it can I'll say adapt. It can adapt in a way that is an improvement. It can it can adapt in a way that it might consider to be an improvement, but you, the owner, might not consider to be an improvement. So you have what's called behavioral drift. Um and as a result of that, your risk assessment that you conducted three months ago um on your agent is is is not valid because the agent today is different to the one that you assessed. So we need to move from static governance to dynamic governance. Um that's that's a risk of uh you know of an individual agent. There are new risks with uh with multiple agents and orchestration. Another big risk category is is security. And and these are all becoming very well documented. The one where we believe requires a greater focus is what we call the human factors. Because right now we're all learning about the technicalities of agents. We need to spend at least as much time learning about the technicalities of handling agents. And and that's that's back at us. How do we make sure we're not overtrusting these things? Because they're not magic, they're just plausibility machines. You know, how do we um how how do we make sure that we uh are maintaining our own skills? Do we have AI challenge roles? Because if you're if you're not challenging your AI, it's learning what it wants to learn. It's not learning what you want it to learn. Uh, you know, sometimes you need to guide it back and and are we doing that? Are we swapping the people around who are in oversight roles uh to make sure that uh you know it doesn't get it doesn't get too repetitive? Are we swapping around the AI challenge roles? Are we having non-AI days to make sure that we can all still do things? Um and then actually doing some tasks that you've had an agent do for a while becomes refreshing because uh you know it it can be exhausting keeping up with a machine moving at at speed and producing incredible output.
SPEAKER_00Yes, and and um I think that that's that's one of the challenges that comes with a more versatile, capable technology in a way. I I use you rightly call them sort of um they're built for uh coherence. What was the term you used, the P-word? Plausibility. Plausibility. So they're they're plausibility engines sounding right. And we've we've often seen in talking to chatbots how if they do something wrong and you challenge them on it, they're often sometimes they'll apologize and and change tack. Other times they'll just double down on what they've said, depending on how you challenge it. It might say, No, I am right, and this is why. And that that's a quite obvious example where you've caught it doing the wrong thing and and you can see that that um that it perhaps considers, well, I can explain this problem away. And and you say, Okay, I've caught you. But but with model drift, you could have a situation in which it's constantly receiving implicit feedback, i.e., I've produced I've executed this process.
SPEAKER_01It's infinitesimally incremental.
SPEAKER_00Yes.
SPEAKER_01I don't know if anyone else played this game. I did when I was a kid, and uh it was kind of like a sort of fun game that we would do, you know, uh on a on a on a warm day at school, you know, if the teacher was feeling fun, uh, you know, and it would be uh you had to wear a blindfold uh and your friend behind you would point you towards a tree and give you a gentle nudge, go in that direction. And you would walk, and and the analogy is you with the blindfold on, you're the agent, you'll walk towards a tree and you'll be so sure that you have not deviated from your direction until uh you know the person running the game says, Okay, blindfolds off, and you will realize that you have incrementally veered off course, sometimes quite dramatically. That's behavioral drift. Yeah.
SPEAKER_00Because if they're not um they might not be receiving proactive feedback on every one of the hundreds of processes they're executing a week. But if they're slightly off in some of their in some of their judgments through that process, their inference is, well, I haven't been told I'm doing this incorrectly in any way. And just slowly and incrementally, as you say, they start to drift off course.
SPEAKER_01They're not learning from the handler along the way, all they're doing is learning from themselves. They have to get feedback from the handler.
SPEAKER_00Which I think uh returns us to the point that you raised earlier, which is um you know, not not just looking at this in terms of how we set up our agents and and bound them and control them, but also about um the handler training. And there, the you know, your friend who's behind you, with you trying with your blindfold on to reach the tree, you know, that that person we're we're now saying is going to have an important and new role essentially in in attending to this agent and making sure that it stays on course.
SPEAKER_01We can't stretch that analogy too far. Um but clearly in that uh in that scenario, um, the person behind has to have a way of actually controlling that the person with the blindfold on actually gets to the destination if they have a vested interest. Let's say if they're accountable for the tree being reached. So uh so yes. And the and that's where I think we need to spend some time on the human side of all of this, because we're all being sprayed with agents at the moment by technology platforms. But this is not just a tech upgrade. This is bigger than that. And I'm gonna end with a message of hope, uh, 100%, especially uh for uh people listening in a regulated industry. But let's be clear, for centuries, the organization has been a human-only structure, right? In our companies, in our public sector organizations, institutions, in our schools, in our universities, you know, we hired people, we gave them roles, we set them inside reporting lines and we and we held them to account. But we now face the prospect of the human agent organization where we work alongside a non-human workforce to whom we delegate uh the completion of tasks to varying degrees of autonomy depending on our own capabilities to control that um autonomy and depending on our own judgment as to what is appropriate uh for the situation. Now we've seen big changes in delegation in the past. Most recently you've got business process outsourcing, where we have delegated lower context tasks, but to humans, not to non-humans. And there's a big difference because a non-human is unable to bear legal, regulatory, or moral accountability. They are operating at machine speed and it's an entire workforce. They can scale elastically. So the move to the human agent organization is best seen as uh a shift in the external environment to which an organization, be it an asset manager or any other organization out there in in any industry, needs to decide how they wish to respond.
SPEAKER_00The human agent organization. Uh so you heard it here first. Um that is the the change that we're moving, hurtling towards. Um and I think that um some people agents seem like quite a distant prospect for them, but um but I think that really it's worth dwelling on the tools that we have that already work in this way that we engage with. So one that I'm familiar with and and lots of listeners perhaps have engaged with already are deep research agents. So we call them agents because as you said earlier, they can you set them a research goal rather than simply to answer one question. They plan how to go about answering that question or or or producing that research. They select their own sources, so they review, uh often they perform a Google search for you and then review the the outputs and then appropriately make judgments on which ones are relevant, how they are relevant to the output, um, and then to synthesize that into a good quality output. And these many steps, they c they can go wrong at any point. So, you know, lots of us have used them know that if you don't properly define how you want it to go about that task, if you don't say who the audience is, if you don't say what what the relevant sources are if you know, then you're liable to just leave it to make those calls itself. You crash your sled. Yes. Yeah, exactly. Yeah. You're just like, go forward and uh get me to you blame the dog? Yeah, exactly. Blaming the dog. And so we do, we do actually, lots of us who who don't already work with um with kind of some of the whizier agents already have experience of this. Uh it just means that we now can start to see how the the mix of skills and and the as you say, the organizational structure might change to reflect the new way we're going to work with agents.
SPEAKER_01I think we can say that it will change. We now have this new workforce that has a vast capacity for information processing. That means organizations need if they wish, if they if they want to um I I guess prevail in in this new era, uh, will want to think about which of their internal processes they want to re-engineer to leverage this new capability. When we re-engineer processes end-to-end, we're essentially transforming the division of labor. Because right now it's which human do we give something to? Uh, but in the human agent organization, it's you know, what's the division of work we give to the humans and work we give to the agents. So I see an org chart that is not just a map of people alone. Um and at the moment we talk about span of control. Uh you know, I think we'll soon be talking about span of agency. And when the structure changes, the skills that we are going to value will change as well. So uh a human role um you know may shift from just producing good work to actually supervising good work to I identifying what is a good use case to designing it. uh to crafting it, to make sure you break it down, make sure you've thought of all the do's and all the don'ts, to proving it, controlling it, overseeing it, and then selecting the best output and and and maintaining your your judgment. And then all of this is going to result in uh, I believe, a um an interesting inflection in culture where we're gonna start talking about, you know, how, under what circumstances do we trust and and challenge a machine?
SPEAKER_00There's been a lot of hand-wringing as well about what this shift might mean for, say, junior workers coming in newly into the workforce and and hearing you describe those new skills there, it does strike me that there's clearly a role for junior workers in this. You know, pe people who imagine that the manager can just define, okay, this is what the agent should do and off it goes and now we don't have junior workers is missing all the points you made there about the importance of ongoing monitoring, you know, making sure it's appropriately bounding, understanding how the AI works, knowing what good looks like. I mean when when I think about my deep research agent, you know, now potentially because the agent can do lots of research and synthesise lots of reports, you know, I have the opportunity to learn from say half a dozen reports as to what good looks like. What did I think was good about that report versus that report? Bring it together, you know, because we're not we're not in g going to go straight to a world in which, you know, an omniscient manager can just put in the perfect prompt and get get exactly the output they want or design the perfect agent that delivers the outcome. It's going to require people who learn the the new skills of of agent uh design, agent monitoring and and also learning what what those good outcomes are and how an agent can achieve those.
SPEAKER_01That's exactly right. Yeah at the start of our careers we're we're gonna we're gonna need to be agent handlers managers will need to be managers of handlers and and seniors will need to spend their time on the division of labour because the the unit of strategic capability is going to shift from just headcount and time to uh the quality of supervision and the quality of of that uh of that workflow.
SPEAKER_00Yes. Qual quality I think is the the term that that captures it, doesn't it? It's sort of if I if I can get an agent to produce me three research reports, you know, I I still need to cultivate the skill of understanding what's what's good in each of them or what what meets the quality mark that we're trying to achieve.
SPEAKER_01Yes. And then a next question if I were an HR director would be well we already have ratios and theories about how many staff, a manager, what their capacity is how many agents can an agent owner oversee whilst maintaining quality without becoming too robotic themselves? We how will we overcome automation bias where we just approve because we overtrust the machine?
SPEAKER_00Yes and we we haven't even got onto the um the vision that some people have at the far end of entirely agent run uh businesses but I think I'm sensing you might have a a note of of skepticism to sound on that point given the role the essential role that you think humans still play in in organizations.
SPEAKER_01Yes there will be humans will play an essential role in organizations, particularly regulated organizations and and that's where I'd probably like to land my final message. But before I do, for those who've been watching closely Argentina has recently allowed the agent only legal entity I I'll go out on a limb. I think it's a mistake I think we should not grant AI legal personhood because we have already had instances where sadly AI has resulted in the worst possible outcome for for a human. And in those instances typically the families want comfort that it can't happen again and just being told that the agent has been turned off is no comfort when another agent only organization that has no incentive, has no ethics has no incentive it can't have its bonus cut it can't have its reputation tarnished it can't become unemployable it can't get taken to court and thrown in prison. All of these ethical controls that we have over humans they they do not apply to agents. So let's bring this um to a hopeful conclusion because it's not going to be an agent that replaces me Adam Granger it's gonna be someone else who has built a an agent that will do that. So I message to everyone listening be to overcome natural fears and trepidation. That Lawrence you know when we when we spoke earlier over lunch we both felt I think it's natural to uh you know um to have those feelings overcome them and learn learn learn and you're only learning when you're making mistakes and learn from those mistakes we mentioned about red teaming finding failure is now the only way that you should the only reason why you should deploy an agent because you know how it fails. And my last point is no regulator's ever going to want to accept accountability from an agent. It's always going to accept accountability from a human. So be pleased that you'll be learning about agents uh in a regulated industry uh because uh it's always gonna need humans.
SPEAKER_00Yes um and I think the the s scene that that conjures in my mind is one of if we all have our own um agents helping us, our our own working dogs say, then that naturally causes us to start to think about okay so what um what does the social infrastructure need to look like to make that work? So you know the the bus and the train would need to be designed differently if everybody got on them with their their guide dog or their helper dog. And in some ways I think that that what we're going through now as organizations is a process of as you say learning what what's out there, imagining what the future might look like and trying to build towards that future because it's one of of enormous possibility if we get it right.
SPEAKER_01You know we can do we can do much more and deliver a much better service with with um with people and and their AI agent helpers essentially but it's not going to happen overnight we're developing an entirely new capability don't just build agents build your own agentic capability deploy them when you're ready to control them.
SPEAKER_00Otherwise and the data's already out on this you know um if you try and run before you walk uh you know you'll trip up um and on that message of hope I I'm very hopeful uh particularly for uh this particular audience I I am too uh I think we we have to build it and get there and to the listeners um I thank you for listening I will include in the description the link to Adam's agentic risk framework I I think it's extremely good reading the dog's analogy is there and it's it's also lots lots of really practical insights that uh readers and listeners can can use to help to start to think about how they can deploy agents in their organizations. So so please do visit that and um get in touch with Adam with any further questions. But otherwise just to thank you all for listening and join us in the next one.