Why Judge LLMs Matter for AI Guardrails in Debt Collection | Ep. 6

Can you trust AI without someone watching it?

Karan Sood of EXL explains why the future of enterprise AI isn't about building bigger models, but smarter systems. Learn how Judge LLMs, multi-agent AI architecture, and prompt guardrails work together to validate responses before consumers ever hear them.

From AI models evaluating other AI models to modular system design, this conversation explores the architecture that enables compliant conversational AI.

Listen now and subscribe for more Applying AI episodes with host Adam Parks and co-host Mike Walsh.

Applying AI Podcast:
https://receivablesinfo.com/applying-ai/judge-llms-ai-guardrails-exl-karan-sood

EXL:
https://www.exlservice.com/

Karan Sood on LinkedIn:
https://www.linkedin.com/in/karansood7/

Mike Walsh on LinkedIn:
https://www.linkedin.com/in/mike-walsh-b88b271/

ai guardrails for debt collection,
judge llm architecture,
compliant conversational ai,
intent classification for ai agents,
multi-agent AI architecture,
customer intent recognition,
ai governance,
enterprise ai,
debt collection technology,
applying ai podcast

#AIGuardrails #CompliantAI #AIinCollections #DebtCollection #EXL #KaranSood

Transcript

SPEAKER_01 0:00

Hello everybody, Adam Parks here with another episode of Applying AI. Here with my co-host, Mike Walsh. And today joining us is Karen coming from EXL to talk to us more about well, keeping your AI voice and written and communications from an Agenic AI standpoint on the right tracks, making sure that your compliant systems from a technology perspective have the backbones they need to keep you safe, be productive, and start actively being able to deploy these types of artificial intelligence tools. So, Karen, thank you so much for joining us today. We appreciate you coming on, sharing your insights. You know, I'm getting to meet you for the first time today. So could you tell everyone a little about yourself and how you got to the seat that you're in today?

SPEAKER_00 0:47

Absolutely, Adam. First of all, nice to meet you. Glad to be here. Um, I lead the AI solutions and product business at EXL. Uh EXL is a data NAI-led company, which it's uh New York Stock Exchange listed since and we are 27 years young. And the reason for that is the space is changing so fast that we keep reinvending our organization every few years. So we're gonna stay young. Uh I uh as part of this, as part of my role, I have a series of solutions, AI solutions, which sit across different workflows. Uh, one of those workflows is collections, uh, where we actually started focusing on this uh about 2018-2019 before before generative AI was cool. Right? So this was this was based on the earlier machine learning models. And given we were already, I would say, a few steps ahead. So when the AI came uh you know roaring in 22, 23, we were the first one to adopt that and uh take it to the market and production as it and deliver value. So that's one of the portfolio. Uh previous to this, uh previous to this role, I was CEO of Godridge's North America business, where I uh ran the country operations as well as its transformation for uh the global business. And that's how I made a switch uh and jumped more deeper into AI in 23.

SPEAKER_01 2:08

And very focused on the product side of the business now, which I think is very interesting. So now that we've got a little bit of understanding of your background and in the organization itself, you know, talk to me a little about the solution, the main solution that you're providing to the collection space. And I then I want to ask and talk a little bit about how you're keeping that on the rails. Like how are we keeping this type of technology on the on the train tracks?

SPEAKER_00 2:33

Excellent. So our solution is called paymentor. And uh the the core thesis behind this is very simple. The core thesis is that for any collections, you need to solve for two problems. You need to solve for customer engagement, which means you need to reach to the customer at the right time, right time of the day at the right frequency. Once you reach the customer, you need to have, you need to persuade them to pay, right? Those are the two axes with which you fundamentally the entire collection's operations run. Uh so with our solution, we reimagine both of these axes, we make it better. So we improve the customer engagement through our AI models, which are dynamic in nature and which at every interaction predict what's the next best channel at the time of the day, the right frequency to contact this customer. And unlike collections operations, which run this static basis, saying, Okay, contact everybody on day four, what we do is you know, we we iterate, we do this on a more much more dynamic basis by overlaying the risk and the behavior segments of the customer. And the second thing we do really well is we leverage AI for persuasion. So this is where you know the humanity comes in, right? You have to understand the context and you have to be able to converse with the customer. You have to, in some cases, you have to empathize, in some cases, you have to explain, in some cases, you have to explain to them what are the consequences of non-payments are, right? Within the crowd. So with AI, with specifically generative AI, where the speech has become much more human-like, and our solution, you know, some sometimes people cannot distinguish between whether it's human calling or AI calling. So with that technology, we've been able to significantly improve the persuasive power of uh of collections and combining the two, which is better strategy, better outreach with AI net persuasion, is fundamentally delivers uh higher collection. So that's what we've been doing. We've been doing it for about 20 plus clients globally across industries, and it's it's delivering some significant results.

SPEAKER_01 4:32

Well, so many of people in our audience equate artificial intelligence to their you know instance of chat GPT, right? Like that's that's I think a lot of how people are viewing it, and so they're pretty used to some crazy hallucinations. And I believe it was one of my lawyers who said that ChatGPT is like a drunken frat boy. It's gonna be very wrong, but it's gonna be extremely confident. Well, it's as wrong as it is, right? So I I think that there's some challenges there, and with those hallucinations comes fear. But when it comes to deploying these types of things at scale in the wild, it's not just about deploying one model that's doing voice, right? It's a series of tools that are being tied together. The term that I've heard used before is judge LLMs. One LLM is judging another LLM is judging another LLM, and different models are basically feeding different pieces of the equation to the whole math problem. Talk to me a little of how you approach that as an organization from a product design perspective.

SPEAKER_00 5:36

Absolutely. So most people are used to AI in voice, right? And that's what they refer to it. But we take a step back and say, you know, we also use AI in chat, right? So think of the common braid uh and think of either chat or voice as a channel of communication, right? So when we think about and when we actually deploy AI, what is a given and what is non-negotiable for us is that it has to not hallucinate. So and for that we put a series of guardrails which catches the errors before it actually is communicated. So here's how here's a few techniques how we do it. In some cases, um, so if you think about the technology, the technology can be fundamentally broken into three components. Let's talk about voice. You have a component which converts what the customer talks into text. That's the first component of the technology. Um, typically, we use the best of the best of the technology out there, which gives about 95% accurate word translation, right? Uh and about a year or a year and a half back, this was about 85%. So this technology is gonna keep improving. So the input which goes into the model as next step is improving so that as data becomes better and cleaner, model output becomes better. So that's why. The second step in this journey is where the model comes into play. This is where the LLM, the brain of the solution, is. But we don't just pass this text into the LLM directly, we actually have a series of prompt guardrails built into it, which get paired with the customer's response before it is fed into the model. So model automatically gets the input with series of guardrails built into it. Once obviously, model does his magic of predicting the right answer to that context. And because these models have been trained by us on the collection specific workflows, so these models also give a just a generally a better output versus the standard off-the-shelf model. So that's the other thing we improve. So one, we bring in the guardrails at the time of prompt, two, we fine-tune the model so that the model is just generally more accustomed to giving collections relevant and compliant answers. And then once the answer is generated, that's where we have another LLM as a judge to figure out whether the answer which is coming out before it gets communicated is meeting the right set of guardrails and the checks and balances we have put in place. That's then gets converted into uh what we call text-to-speech, where we use some of the leading players like 11 labs, which fundamentally converts that text into almost like a human sounding voice. So while the while the core heart of the component is LLM, and if you didn't have all these guardrails and prompt libraries built around it, you could have a hallucination which could completely run a mock, and as you said, it'll be very confidently wrong. But we we you know we put in place these prompt trees, uh, guardrails within the LLM and fine-tune the LLM to control that hallucination.

SPEAKER_01 8:38

Interesting. So you you you do it by breaking it down into three components and then guardrailing each one of those components as it goes through so that there's nothing lost in translation. Now, Mike, you've been out in the marketplace talking about this product and this process, right? Because compliance has been the number one thing. What kind of questions are you hearing directly from the marketplace as you try to explain these technical uh uh tool sets to a simpler audience?

SPEAKER_02 9:09

Yeah, it well, and in an audience that has been kind of taught fear, right? Um so they they're worried, right? Yeah, and and it, you know, that's kind of the frustrating thing is they do think of the drunken chat frat boy horrible experience. That that's what they have. And I think I mean the only way to do it is to like, hey, this is different. The technology has even I don't know, Karen and I started about two months apart at EXL. We have a totally new virtual agent. It is it is it is incredible, and it's way better than what we had before, which was really good and very productive across the globe, right? So I think when you you you have to dig in, you have to demo it, you have to show these different components and break it down where you know I I think compliance is a is a like a gatekeeper in the beginning, but it's your best friend at the end of these onboardings because hey, collectors have bad days, right? Like I've seen really good collectors lose it. Um, it is a hard job. I trained that was my first train in this business, is was as a collector. It was miserable. I wanted, I'm like, hey man, I'm gonna leave if I don't, you know, you don't get me off this floor. He's like, no, no, we're gonna switch you to a new type of client. So you know what that but you realize that it is such a tough job. And these people, meaning our customers, our friends, our neighbors, they don't want to talk to a collector, right? Like, so this I think this tool gives them such a better option, private, clean, easy, at their own pace, at their own time. They have so much more personalization through AI. It seems crazy, but it's true. That I think once you go through those steps, yeah, go ahead, I see you want to do it. Finish my thought, Mike. Well, once you go through those steps and you see that this this solution or any solution, you know, is there to help and make it better, more efficient. I think you get through it, but you have to go through some great questions, you know. Uh, you know, how much how much PAI are you do you need? You know, all these different steps that you know we're used to. I mean, you I and I encourage people to ask those questions. Get them all out. You know, there are no stupid questions. If you three years ago, I I was brand new to this technology, and now you know I live it and breathe it every day.

SPEAKER_01 11:52

I I now you're co-hosting a podcast about it.

SPEAKER_02 11:56

Yeah, yeah. But I still do like you know, a lot of talks where we started at stage one, and and I think it's great that people are people are all, you know, even I worry about this podcast when are we too far ahead? Are we too far behind? Like it changes so fast, as Karen said before, that people are in all different stages. So you get a ton of different questions, and I think people are learning fast. There's a lot of information out there.

SPEAKER_00 12:25

I'll actually say something. You should select the partner not based on the technology. You should first select the partner based on their experience of collections. See, because technology is gonna keep improving, right? So you need partners who are flexible, who bring the best of grief to you. But technology is only as good as how you use it. Yes. And a technology in the hands of people who have never done collections or who don't understand this deeply is what results in hallucinations and all those, you know, complaints, etc. If you give it to the people who have done it, Mike has been here for 30 years, some of our team members have been here. But besides the point, if you give it to the teams who've been doing it for a while, who understand the pain points, who understand the process, and then bring the best of technology, that's where the magic happens.

SPEAKER_01 13:11

The technology doesn't change the learning curve for the business itself. And so that experience of being able to build it out. And and Kira, Mike and I have talked about this at a few different shows. If I told you how many voice AI companies reached out to me in the last 18 months looking to enter the US marketplace from all over the world, mostly I mean Africa, Europe, whatever. And every time I ask them two questions, who's your lawyer, who's your compliance? Because unless you unless if you don't even have an answer to those two questions, then there's nothing that I can do to help bring you to the US marketplace because you're a threat, not an asset to the industry if you don't understand those compliance guardrails. The technology will continue to evolve.

SPEAKER_02 13:52

The tech will change rapidly. Right? Like part of what Kern said is like it's gotta be adapted to your procedures and policies, right? Like it's yes, you know, that you want to let the tech go, you don't want to control it, right? Like, but it's gotta say, hey, this is our policy. How do you do it for this client versus this client versus in this industry we serve? Utility is different than medical, right? Like, so it's gotta be adaptive to and you know, agencies, they'll have government, they'll have credit card, they'll have you know medical, right? All the same agency. It's gotta be able to do all those things, otherwise, you're just buying a headache, right?

SPEAKER_01 14:35

So make sure that all those consumers are different, right? The balance ranges are different, the intent is different, the approach is different. And e each one of those you have to have something that's flexible enough to be able to accomplish these things. But it sounds from the other conversations we've been having, it sounds like a big chunk of the change. And you've mentioned that you're on a totally different agent than you were when you first started called three years ago, and now you guys are onto something new. One of the shifts that I've seen happening, and I I think you guys are are kind of leading the charge with it, is that push to intent driven. The old agents were reacting and responding to things, but the new agents are trying to understand the intent behind the statement, which is where I think there was the biggest differentiator for the human versus the bot communications. But the more that we can understand the intent of what the consumer is trying to say, the more value we can ultimately provide with these tools. And that intent, I would expect, changes culturally, whether that be regional, city-based. I mean, that's got to be a giant spider web in and of itself as to how to predict that intent. And I'll use a silly example, but coke, soda, pop, right? Like they're gonna use different language for it in all different areas just of the United States itself or even within an individual area. Um what does that start to look like as you're deploying tools like this on a global scale? How deep does that web go? And how different has the business become as you've started to better understand the intent of the speaker?

SPEAKER_00 16:21

The the beauty of these language models, specifically the ones we're coming out, they've been trained on uh data. I can't even you can't we can't even imagine the amount of data they've been trained on. So, one thing which these models do really well is they understand the nuances of language. That's fundamentally what has been improving on each of these models. Obviously, the reasoning is improving. That is basically like uh uh the way they write the algorithm that is improving to make the model reason more. So I think that's one thing. As models improve, the context understanding of the conversation or the context understanding of what the customer is saying is becoming better. But that's just half the story because there are still gonna be nuances which model will not understand, right? And that's where our collection experience comes in because we take the intent classification and then you bring in our expertise of that classification and then do the do the reward or the answer, the answer the customer. So I think that's where the partner you choose has to be experienced enough to understand the nuances of intent. And I think, and actually, I'll tell you one step further. If anybody misclassifies the intent, the entire AI journey goes off the track. So I would actually argue that's the single most important point of differentiation on how accurately you can classify the intent of the customer.

unknown 17:41

Right.

SPEAKER_01 17:42

So when we think about classifying that, and go ahead, Mike.

SPEAKER_02 17:45

Yeah, it doesn't matter how good your message is if it's the wrong message, right? Like we've all been on the call, and you're like, I'm not asking for that, you know. Like, so especially in debt collections, where you have someone who's not thrilled at the process, right? Like it's a customer who's you know got a problem. I think it's hypercritical that the understanding of the AI is data. And and I think that's where collection experience is huge and quality of tech, those two things married have to marry to make it make it really productive for you. Sorry, yeah.

SPEAKER_01 18:23

No, no, no. I think you covered it well. You know, I I started thinking about when we're looking at this intent, you know, that's where so much confusion can exist and live, is in a misunderstanding of intent. And we can't afford to have anything going off the rails being misunderstood and being able to bring it back. Now, when we talk about stacking these models on top of models, is intent its own model or is that handled as pieces of the other models? But as you've gone through that evolution, what does that look like from a technical perspective?

SPEAKER_00 19:01

Best intent classification models are machine learning models.

SPEAKER_03 19:05

Okay.

SPEAKER_00 19:06

Right? Because you don't want hallucination at the time of intent. So when the speech gets converted to text, that's where you have a specific intent classification model, which are pretty good, which are pretty accurate these days, uh, which fundamentally starts segregating the journeys. But here's the other bit. Best of the models will also make mistakes, right? The problem happens is when AI is not able to track back and go to the right intent. That's where the next problem comes in. So first one comes in in wrong intent classification, right? Which, you know, customers we have seen are still okay if the AI walks back and talks the right language, immediately admitting the mistake. So I think that's where the second thing comes in, where your AI bots need to be flexible enough to go back to the starting point of the journey and then in and then reroute it to the right intent and then trigger the journey again. So I think those are the two bits which which have to go hand in hand. But I think the intent classification models are becoming very, very, very, very accurate.

SPEAKER_02 20:04

So what you're saying is that if sorry, I'm gonna cut you off again. That's the same thing. No, go ahead. But I think that's I I think that's the drastic improvement too, right? That flexibility current and the scalability. Like if it used to be, you know, maybe two tries and send it to a live agent. Now it can that second try, if it eliminated the first mistake or the first utterance where it didn't get it right, it might have them rephrase and then boom, oh, I'm sorry, and then take that on the right track. I mean, you just if you think about how many times a human being even does that wrong, a collector, right? Like they think you wanted this or said that that's always gonna happen. And it could be back to your pop versus soda versus sub versus hero versus you know grinder. You know, it could be a collector in Massachusetts versus you know someone you know in rural North Carolina, uh uh and it's just different versus then they come to an understanding and then they go back down the track. I think that to me is the biggest difference in the last three years is it used to be boom, boom, very robotic in the understanding, and now it's way more uh flexible.

SPEAKER_00 21:17

And that also actually, uh, you know, if you think about how do customers get frustrated? They get frustrated when the bot doesn't understand what they say. That's the core source of frustration. They say, you know, correct me to an agent. And that is actually improving because bots are getting smarter, they can re-classify the intent, and they can trigger the right answers. I think that that's how it is. Evolving is gonna keep getting better.

SPEAKER_01 21:40

And retrace their steps back to the beginning. So if there's a misunderstanding from an intent perspective, the bot can go back to the beginning of that call, it can rewind, relearn, and try to determine with additional context. Whereas even as a human, I mean, you might be able to remember the call that you're on, but you're not you're not reciting it. Again, a second time. Uh right? Like it's just it's a different animal altogether with that level of context to that conversation and understanding that intent and where that consumer is trying to go. Now, when we think about these models continuing to improve, and as you you've mentioned that you've gone from one agent to another, how much of the process is moving from, let's call it, one agent to another versus supercharging one of the existing agents? So clearly there was a versioning break point between those two models where they're like, yeah, this one's as good as it's gonna be. Now we're gonna change whatever it is, core infrastructure, whatever, whatever the version change catalyst was. But what does that look like? How much does the model or how much does that agent expand before it's necessary to move to the next one and have those timelines shortened at all?

SPEAKER_00 22:59

We've actually designed it slightly differently. We we have designed it as a multi-agent orchestration for our voice bots. What that means is we don't have a single agent which is doing multiple things. Because what we have learned is that one agent with multiple goals will make mistakes more often than multiple sub-agents which are dedicated to a goal. Because at the end of the day, the agents are a goal-seeking piece of code, right? You you muddy it by adding too many goals to that agent, it is going to make mistakes and it is going to hallucinate. So that's one way we solve it. So we almost like have an orchestrated agent at the top, and we have specific agents to each specific goal, which then intelligently gets routed to, and there is obviously a connection between the agents to hand over and keep the journey rolling properly. So that's how we design it. And as models improve, we fundamentally change one component of these agents rather than having to rebuild these agents. So these are built in a modular manner that okay, if out of the let's say the tools improved or the memory has to be improved or the model has to be improved, we can take that piece out and we can bring in the new one. So that's that's that's how we've architected this solution so that we don't have to you know boil the ocean every time a better model or a better tool or construction uh comes out.

SPEAKER_01 24:25

Makes sense. Modular management, right? Be able to place the carburetor, not the whole car. Um I think makes a a lot of sense from that perspective. But what is that what does that learning process look like? How are you constantly evaluating those models to determine what is the the next best thing? Because the the new thing, the shiny thing, is not always the best solution. So how do you look at that learning process?

SPEAKER_00 24:50

See, the the you have to always balance three things when it comes to voice. You have to balance the latency, you cannot have a conversation which doesn't feel real time, right? So it'll spoil the experience. You obviously uh cannot get the accuracy wrong, you cannot hallucinate. At the same time, you can also not incur so much cost that you know it just becomes unprofitable to do this, right? So it's always the balancing of this traffecta of accuracy, latency, and cost which we have to manage. And there's also, you know, while the world is probably, I would say, accelerating at a much faster pace than ever before in terms of the new things coming out. But we also have a philosophy that if it's not it's not broke, don't fix it. Right? We don't need to use the best of the breed models today because that's just an overkill which will just give you more cost, and which is probably not the best thing. So, what we do is you know, we obviously have our R ⁇ D team which constantly monitor what are the next, what is the best model coming out, what is the best suited, what is the best use cases for these models. So we obviously have an RNG team given we are part of a larger setup, which keeps an eye out on the net new. But our production teams always focuses on the outcomes. And if the outcomes are on track, they're delivering the right outcomes. We only change something if it's if it misses the outcome we are looking for. So it's all towards delivering the best outcome rather than worrying about do I need to upgrade each component periodically. I mean, it comes down to collecting performance, right?

SPEAKER_02 26:19

Like that's also like yes, the tech is doing its job, but what is the job? The job is to collect money in a non-complaint environment, right? Like in a customer-friendly environment. And if you're doing that and it's improving, that's great. If it's missing a segment or something like that, that's when you're looking, you start tweaking. If the tweaks don't work, then you say, okay, what's wrong with this thing? And then you kick it up, and that RD team's like, oh, we can solve that, right? Like, so it is it's not just the check that like it's the strategy of collections also comes into play. You know, you can have a great model, but it's not designed for collections, but it's a big deal, right?

SPEAKER_00 26:59

So we we do update our strategy quite often, though, just to build on Mike's point, because while execution, if it's if it's not broke, don't fix it. Strategy, however, has to evolve because I have to collect more today than I collected yesterday. So which means our strategy and our intelligence modeling team, they are always on their toes to make sure that the next version of the strategy is getting released periodically to improve the performance.

SPEAKER_01 27:22

Well, and then you've you've also mentioned that each one of the tools that you're deploying is customized for that particular deployment for that organization, for that product type. What's that process of learning or teaching the model actually look like and what's required to optimize it?

SPEAKER_00 27:43

When we started entering into new markets and new industries, it used to take us, I would say, a few months to learn. But given now we have done across, I would say, like four or five industries, about six or seven geographies, we have started to see patterns emerge, uh which gets us started much faster. I think we've been able to compress our timeline to about a month. Actually, once we go live within 10 days, within 10 days, we can we can broadly 90% confidence tell you which customer has has preference to what channel in about 10 days of going live.

SPEAKER_01 28:16

Wow. Okay. And so that's um, and Mike, we've talked about what data is actually needed. It's not necessarily the PII here either. So talk to me a little about what does an agent or an agency need to provide to you in order to have a successful learning.

SPEAKER_02 28:33

Agencies are like they're they're they're kind of tricky, right? Because they're dealing with multiple, they usually go across multiple industries and multiple, you know, products. Um, they have multiple clients with different settlement payment plan requirements. So really, you know, getting that set up is part of the part of the process, but usually they've already done that with their payment processor nine times out of ten, right? Like they have up some something's collecting digital payments for them, and there's rules for each client. We just copy those, right? Like that's quick cheat. You get that up. So you you need your guardrails, you need your collection process. Um there's slight verbiage, people are comfortable with different verbiage, like a little bit of compliance, you know. Um, and then it's really you set up these parameters, and then it's connecting, connecting to get that data back and forth, and there you go. It's it's so how do you use it?

SPEAKER_00 29:35

It's just like an agrifile fundamentally.

SPEAKER_01 29:38

Yeah, okay. And but like specific, like what are they sending over for that understanding? It's just account level information. Is there any behavioral analytics being collected? You know, what are what is the what what do you ask for from that agency to be able to actually execute on that? Right? Like, I know that you can copy the the rules uh from whatever other digital channel that they have, but if you're gonna be doing this reinforced learning, it's it's just that simple. It's seven fields and and you guys are off to the races. Like, what's it really look like?

SPEAKER_00 30:10

What we generally need is just a file about the person, the person, the the contact information, uh, the loan history is typically the three fields we need on a daily basis. We will learn about your policies as a as a one-time exercise, unless there is changing, which we can obviously update. So those are the two major up, and there is a reliance on the compliance team to just sign off on the templates we already have built, so that you know your your the organization is comfortable with what we are communicating. That's pretty much it. Actually, that's our only reliance. All the behavioral signals we measure on our own because that's our solutions, I would say, differentiating factor because we understand how the customer is reacting to our outreach, and that's what the signal which goes into our models to decide the next best action.

SPEAKER_02 30:57

Yeah, like consider like a model suite that's pre-built for collections, right? It's not like it's gonna learn everything from ABC agency to get started. It it says this is agency second place, you know, 24 month-old or or let's say 12 month old after charge off. That's that's enough to prep the model. And then you you're gonna then it's gonna do a lot of uh rapid experimentation and and see what's working, what's what engagement, what time, all those different good stuff to get people to you know you're you're trying to start a right party contact through digital means, really, or a virtual agent. And that's really what you're trying to do, get the person engaged. The advantage is that can be 24-7 with the tech, right? It can't it's not it's at the customer's convenience.

SPEAKER_01 31:57

Um so you already know what patterns and correlations to look for within the data set, and now it's just about putting a new data set in there and identifying those same patterns and correlations because the the behavioral patterns across portfolios are going to be at the very least similar. Um that makes a lot of sense. How does the how does that start to work as we start deploying with these organizations? Does the learning still so we do we do the one-time learning, we've got the behavioral analytics, is there additional learning that's happening around that, or is that kind of the focal point of the compliance concentration? I think you're on mute.

SPEAKER_02 32:43

We lost you there for a second. Well, as he tries to get back up, I'll take that question, right? So yeah, it's like you're recalibrating to each portfolio, right? And or subportfolio that is an agency. Um and then once you get it, once you know, there is a lot of learning going on, right? So it's not there's reinforcement learning going on. So your behavior or your lack of behavior is going to teach um so that so it's gonna teach the machine about that specific portfolio or that specific you know agency's paper. Um I always say it's just calibrating, like it works, it's proven, it's been working for years. Now is it now there's two uh that's all right, I'll deal with that later. Now, how do we fine-tune it to make it work specifically and learn? So you're using basically the models that are there that know collections, then you're doing reinforcement learning, so they adjust on the fly to adapt to that specific portfolio is the best way. I say it's simply Karen can probably give up more technical.

SPEAKER_00 33:58

Yeah. So I don't know what happened. So uh I hope you guys can hear me now. You can now, yeah. Okay, awesome. So think of us as partners who will bring on day one a strategy of on your portfolio, and we'll say, you know what, from our experience looking at your portfolio today, without any further information, we believe this is how the segmentation of day one should look like. And in the next two weeks, we will do A-B testing to start rearranging people into different buckets based on our models, and that's how so then so we will bring a day one strategy, which is our learning across countries and markets, and which is suited to your portfolio, and then based on your data, we will continuously improve it. So then there'll be a version two of strategy, there'll be a version three of strategy, a version four of strategy, all of which will be fine-tuned on your customers' intent and how they interact with us. So that's how we iteratively improve it, and that's how we that's what we call it reinforcement model because in the back, our models are rigorously doing A-B testing. So, for example, if our strategy says send this, send this quote of customers three SMSs a week, there will also be a challenger within our own strategy which will say, okay, to 5% send two times, to 5% send four times, and see how they react better. So that's the constant A B testing which we have built in as a reinforcement loop, which continuously improves the strategy and the decisions, which leads to the next version of our strategy.

SPEAKER_01 35:27

And so putting on looking into your crystal ball, what happens over the next five years?

SPEAKER_00 35:33

Oh man, the way the technology has changed.

SPEAKER_01 35:36

I went with five years on purpose because I, you know, in five flying cars is possible in five years, right? Like the whole world opens up. But I'm curious because I'm I'm just for someone who spends so much time engaged in these tool sets, where do you see that technology evolving over let's call it two years, five years?

SPEAKER_00 35:56

I think the models will continue to get better and smarter, right? Um just just the comprehension, the intent classification, the ability to answer back to the customer, that's just gonna improve, which means the number of calls which can be fully contained by AI is gonna increase. That's one. Second thing which will happen is all the associated pieces around the models, the speech-to-text, this text-to-speech, uh, all of that is also gonna improve. So fundamentally, the number of calls which you can just contain through AI directly is just gonna be increasing as the technology improves. But I think the second axis which gets ignored is the human behavior. People also in three to five years' time are gonna get super comfortable talking to AI. You know, I'm just gonna throw it out. There could be a there could be a cohort of customers who will build their own persona of their own personal agents who would actually talk to the other agents. That is also not out of the realm of possibility, right? But I do know that the average population is gonna be much more comfortable talking to AI two years from now than they are today. And that curve will only improve and probably flatline in about five to five to seven years when enough people have as a population are comfortable talking to AI. So I think those two trends will converge, and that's what will drive this adoption.

SPEAKER_01 37:10

Well, we saw consumers' behavior change over the past couple of years as they become more comfortable with subscriptions, and that modification took about 10 years from moving from going into a blockbuster to just paying Netflix on a monthly basis, right? And accept even your car's got subscriptions now, which is ridiculous. But um, I think you're right. I think that consumer change. Now, the bot-to-bot conversation is a whole other ballgame because that's already starting. The consumer bots are already starting to exist, the debt settlement companies are creating their bots to outreach and do those negotiations, and what's that going to start to look like? I can only imagine that there will be more services in the coming 24 months that will be selling that service to consumers directly. Don't talk to collectors, we'll handle it for you, or our bot will handle it for you, and then the bots are talking to the bots and and the terminator, right? Like this is where it all goes off the rails. But uh well, it looks like a different flavor of a DSA. It's like an alternate reality. Uh, but I think it's interesting as we start looking at that consumer behavior, because if we're collecting, right, everything that we do is driven based on how's the consumer going to react, how are they going to behave, what's the next action that they're going to be willing to take. And as we look at all of those pieces coming together and we give it a little bit more time, I mean, the acceleration of use case of what called the the retail LLMs, the chat GPTs, the clods, perplexity, whatever your whatever your favorite flavor ice cream is, uh, is irrelevant. But as people get more comfortable with that, then the bot conversations start to improve and everything goes down the line. I remember the first real conversation I had with like a chat GPT just trying to learn. It was when my daughter was first born. I had no hands because I'm feeding the baby. So now I'm just talking to the models and trying to learn. But from that came some really interesting article ideas, like the eight personas of uh consumers and debt collection, things of that nature, to where it gave kind of freed up that time to think. You know, I the I I'm curious to get your thoughts on this one. And I'm I'm sure you've seen the anthropic report back from March. But do you as we think about the consumers, do you think that we're going to see that AI will have a direct impact on the collectibility of consumer accounts over the next five years or so? Mike, do you want to take a shot?

SPEAKER_02 39:33

I I I I think it will, right? Like you know, that has been around for a long, long time. And really, you know, to me, it's almost like it's gonna come down to the job market, right? That's gonna be it's always number one, right? Um if if AI takes too many jobs, then it won't be as collectible, right? Like you know, this will be more looked at as I have to use AI because it's cheaper. Um, you also have the stress that to save jobs, the government's looking to keep US jobs and US call center jobs in the US, right? Like, so I think there's gonna I think the market's changing for sure. Um I think consumers are I I think it's overblown the whole AI is gonna take everybody's job. Um I think it's gonna create so much growth, you know. And you know, I know a lot more. Uh I had a conversation last night at a baseball game. Um, my son's decided because AI, he's gonna become a plumber. And I'm like, well, he's gonna make a ton of money in Charlotte. Like all we have, you know, people just keep building here, right? Like, so he'll be busy, so that's great. Like, yeah, so maybe people look at different jobs differently, you know, and different jobs like are gonna be more in demand than they were, or or I think get more prestige than they used to have, right? Like starting your own business.

SPEAKER_01 41:06

But to me, there's each job becomes more important, correct? Right? Like as we accentuate our capabilities by augmenting with artificial intelligence, each individual job has to be more important. And in that anthropic, it was in the anthropic um report from March, they talked about how there's been no observable reduction in labor force. The only thing that has happened is new jobs at the entry level that used to be created are not necessarily being created, but nobody's being replaced. It's too big of a risk and it doesn't add enough value. There's a few reports out there that talk about the augmentation of these tools, right? Using copilots and other things to enhance the capabilities of an agent so that they can better handle those exceptions. And as the models get better, the exceptions get smaller, and you know, there's less humans that are needed for it, but that increase that gives the organizations the capability of having the capacity that they can then go out there and sell right now. They've got additional capacity for their organization.

SPEAKER_02 42:11

And I wonder if those new job creations are just because it's moving so fast they don't know where to land that job, right? Like they don't know how, like the job's gonna change and they know it, right? So do we do we launch it now? What skill set do we need? I think this is a transition period that, yeah, some of these like even think of a collection manager at an agency, right? Your or operations manager, they're gonna have to know some data analytics or find a tool they can understand that teaches, gives them what they want to know quickly, right? Like so as the roles change, I think we're adjusting to that now. And and I think it's I think they're gonna be there. It's just we have to figure out exactly what they are and exactly what we're looking for as organizations because they're gonna change and then people are gonna find great jobs that they love and are cool and they learn stuff and they're they're gonna be involved every day. And and just think of like what you've been doing with AI, right? Like, I'm gonna plug Adam here. Like, how many, you know, the index is coming out, how much how much just because of your had time to do think like you with a baby and do all the created different roles in your company because of that, right? Like, so I think it's changing, and I think this is a transition period, and I I think that was a really good app article. Um, you know, I I know you shared it with me, but I I think those entry-level jobs are just gonna change, and we just have to make sure we're training our workforce for them, or we're gonna have to do it ourselves, right?

SPEAKER_00 43:54

Well, let's make sure we're training our kids. Go ahead, Karen. No, I I I think that the workforce which is now entering the labor market is just much, much AI savvy than we are, even today, right? So as the as the work of the future evolves, more companies will need people who are comfortable with AI and can work with AI and can work on AI, right? Actually, I would say that in the in the world, about 90% of the work is gonna be people who are comfortable working with AI. It's only the 10% of people who actually work on AI, I would say, maybe less. So I think I think that's the shift which we have, I think which will which will happen pretty rapidly, that you will need just fundamentally you'll need a skill set that you know how to work with AI. And I think as people learn through that, I think the new jobs will emerge. There will be, I I do believe that there will be some industries, some areas specifically which might be more impacted than less impacted. Software engineering, for example, right? That software engineering has just in the span of three years, has gone from the hottest, uh, you know, hottest market to you know the most stagnated market. But even today there are more software engineers than they were four years back.

SPEAKER_02 45:07

Right. Pace of growth, right?

SPEAKER_00 45:09

Yeah, the the pace of growth obviously has tapered, right? Uh it'll create more issues later because you know, if you've never coded and something goes wrong, I think you have to then diagnose it. So I think there will be some learning curve over the next three to five years. But I think 10 years out, I I think it's gonna be like internet. Like people know how to use their force, people know how to get to the internet. I think is AI is gonna become like internet, where it's just without that, you're not gonna function.

SPEAKER_01 45:33

I think it's a pretty fair statement. And gentlemen, I can't thank you enough because every time I have a conversation with somebody from the EXL team, I walk out of it thinking about my future, the future of the industry, and really the direction that all of this is going, because we're there's no way to avoid it. And those organizations that are not deploying the tool sets are going to find themselves behind the eight ball. And you know, even though the cost of computing power is is getting less expensive, being a first mover allows you to get out in front of challenges, go through the organizational learning processes that you can't shortcut, no matter how cheap the production power becomes or the computing power becomes. We're never going to get to a point where you can just magically learn as an organization how to deploy these tools and the nuances of making them successful within your organization.

SPEAKER_00 46:27

Very well said, Adam, that the learning of can only be you can only be learned by actually doing it. So I completely agree.

SPEAKER_01 46:34

Fair. So for those of you that are watching, if you have additional questions you'd like to ask Karen, Mike, or myself, you can leave those in the comments on LinkedIn and YouTube, and we'll be responding to those. Or if you have additional topics you'd like to see us discuss, leave those in the comments below as well. And hopefully I can get Karen back here at least one more time to help us continue to create great content for a great industry. But until next time, gentlemen, thank you so much for your insights today. I really do appreciate you.

SPEAKER_00 46:56

Thank you so much. Thank you, Adam. Wonderful to be here. I'm looking forward to coming back.

SPEAKER_01 47:00

I look forward to it as well. And for those of you that are watching, we appreciate your time and attention today. We'll see you all again soon. Bye, everyone. Bye bye.