Stop Letting Your AI Agents Off the Hook

Rhys Fisher, Associate Editor at CX Today, is joined by Dave Rennyson, CEO of SuccessKPI, for a candid conversation about what it actually means to run a contact center where humans and AI agents work side by side

As agentic AI takes on a growing share of customer interactions, Dave makes the case that the standards we apply to human agents must apply equally to AI, and that the organizations skipping that step are already storing up problems. Honest, specific, and refreshingly free of hype.

The hybrid contact center isn't a future concept, it's the reality most operations leaders are already navigating.

Dave Rennyson breaks down what it takes to manage it properly.

AI agents are handling the simpler conversations, which means human agents are increasingly left with the harder, more complex ones. Dave argues this demands more recognition and support for human agents, not less.

Quality management for AI agents is non-negotiable. Treating AI like a self-regulating system is, in Dave's words, like letting your builder do his own home inspections. Third-party QM, whether manual or automated, has to sit on top.

CSAT isn't dead, it's due a rebirth. Dave makes the case for using AI to score every conversation on a normalized scale, cutting through the "haters and lovers" bias that has always made survey data unreliable.

"Good" in a hybrid contact center means managing the whole. Dave's vision is workforce decisions made minute-to-minute, based on task complexity, live performance data, and force-to-load, with humans and AI operating as a unified system.

Transcript

SPEAKER_00 0:08

Hello and welcome to CX Today. I'm Reese Fisher, Associate Editor, and today I am delighted to be joined by Dave Renaton, the CEO of Success KPI. Dave, how are you doing?

SPEAKER_01 0:21

Doing great. Thank you. Thank you for having me here today.

SPEAKER_00 0:24

No, absolutely. Thank you for joining me. You know, we've got, I think, a really interesting topic lined up for today. We're going to be discussing the dynamic and I guess the management challenges that come with human and AI agents working side by side in the contact center as they do in this modern situation. I guess leading on from that, you know, we know that AI agents are now handling, you know, real customer interactions at scale. From a management perspective, what's perhaps fundamentally different about overseeing a workforce that is part human, part machine?

SPEAKER_01 1:02

I I think the first part is that many of the tasks that are being handled by the conversational agents, the agentic uh bots, are probably some of your simpler conversations. And so you're dealing with a process of trimming and pulling in things that are easy to handle with the bot, and you're building that bot's capabilities up. So what's left for the human agents are actually the harder bits of work. I think the first part about this is having to understand what it is that is in those harder conversations and having some grace for the agents that have to deal with these much more complicated long-tail transactions.

SPEAKER_00 1:45

Yeah, yeah, makes a lot of sense. I wanted to ask you next. I think I think it's an area that perhaps doesn't get explored a lot, but I think it's very interesting. And that's kind of quality management when it comes to AI agents. You know, how I guess first of all, what when an AI agent underperforms, how can you even tell, I guess? And what does quality management look like when one of your agents is, you know, obviously an autonomous system?

SPEAKER_01 2:12

You know, I think if you're going to take the leap, then I do take this leap, that conversational AI is able to have a much more durable, thorough, and consistent conversation on par with humans for certain types of tasks, then you need to be managing and quality managing these conversations the same way. So if you think back to where we were prior to automated quality management, you know, supervisors are responsible for scoring a certain number of calls. We all have to do this in contact centers, you know, get your 25 calls done a week or whatever the assigned task was. When we moved into the automated quality management era, you know, best in class contact centers still had their supervisors doing a certain number, maybe eight or 10 instead of 25, and then reviewing the hundred, two hundred, or a thousand that the automated quality manager management system done, did calibration sessions to bring all that together, and then presented a unified set of data to your agents. I don't think that managing a bot is any different. If you don't have a quality management program that's a third party sitting on top of that system to validate and vet its performance, uh, you're gonna have a problem. You know, it's it'd be the equivalent of letting your builder uh complete his own home inspections, right? You know, eventually at some point you want to make sure that there's a third party checking out that things are going right. And that can take the form of manual review by uh by the contact center uh leaders or systematic review uh using third-party automated quality management, manual quality management tools. But we will have to be managing these bots similarly to how we're managing humans. And you're not gonna be able to sit down with the bot and have a one-on-one conversation to make them better, but you can provide feedback loops to the designers and those that are working with it to make them better. And the good news is consistent feedback to that bot will affect all future conversations in that way. But you better be right because whatever you change is gonna change all the conversations as well.

SPEAKER_00 4:11

Yeah, it's interesting they'll kind of treat in these AI agents like human agents, and you touched on this a little bit, but there is are there any differences in terms of that management or overviewing of the two agents in terms of how you do deliver or that quality management?

SPEAKER_01 4:29

I think there's a difference in the data that we can get. Many of these advanced systems can govern data exhaust equivalent to um you know utterance failures, no input, no match type failures that we've seen in IVR. Uh similarly, when the agent is having trouble, it does have the ability to eject data that will allow you to manage it more tightly. Um, these things can be used in real time to affect call flows, or they can be used in near-real time after the fact to guide improvement to the bots. Uh, but again, a systematic approach to how you tackle these problems is critical and again, in my opinion, done best from a third-party point of view. Let the bot builders build the bots, let the quality managers build the quality tools.

SPEAKER_00 5:10

Yeah, yeah. I think uh that's a good way of looking at it. I guess leading on from this idea of kind of some of these traditional ways of of overviewing the contact center with QA, you know, contact centers have also relied traditionally on surveys, on CSAT, on MPS, you know, to really understand that customer experience. In a hybrid world where I guess AI is handling more of these interactions, do you think those measurements still hold up?

SPEAKER_01 5:40

I do. Um, in fact, I I'm not a big believer that the CSAT is dead. Um, there's something great about some normative scale, either one to five or zero to ten for MPS, etc., that allows you to break things down. And in fact, using AI tools, um, they can be exceptionally good at taking a million phone calls and rating each of them on a scale of zero to ten, both human and agentic, and finding areas of improvement for those that went well, pointing out what was great, for those that were not, you know, what could be improved, and then summarizing that data into a new view. So I think we're actually in an age where there could be a rebirth of these C SEP metrics, but done using agentic tools, which would get a better scan of all the phone conversations than you would get if you just waited and hoped for your top 10% of happy people and bottom 10% of unhappy people to provide a score.

SPEAKER_00 6:38

Yeah, yeah, like uh you should try and um coin the phrase the rebirth of CSAT, I think. I think that could uh a lot of companies would like to use that. Um I wanted to come back to this kind of human AI debate almost. Sometimes something that we often read is the suggestion that maybe there could be a two-tier system emerging there, you know, with kind of that traditional rigorous oversight for human agents, but maybe less visibility for the AI agents. How does a company guard against that?

SPEAKER_01 7:12

I mean, I I I think ultimately, you know, I wouldn't guard against it. The amount of AI that that is right for your business is is based on your business. And you have to study which transactions are meaningful and easy to do at a reasonable level of customer experience. You know, I don't think anybody minds calling into a hotline that quickly gives you the bank balance, and they're not upset that they didn't talk to a human. In fact, if they could get automatically authenticated by their phone number and an SMS push to their phone and then quickly get the balance, make a trade, do whatever the simple tasks are. No one's complained about that you know forever, right? For two decades, we've been doing that effectively with IBR. The trick is to make sure that you're dealing with the right conversations. Things that are more sensitive, that require human touch, things that um you know are going to eventually get to a human anyway. Like, don't try to automate something by half and then say, okay, great, now let me pass you to the human who's going to finish this. That's a frustrating moment for a lot of folks when they've exhausted themselves doing simple things and you've forced them to work for you in order to make your efficiency go up by 20%. I'd rather you take the task that can be fully automated and then figure out a graceful way for humans to do it. And let's not forget that the data dips and the ability to bring information into a conversation can be just as powerful as anything the machine can do on its own. I do hope that we don't lose all of the art and uh and craft in the work that we've been building for the past 20 years to make these conversational uh AI work better. You know, now that we have a better way of consuming information and a machine that's more adaptable and flexible, we should use it to make it better than human, not just to create something that is you know simply a little more automated.

SPEAKER_00 9:08

Yeah, yeah, I really like that phrase of kind of not forcing the customer to work for you. I think that's a really, a really nice way of thinking about customer service.

SPEAKER_01 9:17

I think that's and James, sorry to interrupt you, but you you feel that, don't you, when you get that conversation? Like you know, and they're making you do the work, and it it does get a little frustrating, right? Oh, yeah. People don't want to feel that. That's not a great CX.

SPEAKER_00 9:31

No, no, completely agree. I and you know, speaking of of great CX, my last question here, and you know, it is a little cliched, I guess. I do apologize, but I think it is always worthwhile asking, you know, what does good look like in a hybrid contact center, and how do you how do you know when you've really truly achieved that?

SPEAKER_01 9:54

You know, it's funny, I think if people are going to have this sense and feeling that AI is taking jobs, etc., right? I think we've got a lot of responsibility here to make sure, first of all, that the agenc uh conversations are being graded, measured, and appraised as much as the humans, right? You know, I don't I don't want to be in some John Henry conversation about, yeah, we're gonna just outwork the AI. I mean, that's not the what I'm trying to say. What I'm suggesting is that many of our agents are working very hard and have worked for entire careers to satisfy and serve customers and to build up the knowledge on how to serve customers well. We need to respect that and to applaud those folks that are in these contact centers working hard and want to do a great job on those harder, more complex transactions. At the same time, we need to be measuring and managing the agenic conversations equivalently and holding them accountable and not letting a poor agenic conversation let an agent end up on the wrong foot. So, in a in a perfect world, we would put in a framework that manages performance across human and agenc in a way that it's part of a whole, and decisions are made along the lines of what's best for all of our customers today, this minute, this five minutes, etc. I think we'll also even get even more sophisticated where our workforce management decisions are made based on force to load. You know, maybe the agent's a B plus on this task and the human is in an A plus, but it's 1030 on Monday morning and I'm awash in the same call as I was awashing for the last two decades. Maybe I let the agents step in and pinch hit during a difficult time in the force to load matrix. On the other hand, it's 2.30 on a Thursday afternoon and the load is waning. I've already paid for several human agents to be here. Let them bring that exquisite level of care that they can at a calmer time in the contact center and maybe turn automation slightly down to find the optimal use of the resources we've already expended. So I think there's going to be an interplay based on task, an interplay based on surveillance. And it's going to take really sophisticated quality management tools to be able to make these decisions, not just on a month-to-month basis, but on a minute-to-minute basis in a properly managed blended contact center, you know, next year and in the next two to three years ahead.

SPEAKER_00 12:24

Yeah, thanks for that, Dave. I think yeah, it's been a really, a really enjoyable chat. I think inevitably that human AI discussion is always quite a delicate topic, but I think it's been some really some really insightful and really honest discussions around how that can coexist and you can kind of get the best of both worlds. So yeah, thank you very much for your time.

SPEAKER_01 12:45

Thank you. I enjoy being here.

SPEAKER_00 12:47

Great. I'd also like to just quickly thank our audience for tuned in as well. If you enjoyed this, and I'm sure you did, please do remember to like and subscribe to the channel and head on over to CX.com for more stories like these. Until next time.