302: Using AI to identify coaching opportunities; with Rob Dwyer

Most coaching goes wrong before the coaching conversation even starts. If we only review a tiny sample of calls, chats, or tickets, we end up “coaching the seventh hole” and missing the real issue that shows up earlier in the customer journey, like unclear expectation setting or a pattern that repeats across channels.

I’m joined by Rob Dwyer, Senior Technical Account Manager at Level AI and CX Executive in Residence, to unpack how AI can identify coaching opportunities without pretending to replace leadership. We use a surprisingly useful golf analogy to explain the core problem: agents are “playing” all day long, on different levels of difficulty, while supervisors are juggling escalations, PTO, reporting, and intraday chaos. In that reality, humans cannot reliably watch enough work to spot true patterns, but AI can, because pattern recognition is exactly what these models do well.

We also get practical about what “signals” can look like, including an omnichannel example where an agent sounds fantastic on the phone but struggles with written communication in email and ticket replies. That’s a hidden coaching gap unless you’re analysing conversations across voice and async work. Along the way, we talk about using AI to highlight strengths so recognition is specific, and how this can help newly promoted managers calibrate quickly between foundational coaching and skill refinement.

If you want coaching that feels fair, evidence-based, and actually improves customer experience, listen now, then subscribe, share this with a fellow support leader, and leave a review with the biggest coaching blind spot you see on your team.

Transcript

Welcome And Why Coaching Misses

Charlotte Ward 0:13

Hello and welcome to episode 302 of the Customer Support Leaders Podcast. I'm Charlotte Ward. Please welcome Rob Dwyer to talk about using AI to identify coaching opportunities. I'd like to welcome back to the podcast today. Rob Dwyer. Um, Rob, it's lovely to see you after a brief interlude, I think of about a year and a bit-ish-ish. Um lovely to see you. Ish. Um, for the benefit of uh new listeners and returning listeners, would you please introduce yourself?

Rob Dwyer 0:51

Yeah, absolutely. Um I have been in the contact center and related space for roughly two decades. And currently I am a senior technical account manager with Level AI and recently named CX Executive in Residence, which is a fancy way to say, I guess I know something about uh this industry, and I spend my time talking to people who are still doing the work and to our product team to try and make our products even better.

Charlotte Ward 1:29

That is that is a little bit fancy schmancy.

Rob Dwyer 1:32

I know, I know.

Charlotte Ward 1:34

I don't know. I don't know what bit of that title gives it the fanciness. I I feel like it might be the in residence.

Rob Dwyer 1:44

I think so too. And you know what? I don't put a lot of stock in titles. In fact, when I was coming aboard level AI, my boss asked me uh during the interview process if I was concerned about coming in because I had previously been uh a VP. And I said, you know what? I don't care about the title, I care about the work that really is what's important.

Charlotte Ward 2:08

Correct answer for all people everywhere speaking at any level of you know, career progression or interviews or anything in between. I don't care, right? But but I want in residence in there.

Rob Dwyer 2:21

I feel like I did not ask for this. This was brought to me, and I was like, well, that's pretty cool. I'm not gonna turn that down.

Charlotte Ward 2:33

I'm sorry, I'm teasing, uh, but I know that you're you're here for the teasing primarily. Um right. Um, I'm sure there'll be plenty more to come over the course of the next half hour. But uh nonetheless, thank you for joining me. It's lovely to see you again. Um uh you've guested on my podcast before, and I have guested on yours, but times they are changing. Um, we are here to talk a little bit about AI, but with a particular um topic in mind, uh, about uh about using AI to identify coaching opportunities, not necessarily strategies, not doing the coaching, not not running the coaching, but but identifying those opportunities. So let's begin with then. Like, what does that actually even mean? Like,

The Golf Coach Problem

Charlotte Ward 3:20

explain it to me like I'm five.

Rob Dwyer 3:22

Well, I think we should start by just talking about what the role of a coach is in a contact center or support center. And I will, I will I like to make a delineation between contact centers and support centers because there is a difference between handling synchronous and asynchronous and kind of the teamwork that happens depending on which of those that you're in. But my experience primarily has been in the contact center where it's voice, it's chat, and everybody's working at the same time. And I love sports analogies, but I think they tend to break down when you start talking about coaching most of the time. But recently, I have thought, you know, as a supervisor, if I have, let's say, a team of 15 agents, it's almost like they're all golfers, and I'm their golf coach. But they golf every single hour of their shift, and I'm doing other things, and maybe depending on my organization and how we have things set up, I get maybe half an hour a week or every two weeks, or god forbid, every month. That's my time to coach them. But did you know there are like I don't know, 300,000 golf courses? And a golf course is like 18 holes, right? And so if you think about how many interactions an agent has, they might be uh but like one per hole is an interaction that they're dealing with a customer, but a hole is unique in that they typically actually set the pin in one of nine different spots on a green every day. They reset where the where that cup is. Yeah, really.

Charlotte Ward 5:19

That's not where I thought you were gonna go because I was just thinking, well, there's you know, for a par three, it's like three shots, so you might there's a couple of shots to get there, but do they really reset the pin that much?

Rob Dwyer 5:30

Well, they do, they do, and but you get to something else that adds that I think it really works with this analogy, is that there are different levels of difficulty, right? Par three, par four, par five. And we see that in support and in customer service. Like some of your interactions you get relatively easy, right? Those are your par threes, but then you'll get things that are a little tougher, a little more complex, and that's maybe your par five. But imagine now that all of my golfers, my agents, are all golfing on different golf courses, all with totally different holes, different pin sets. I cannot watch them all at the same time. I I simply cannot do this.

Charlotte Ward 6:21

And they have different strengths. Some people are great at the the tee off, some are great, you know, mid-rive. I'm gonna go with it, mid-drive. And some are are really excellent, like the you know, potting on a green on a slope uh when it's been raining.

Rob Dwyer 6:35

Absolutely. And so here's what happens that one time, whatever that cadence is, once a week, the expectation is that I meet, maybe Charlotte, maybe you're one of my star golfers, but I meet with you, and I'm supposed to coach you to help you be better at golfing. But how much time can I spend observing you actually do the job? And the answer is very little because I've got 15 people and it I may pick a hole, and I picked the seventh hole on the course that you did yesterday, and I observed that par four, and I can talk to you maybe about your approach when you were putting because you missed the putt. But it's quite possible that every other green that day, you actually sank the putt, and it was actually your drive that really needs work. But because I picked the seventh hole, that's what I saw, and that's what I coached on. And we do this, like that this has traditionally been the case, and I think one of the really amazing use cases that we have for AI is AI can actually watch every single shot, every single hole, and know exactly how you performed and understand that actually, as it turns out, Charlotte does not need help putting. She's an amazing putter. She just misjudged this one green. Yeah. Now, t-shots Charlotte could use some help because she tends to pull.

AI Sees Patterns Humans Miss

Rob Dwyer 8:25

And so the the ball gets shanked off to the left hand side of the group of the fairway, and that puts her in a bad position. And when we think about what AI is really good at, it's really good at pattern recognition. Like it's literally what LLMs do, they are pattern recognizers. And so being able to analyze all of this data, which we can now, gives me as a coach an opportunity to not just enter a coaching session with a plan, which I I like to think that you know, our supervisors enter those one-on-ones with a plan, but often they don't because they haven't had time to put together a plan.

Charlotte Ward 9:10

Sure. And and and you know, they maybe they only have a library of a few, you know, maybe they've never coached for that poor T off before, like they know how to coach for a putt. I I love this analogy because, and I didn't think I would as much at the start, because I know enough, I grew up with a golfing dad, so I know just enough, you know, not a golfer. I am, by the way, an excellent putter. Um, but in the support sense. But but what I love about this is is while you're drawing this kind of um uh visual um of the cause of the you know problem, let's say, or the the the driver for the opportunity, if we're talking about this in coaching language, is actually potentially upstream. It's further, you know, up the green, as it were. Um that uh that like I'm immediately thinking about all of those conversations I had, actual support conversations, not golf conversations, where you know, a support engineer or a support agent has said to me, This customer's being unreasonable. Like I told them this thing, and they're they're already angry, or they're already impatient, or they're demanding so much, and this, and it's like uh to be in the moment on that particular um hole on that particular golf course, if you get lucky, as an experienced leader and as an experienced coach, you can spot those opportunities uh in the moment where you can say, Well, yes, but if you go back four interactions to where you set this expectation, and now that's the thing that the customer is calibrating against, and so they've had four more interactions, and we haven't met an expectation that we set last week. Um that's a conversation that like you'd be pretty damn lucky to hit that example, wouldn't you? But they do it, they do exist, they exist often enough that we can just coach for them when we happen to be there, but we're not often there to your point in the right moment.

Rob Dwyer 11:28

Yeah, I also love too, and uh sometimes we over-index on opportunities. AI can help us identify what Charlotte's really good at. And we already talked about your putting, and so I can come into that conversation and recognize and celebrate and reinforce the thing that you're good at, the things that you have made progress on, because I want you to continue to excel in those things. And we all, as much as some of us may not want to admit it, can use a little pat on the back sometimes, right? Shouldn't always be shouldn't always be me patting myself on the back.

Charlotte Ward 12:16

I want my supervisor to pat me on the back sometimes, and so being you know what, Rob, sometimes I'm even just happy that AI pats me on the back.

Rob Dwyer 12:26

No, I know.

Charlotte Ward 12:28

It's just like take what you can get, right? Some of us are very praise-driven.

Rob Dwyer 12:33

Yes, absolutely. And I just feel like there is right now a big concern about AI from some people, and there's a rush to implement AI in other areas, and I I know not I think, I know that this is an area where AI can excel and take a human and make them better at their job. We focus a lot on the agents and the people who are dealing with our customers all the time, whatever their title is. That next level up has opportunities as well. And traditionally, in many organizations, we have done a very pisspoor job of preparing them or giving them the bandwidth to actually do this. I think about how much time supervisors

Bandwidth Limits And Flatter Teams

Rob Dwyer 13:40

spend on putting out fires, on dealing with your PTO requests, on all of the HR-related things that we ask them to do. Maybe they're doing interviews as well as we're bringing people on. Maybe they're taking escalations. Like there are so many things that we put on their plate running reports. Maybe they're also um doing intraday and trying to determine do we need more people or less people right now? Can someone take a break right now that's unscheduled? Like all of the things. And oh, by the way, by the way, know everything about your agent's performance so that when you have that coaching conversation, that it that it can be really good. And I just I don't buy that most organizations give supervisors or managers or whatever you call them the bandwidth for that.

Charlotte Ward 14:41

Yeah, yeah, absolutely. And all of the well, I mean, those people are are are being spread to your point increasingly thinly. Um organizations are getting flatter. We are meeting. I mean, I have d 11 direct reports. I meet most of them every three weeks. I haven't quite sunk to the monthly. I was like, I was like every fortnight's quite a lot. Um I'll meet the more senior folks a bit more regularly because I have more touch points with them for other reasons. But uh monthly is not enough. I know. Brainwave, three weekly, that's the answer. It's a weird cadence, but it's it works right now. But but we're being spread increasingly thinly, and um I think that there's there's like in increasingly flat organizations that's to be expected that you spend less uh time or uh have time on a lower cadence, let's say, with folks. I think that's natural, it's like it's a bandwidth issue, you know. Um but I think also the quality of that time needs to improve and the opportunities are twofold when we're talking about AI. We're talking about AI both in as you said, like spotting and uh raising and uh providing context for coaching conversations, but also in freeing up some of that bandwidth. And and so I think that it has to be twofold. This isn't there isn't a magic bullet here in terms of like if I apply AI to my coaching, suddenly I'm just gonna be such a better leader. Like you actually need to buy into I mean, let's just say tooling, because I think AI is only one we we over-index a little bit to AI because it's the zeitgeist. But frankly, if there's anything you are doing that a machine can do, the machine should be doing it, and you should be applying your skills, your unique human skills and experience to unique human needs and novel problems. And and like coaching is definitely in that uh that it's at that end of the spectrum, right?

Rob Dwyer 17:21

Yeah, yeah, I absolutely agree. And I think, you know, when I say like the sports analogy breaks down, it has always broken down for me because in sports there's practice, right? And your coaches observing practice, and there's usually only a couple of people at most doing anything important at any given moment, and so you can put your attention where it needs to go. That doesn't work in support, that doesn't work in customer service, it doesn't work in uh in a sales organization, like all the people are doing all the things all the time, and I don't have enough bandwidth to stay up with it.

Charlotte Ward 18:05

Because a coach in a sporting scenario is one-to-one or one to two, they're watching, they're watching you most of the time, they're watching recordings of you most of the time, they're talking you through recordings of you most of the time. Um, and you become as a like as a sports person, i.e. as an IC, so tuned to your coach's own uh advice that you begin to spot it in your own recordings and your own interactions more real time as well. And that's a real luxury to have that level of attention and time to dedicate to the people you are coaching and to oneself as an IC purely from an improvement point of view. It's bandwidth that no one realistically has. Um, it strikes me, and I don't know that this

Coaching Newbies Versus Veterans

Charlotte Ward 18:55

is that flattering to individual contributors, so I apologize in advance, but I'm gonna go with I'm gonna go with it anyway. It strikes me that the environment that we're talking about from a leadership point of view is much more like a you know grade school sports teacher, like who's teaching a class, teaching a class of 30 kids, right? Yeah, um, and let's say you have your kids who have been you know coached by dad at home or have or who've been playing whatever the sport is. Let's say it's probably golf breaks down at this point, but let's say what's a what's a common grade school sport? I don't know.

Rob Dwyer 19:32

Uh I mean basketball, basketball, soccer, football, sorry, football.

Charlotte Ward 19:41

Soccer.

Rob Dwyer 19:42

Yeah, what we call it over. I know.

Charlotte Ward 19:45

Let's say let's say some sort of ball that's in an average length to mid-sized team, where there are definitely people out there who have access to this is where my sports knowledge completely breaks down, whatever the soccer or basketball, or well, I mean little league, right? Like the the kind of you know, they have some of if you're a grade school or primary school, as I would call it, sports teacher, some of the kids on your class have exposure to other experience. Like they've they have a longer tenure, literally. They've been playing with their parents of you know, or in some like club or league scenario since they were three years old.

Rob Dwyer 20:31

And so they've got three older siblings that they play with at home.

Charlotte Ward 20:37

And they're down the park with the kids from the neighborhood every day after school, and then you've got other kids who are newer to the sport, newer to the team, newer to the um, you know, the whole ecosystem, frankly, of being.

Rob Dwyer 20:55

That was me. That was me.

Charlotte Ward 20:57

I mean, I I'm not sure I even arrived in the sport, never mind. Like, anyway, uh, but you know, like people who are arriving, like, okay, I have been sat looking at uh, you know, Minecraft for three years or whatever. I'm sorry, I'm sorry, everyone. But but like, do you know what I mean? Like, I've not been a sports kid in any respect. I have no exposure to this or any other sport. I'm arri I've arrived. I've either got to do it or I'm keen to learn, or maybe somewhere between the two. Um and uh and there's an opportunity as a sports teacher in that scenario to div to give some proportion of your resources to those kids, as much as your superstar kids, senior kids, more experienced kids, you know, who have arrived and know how to frankly knock one out of the park. I don't know if that's a sporting analogy, but it feels like one, you know.

Rob Dwyer 21:57

It is. And that coaching is different. Depending on who you're talking to, right? The the the kid that has been doing it and you can tell has attained a certain level of skill. Like you're refining things.

Charlotte Ward 22:11

Yeah.

Rob Dwyer 22:12

But the kid who just showed up and you're like, hey man, we need to get your shoes on the right feet before you even step on the pitch.

Charlotte Ward 22:19

Yeah.

Rob Dwyer 22:20

And that's that's what you call it over there. Um that one, yeah, right. We're we're going to focus on very basic foundational things, not refinement, because we're not ready for that. And I think that exists in teams to your point. And so the other challenge is what happens when I'm a new supervisor. I'm I'm brand new. I just got promoted. And I get 15 people handed to me that I have no idea where their performance levels are. This is where AI can be a huge help because it can lay out for you, Charlotte. We're going to be working on refining skills. And here are those skills. Here's what she's already a superstar at. But I've got Rob over here, and Rob has his shoes on the wrong feet. And so we're we're not ready for refinement. We need to work on some more foundational things. And that is a huge, a huge boon to those people that are often stepping into a role where they feel a little unsure of themselves, perhaps to begin with. And they're trying to establish these new relationships that they haven't before and they want to gain respect. It can be really amazing to be able to go into your first coaching session with a tenured agent and recognize that they are already really good at these things, and I'm not trying to coach them on something that they already are a superstar at.

Charlotte Ward 24:04

Yeah, yeah. No, I love that. And you know, the that is, you know, I mean, there's so many points um at which you have to do that kind of calibration, right? You know, you are you are a new leader in an organization, or you're stepping up into a leadership role, or you're bringing in staff with an unknown background. So, or or you know, you've hired them for a specific skill set, and the rest is kind of learning process for you as a leader for this new hire, you know. Uh, so I'm gonna I'm gonna seek to understand what their other skills are, where where those opportunities are. Um let's

What Data Becomes Coaching Signals

Charlotte Ward 24:50

get into the nuts and bolts just for a little bit here, then. We've talked a lot about golf, but I think it's been a re really useful and soccer and baseball and basketball. I think there's been some useful analogies there, because I think I think that um we general we generalize the term coaching too much. Um there are some things we've just begun to unpack which are potential gaps in our understanding, both um as we come into a leadership role, as we take on new folks, as we spread ourselves very thinly. Like if we're actually talking about what this ecosystem looks like as a leader, you know, not necessarily asking you to commit us to platforms, but give us some examples. Like what what are the signals? What are the sources? What are the opportunities we have for extracting and synthesizing and uh you know actioning these kind of gaps?

Rob Dwyer 26:04

Well, let me just provide a real-world example. This is one I've seen recently with a client that I'm working with, and uh, they are a SaaS provider, and so they are more of this support organization, but they offer multi-channel support, and so they do both voice and written async comms, right? Email ticket management, if you will. And and uh right, not a plug for what we do at level AI, but I think this really uncovers something that was for the person there that uncovered this, it was enlightening. We had an agent really, really great on the phone conversationally,

Real Example Voice Versus Writing

Rob Dwyer 26:55

and often we focus on that because maybe I can I can listen in. But it turns out that agents' written skills were not great, and so here they are excelling, and you make some assumptions when you hear an interaction and you go, Oh, yeah, they they really know their stuff. But the written skill set, which encompasses a lot of their interactions because they are working on tickets that come in, that was where the huge opportunity lied. It wasn't this person isn't conversational, it wasn't that this person didn't understand the product or how to support customers, but the way they represented themselves in these written communications was not up to the standard that they wanted. And so being able to come to this conversation and celebrate all of the good things that this agent is bringing to the table. And then also identifying here's some specific examples. Let's work on taking what are actual written communications that you have and identifying where we can make some improvements so that going forward, your emails to customers are better than they were before and they're easier to digest, and they have all the right formatting and all of these things. So I think when we talk about actually identifying opportunities, it goes beyond just the fact that, oh, you you know, you you can give a good empathy statement when someone's frustrated. Like, sure, maybe that is an opportunity, but there are there are deeper ways to uncover opportunities that may have gone unnoticed for far too long by analyzing all of these conversations. And that's where I think AI really takes things up a notch because I probably as a soup don't have time to comb through three different interactions in all of the channels that you support.

Charlotte Ward 29:17

Yeah, yeah. And it's but it but it's also it's it's not just the time to see the interactions, but to draw the lines. Yeah. And to draw the threads between them. And I think this comes back to you know something we said earlier, which is that the problem might not exist in an interaction, it might be in something three interactions ago. Um, and I mean, if you don't have enough time to get to all of your interactions individually as a supervisor, damn I don't have enough time to draw the lines across every interaction for a customer or a let alone, I mean, a ticket, let alone a customer, you know. And actually, I I mean it's interesting because I'm trying to think of whether I know a platform or not that does that well right now, just inside the context of a ticket, never mind across all interactions, and I'm not sure there is one. So if anyone listening knows of a platform, that that is there a platform out there, Rob, that can draw across multiple interactions and draw those lines for me as a supervisor.

Rob Dwyer 30:28

There might be one. You can you can talk to me about that. I'm not going to uh pitch, but I will say aside from this, the one thing that I will say is we still need as an industry to be mindful that identifying what to coach, what behaviors to coach, is not the same as identifying how I approach that and how I build relationships with people that support to me and and all of that. And I I think we can always do a better job of preparing people for leadership roles,

Coaching Skills Versus Leading People

Rob Dwyer 31:10

and it's the easiest thing to skip, quite honestly.

Charlotte Ward 31:16

Yeah, yeah, it really is, it really is. Um and I think I do know I think that's true every level of leadership. It's the easiest thing to skip when you're bringing bringing on new leaders, like when you're growing leaders in your team. It's like just just get to grips with the operational and just do that. Because that because particularly when we're being spread increasingly thinly at a slightly more senior level, the thing we need taken off our plates quickest is just the day-to-day. And so we tend to focus less when we're growing leaders, particularly in the current climate, I think. We tend to focus this less on coaching coaches.

Rob Dwyer 32:11

We do, and you and I could spend another probably six hours talking about that, but uh clearly we we are or golf, or golf or golf. I will just say that I think this is one of the ways that we can as an industry accelerate having the time to do that. When when our leaders are prepared to go into a session and they understand what they should be coaching, and we're not spending our time trying to help people identify that piece, then we can help them become better coaches because at least we're going in with the right intentions, the right topics, the right behaviors that we should be spending our time on. And that then becomes one less thing that I have to worry about from a strategy standpoint. It's really critical that we approach those conversations with the right behaviors in mind. And that now I can focus more on the mechanics of actually executing that coaching session and what kind of tactical approaches I should be taking based off of this person's skill, maybe, maybe their will, maybe, you know, sometimes it's not a skill problem. Let's be honest. Sometimes somebody just doesn't want to do something. And so, you know, are we holding them accountable? Are we setting expectations for them? Um, just like you mentioned before, right? It could be about um from a customer standpoint, we set expectations, and then yeah, that's where they start to gauge things from. We need to do that with our people that report to us too. We need to set clear expectations so that they know what's expected of them.

Charlotte Ward 34:09

I yeah, I couldn't agree more, and um, I'm going to draw on something that uh uh Reagan Helms said in a very recent uh recording, which I encourage you to go and listen to um after this, is uh if you haven't already, which is that also applying AI to holding ourselves accountable as leaders is a huge superpower, I think. I'm not quoting him verbatim there, but that was the essence of one thread of the conversation we had, which was like he uses AI to ensure he meets the expectations he has set in one-to-one's with his team. So that promise I made that I would connect that

Accountability And Closing Thoughts

Charlotte Ward 34:55

person, that I would deliver that thing, that I would ensure that person was aware, like he holds himself accountable, which actually is is part of uh part of what we're talking about, right? Unless we are um holding ourselves to a certain standard, it becomes harder to hold other people to that standard too.

Rob Dwyer 35:15

It does. Yeah, and shout out to Reagan, what a great guy.

Charlotte Ward 35:19

Oh man, yeah, yeah, yeah. Absolutely smart. Yeah, oh yeah. Um shout out to you as well, Rob. Thank you so much for joining me today. See what I did there? See, I'm I'm nothing if not a consummate putter when it comes to closing the podcast. Red in the cup, red in the cup, everything the pin could be anywhere, and I will I will close out an episode, I'm telling you. Um, listen, thank you so much for joining me today. It's been an absolute pleasure. Um, despite the promise of this being very heavily sports, sports analogy loaded, uh, which I uh was slightly worried about at the start. I have massively enjoyed it, and who knew I knew so much about golf? Um, so uh yeah, thank you for joining me. It's been a pleasure to have you back after quite such a long time.

Rob Dwyer 36:08

Pleasure was all mine, Shaw. All mine.

Charlotte Ward 36:10

It really wasn't. Thank you so much. You're gonna come back and have another conversation very soon, please.

Rob Dwyer 36:15

Sounds great.

Charlotte Ward 36:16

Okay, all right. Thank you so much, Rob.

Rob Dwyer 36:19

Thank you.

Charlotte Ward 36:22

That's it for today. Go to customersupportleaders.com forward slash three zero two for the show notes, and I'll see you next time.