Ever wondered how your contact center could save significant costs while keeping up with the QA process? Brace yourself for an enlightening discussion and AMA where we unravel the potential of AI in transforming Quality Assurance in your contact center.
We'll take you through our early journey of automating QA processes, sharing our experiments with chat GPT and how its possible for any contact center to start to think through the process of using AI for QA.
If you are looking for USA outsourced customer service or sales support, we here at Expivia would really like to help you support your customers.
Please check us out at expiviausa.com, or email us at info@expivia.net!
Follow Tom: @tlaird_expivia
Join our Facebook Call Center Community: www.facebook.com/callcentergeek
Connect on LinkedIn: https://www.linkedin.com/in/tlairdexpivia/
Follow on TikTok: https://www.tiktok.com/@callcenter_geek
Linkedin Group: https://www.linkedin.com/groups/9041993/
Watch us: Advice from a Call Center Geek Youtube Channel
Ever wondered how your contact center could save significant costs while keeping up with the QA process? Brace yourself for an enlightening discussion and AMA where we unravel the potential of AI in transforming Quality Assurance in your contact center.
We'll take you through our early journey of automating QA processes, sharing our experiments with chat GPT and how its possible for any contact center to start to think through the process of using AI for QA.
If you are looking for USA outsourced customer service or sales support, we here at Expivia would really like to help you support your customers.
Please check us out at expiviausa.com, or email us at info@expivia.net!
Follow Tom: @tlaird_expivia
Join our Facebook Call Center Community: www.facebook.com/callcentergeek
Connect on LinkedIn: https://www.linkedin.com/in/tlairdexpivia/
Follow on TikTok: https://www.tiktok.com/@callcenter_geek
Linkedin Group: https://www.linkedin.com/groups/9041993/
Watch us: Advice from a Call Center Geek Youtube Channel
This is advice from a call center geek a weekly podcast with a focus on all things call center. We'll cover it all, from call center operations, hiring, culture, technology and education. We're here to give you actionable items to improve the quality of yours and your customer's experience. This is an evolving industry with creative minds and ambitious people like this guy. Not only is his passion call center operations, but he's our host. He's the CEO of Expedia Interaction Marketing Group and the call center geek himself, tom Laird.
Speaker 2:Welcome back everybody to another episode of advice from a call center geek the call center, contact center podcast. We try to give you some actionable items to take back in your contact center improve the overall quality, improve the agent experience, hopefully improve your customer experience as well. My name is Tom Laird. I'm the CEO here at Expedia Interaction Marketing. Expedia is a 600 seat call center outsourcer located here in beautiful, sunny and kind of steamy Northwestern Pennsylvania right now. How was everybody doing? I want to kind of talk. I think that this is a little bit of a special episode A lot of questions that have come out of some of the posts that I did last week on LinkedIn on how we've been able to somewhat get to the point of being able to totally automate QA, or at least have the proof of concept to automate our quality assurance processes in our contact center with all of our different clients using the actual scoring forms that our client utilizes our clients.
Speaker 2:This was kind of an eye opening experience for me. I don't have all the answers on this. I'm not an expert on this. We are not a software company. This is something that we've tried to figure out to be an added benefit for our clients. I think the lessons that we're learning can be taken by the entire contact center industry as a whole to say, hey, this can be done, this can really be done. It's not as difficult, I think, as it may kind of come out to be. To be honest, I think internal contact centers and BPO's and non third parties I think it's almost easier for us to do. This is one of the few tools that I think might be easier to do in-house or do on your own than actually trying to go out and purchase a third party software. Not to say that there's not stuff out there now, because, of course, after I post that, I'm inundated with so many companies that are saying that they're doing this. They are to maybe a certain extent, but I still really haven't seen the in-depth how difficult some of our stuff could be that you really would have to figure out. I think it would be very difficult. I'm going to talk more about that as we go.
Speaker 2:Let me set the table here. I'm live on TikTok. I am live on LinkedIn. A couple of you guys Sean, how are you doing? Kimberly Already on LinkedIn, some of you guys here on TikTok as well.
Speaker 2:I'm going to do this as an AMA again, any questions that come up. If you guys have anything that you want to talk about with this stuff, I'm going to get into it, how we've done it, how we're doing it and let me know if there's anything that I can help. Again, I'm an open book with this stuff. It's not like this is going to be a product that I think that we can really go sell because, to be honest, I think it's too easy. The thought behind this it's very difficult and very easy at the same time, which I kind of will get into. Again, I'm kind of just rolling off the top here. I really know agenda, because I think that there's going to be a lot of questions, but the only agenda I want to talk to you guys about how we did this.
Speaker 2:Why did we think about automating our QA process? First, I'm a huge believer in sentiment scoring. I think sentiment is, as you guys know if you've been following the podcast, can kind of trump CSEC, can kind of trump NPS, because, again, it's 100% of all of the calls and it's scoring them in kind of a way that I can feel comfortable that we're using proper tone, we're using proper word choices. It has definitely correlated to CSAT scores. It also has correlated to who's successful on the program from an agent standpoint. So I wanted to take that to another step and all of us have been playing around with chat gpt.
Speaker 2:We've been doing different prompts for different things. You know, as we were sitting here and talking through, we said why can't we develop prompts that will go through our QA process? Do it exactly like an agent? Now, these are in-depth prompts, right? So this is, this is where I think the the difficulty, right, it's, it's really easy to do if you can figure out the prompt. It's very difficult to do if you can't and you don't understand the prompt or you know you're lacking in some of that manner, which we've done a lot of experimenting with this to kind of figure that out. So so the first thing that we did is we said, alright, we need transcripts, and I think transcripts about just as a side thing, transcripts, you know, in talking to a lot of my industry friends, especially the CCAS platforms, they're going away very quickly.
Speaker 2:Voice is going to be coming really soon to to AI, to where you're not going to need the transcript. We're going to be able to do it directly at voice at, you know, february, I'm sorry, july 31st 2023. We don't have that yet. In a way that's, I think, affordable. So what we have done is we've taken all of the chat transcripts, or, as we have basically all of them in our interact interaction analytics, right. So our analytics platform, we are utilizing and pooling Our, our transcripts. Now, if you don't have transcripts, this thing becomes a little bit more difficult, right, and it becomes extremely expensive as well if you have to take your voice calls and go get them translated. So, again, I think one of the the added benefits of Having analytics, right, is you do get the transcripts, and we are still working through this process.
Speaker 2:So let me again Let me say this as well this is a work in progress, this is a proof of concept. This is not a finished product. We're not even rolling this out to our clients yet. This is just us figuring it out, and last week was like the, the big eye opener, like holy Jeez, we, I think we can really really do this. Again, we take the transcripts and then we we started testing them on the, on the chat, gpt for UI, ux, the desktop.
Speaker 2:You know the basic, you know the $20 a month thing, right, just to kind of test things out, test our prompts out, and some of these can be difficult. You know, if you have a QA platform, that is just. And we do have a couple of these with a couple clients they were the first one who tested that are just kind of yes, no right. Did they do it? Yes, did they not do it? No right, all points, zero points, right. So a yes maybe five points, a no is zero points.
Speaker 2:And again, we're talking about scoring off of actual scorecards that clients or you have had for the last 10 years. That you like, that you benchmarked, that you know work well, that you know if the 80% or 90% or whatever that number is, you know 90 out of 100. You know that that's going to correlate to a pretty good customer experience, that that agent knows what they're doing. So you feel really comfortable with that. I think that was another problem, right? So many people said, hey, we we're automating QA, but they're automating it with, like, their own proprietary numbers. They don't correlate well to you know what we have used from a scoring system, so we wanted to do that.
Speaker 2:So, again, if you have a yes, no, you know maybe you have 25 questions. You know each of those are worth. You know four points right for a hundred point score. And then you can very easily just write a prompt To say, hey, look at the transcript, answer these questions scored at the end, right? Let me throw this to, and again, I just have so much on my mind. It's pretty exciting. The prompt can also give you and I mean this is obvious, but the prompt tells you what you want your output to be as well Right, so you could just say, hey, go score the call, but you can take that different. Can you please score the call? Can you please show the sentiment of the customer and the agent based on the transcript? Can you give us, you know, five ways that the agent can improve, even if they had a hundred percent score or if you know they had a lower score. What are some of the ways that they can improve upon that? There's a ton of different things that you can ask to get outputs that would be extremely difficult to do In a time-consuming way for it, for an agent taking this. Writing prompts. Easy questions, that's an easy one.
Speaker 2:What happens if this and this is, I know, a lot of you guys right, and let me just say this to the two things that you need. I mean and I think that again, I think all this stuff is obvious but it's not being said the two things that you need are number one is we need to be able to have access to the data, right. So chat, gpt, whatever Model that you're using, has to have access and be able to see it. So, right now, we're using a transcript. If you have things that are on your that they have to check on their screen, right, that it that ages to it, that's gonna be hard, it's gonna be very difficult. So, again, I'm not saying this is the NLBL. There are a lot of things that it's still not gonna be able to do.
Speaker 2:But from an actual, just transcript or voiced type Interactions, I think it can do things pretty well. So here's a more difficult use case that we figured out right, we have calls that come in, but there could be, you know, four or five different types of calls. Right, there could be a sales call, there could be a service call, there could be somebody asking about a provider, there could be somebody canceling, right, in. All of those types of calls Correlate to different questions that are being asked. Right, everything from verification to you know, asking. You know what actually happened on the call to disclosures at the end, To making sure that everything from a, from a compliance it's you know issue, if we do have a sale, from masking to doing all the stuff right, that becomes extremely More robust I don't want to say difficult, but I think more robust when you're thinking through the prompt Right.
Speaker 2:So you have to, you have to have a really analytical, logical mind because again, this is still a, a kind of computer process but you have to have the Program knowledge and kind of the art right with that too. So again, we've been trying to bring in our whole team of IT, of management, of the on-floor guys that deal with this every single day Are obviously our QA to come up with with really robust prompts that are kind of there's a lot of if vans that are a part of it, you know, looking at specific types of language. If a customer says this, you know, really getting in depth Right into what could be happening on a call. That would then correlate to a specific type of call that would then move down the channel to a, a specific type of questions from a QA standpoint. That's the reason I think it's going to be very difficult, and I'm not saying it's impossible, for you know a lot of these and I did a video on this a lot of the you know kind of sass Players that are trying to do this as a business. It's extremely difficult unless you want to sit down with a customer and really sit through and talk through a prompt. I think you know there's a lot of things that you really need to look through and there's a lot of trial and error that that will go into what actually works and what doesn't. I mean, and to be honest, you know we have seen slight changes in verbiage right, do have some can have some significant changes in what happens with the, the output of the score. But I think when we got it to a point where we felt comfortable and then we would correlate it with our QA we call it EQA, xp, be a quality shirt we when we would do that, they would correlate extremely well. I think from that aspect, you know we were, we were doing, you know, a Pretty good job when it came to how do we now scale this, because we feel like we have a prompt that we can scale.
Speaker 2:Let me stop there and and there's a couple questions here that I want to go on LinkedIn. What is the? What is? What do you expect the total upfront investment to be to get this working? Is it already integrated with chat GPT API? Yes and no, I mean I'm not there yet. I mean to be totally frank, we're not there yet. We have connectivity. Our APIs are connected to chat GPT. We have been over the weekend. We've tried to scale, and when I say scale, we try to do, like you know, 10 to 15 different of the same thing. I'm looking at that today, scoring 10 to 15 different Calls or transcripts. Everything looks pretty good so far.
Speaker 2:There's a couple things that we have to change from prompts, from a, you know, when somebody calls into cancel, right. We found some things that we need to tweak there. But from a cost standpoint, you know what we have found is chat GPT 3 or 3.5. We can get pretty close to the same answers that we're getting in chat GPT4. The cost is crazy. It's crazy cheap when you're using chat GPT 3. So I I don't have it in my notes, I don't have my notes in front, but I believe that if we were going to use chatGPT4, I forget what it was. Let me find out. I think it was 100 to 150 calls that we could do for $20, which isn't a lot when you're talking about thousands of calls. How that correlated with the input and the output costs using the API With chatGPT3 or 3.5, if we can get that to work with that.
Speaker 2:It was create. It was thousands of calls. That's something that we have to look at. But again, for me being a BPO, I think that this could be an added benefit for our clients. So if I go to my client I say, hey, listen, we have a way now to score, not just give sentiment, not just give an arbitrary number, but use your scoring platform, use your scoring form that you guys wanted us to do. We can score 100% of those calls. And again, there's other questions that go with that, like, what do we do with all these calls? Then we have thousands of score calls. We can have that conversation here as well, but I think from us there's a value add there that makes total sense.
Speaker 2:And also, let's talk about the inevitability here. If this does work and I'm saying if I'm not saying that this is 100% I initially started this last week saying by the end of the year, I think that QA could be totally automated by these type of processes and I really do believe that Six months of work on this, I mean this thing's going to be For us. It's going to be awesome. I can't imagine if you have 200 programmers just working and focusing on this. They're going to be really good at it as well.
Speaker 2:So, yeah, I mean I think that there will be a human toll when it comes to scoring. I think that this is the low hanging fruit where we could see AI really take some of these jobs from a scoring aspect. But I think that moves right, because now we have so much more data scored, so now we have to look at what do we do with all that? And I think a lot of these QA guys which they've been for a lot of organizations, they've been doing already is coaching and getting better with the agents, because you're not going to have to score. You're going to just have to be able to look at certain things, get outputs that you think can be helpful, find certain things that can improve the agent and then go and start really coaching and get really in depth with that.
Speaker 2:So again from a cost standpoint, I think it is negligible. When the third party guys come out, when the CCAST players come out, I think it's going to be expensive. So, I mean, I think it almost pays to think this through yourself Now. Granted, it's not there yet. There's security issues, there's HIPAA issues. I'm not again. I'm not saying this is the end, all be all, but how far this is right now is pretty insane. What you can do right now is crazy, and the scoring of calls at scale, if you have a good prompt, is really amazing. All right, another question there's a lot that can be done in the space. Using Chappity is not the right thing to do, as this is a generic data set. I've just checked with some of our customers. They have 30 plus different types of audits. First step bring all your data into a platform that can use your train. Yeah, and I think that that's what is going to happen eventually. Right, that I think people are going to almost have their own instance of chat GPT. Right, like very large companies are going to do that.
Speaker 2:I don't care about the learning piece of this at this point, and I think that's where we go too far. The AI learning of what you want to do is something that's going to happen and it's really awesome. But what I'm saying is you don't need that right now to score calls. You need an AI engine to actually just score calls. It doesn't have to be this learning about each specific agent and what the nuance is and kind of getting deep into that. I don't think you need that yet. I mean, it'd be awesome, but we're just kind of taking a tool that we're doing that probably doesn't have to be done by humans, right, and we're scoring calls and then giving kind of just some insights into what happened on that call and how to get things better. So, from a learning and a machine learning, I don't really care. To be honest, I mean, I don't think to do that. That's like a roadmap, cool. That's not a need. I think the need is can you develop a prompt that can actually score calls like a human being or as close to a human being? Ok, let me see I'm on TikTok here. If we got any questions, a lot of you know the same what you would think you would get on TikTok. Okay, I think what we've done is cool. I think maybe just because we're talking about it, I am positive there's a ton of you guys out there, or some of you guys out there, that I've already kind of figured this out Right, if you can program and you have the logic to program, then you definitely have the brain and the logic to develop specific prompts right on different types of audits, right On different types of QA. You can score calls exactly how a human being is doing, asking the same type of questions that humans are asking, but you're actually just typing it out and having the chat GPT process do it.
Speaker 2:Now there's security issues right now, right Now. The good thing is it made me feel at least a little bit better that chat GPT does not model off of API connections, right, so that data that goes back and forth they're not modeling off of. That does not mean that things are totally secure. That does not mean we don't have HIPAA issues or PCI issues. I think that there needs to be a lot of thinking that through, especially with what we're doing to look at how do you manipulate, at least right now, the transcript? A lot of you know, from a PCI standpoint, we mask all the calls. Still, you know there's customer information, there's customer names, those types of things.
Speaker 2:From a HIPAA standpoint, obviously there's some things that need to be pre-done on the transcript, and I think that's an AI solution. There's something there as well. But, yeah, guys, I mean literally, I mean I cannot explain that in 20 minutes. That's really all we're doing is we're taking our chat transcripts from analytics, we're pulling them in, we're indexing them so that we know who is what and what program they're on and what skills. Right, we have an API that can then come and look and take the prompt that we have created, look at the transcript and then score the call and give the outputs that we want, based on what the client says.
Speaker 2:Now, here's the next pieces, and that's all we have right now. Right, we were developing kind of a, I guess, a way to view the outputs in a cleaner, better way. There's no APIs right now that we have with nice CX1, to put them back into our QMAPalform which would be awesome, right, and I think that that will probably happen right, if we can figure out the scoring then to be able to put them back in so that we can, you know, maintain all that agent integrity when it came to the QA. I think that this will be an added, a huge benefit for our customers. I think that if you have one programmer on your staff, you can figure, start to figure this out to do on your own. Don't get too complicated with it. Don't start thinking like we have to do all of this like crazy machine learning. No, you're just basically having a. It's the same thing of transcribing a call at the end and putting it into a CRM, right, there's no learning that has to happen with that. It's a job that it needs to handle and I think if you can think of this QA piece of scoring on your calls as a job that AI can handle, you don't get caught in the weeds then with a lot of the things that I think people are thinking through.
Speaker 2:Take that first step Develop your prompts, practice it on the UI, ux on the ChatGPT4 or the 3.5 on the desktop. Find a prompt that works right. Go to your QA, have them score the calls right. Make sure that you're within I don't know. The one example that I gave the other day was we scored it as an 82 point. The ChatGPT or the AI scored it at an 82.5. We scored things at an 84.5. I'm okay with that, right. We're in that world. And I'll tell you what if I have five people in our QA department all score a call. I guarantee you we're going to have a lot of similarities, but it's not going to be always perfect on what they think, based on how maybe robust that scoring is. So think that through as well.
Speaker 2:Yeah, so practice there. Find a prompt that you like, right. Use the API then to kind of put that in there, right. Have a mechanism to grab your transcripts, which I know can be that. To be honest, that's the hardest piece. The hardest piece is finding and getting the transcript not the prompts right, because it's not something that we normally do and if you have a transcript, it's a little bit different to index it and kind of going through all that process. And then, if you find a prompt, you have the transcripts. You can then do some things at scale and really score and then use all of that data to then try to make your contact center better and realize it's not that difficult.
Speaker 2:And while there still needs work to be done on this, think about where this thing's going to be in six months Just six months. I know some of the huge players that are working on this stuff right now and I think it'd be very difficult. I think you have to almost have an onboarding team to sit down and really go through. I can peel Sharma here on on LinkedIn. He has a really good point. They have customers with 30 plus different types of audits and there's so many paths that QA can take based on human interaction, what humans can say. But you have to be able to account for as much of that as you possibly can with the prompt. So, but if that can be, if there's a handholding to a prompt that everybody loves and likes, and then maybe they can just pass it off, this thing could be done. This thing could be done at scale, this thing could be done as a business.
Speaker 2:I know a lot of the C-Cast players are kind of thinking that through, but from what I'm hearing, there's more arbitrary, just kind of numbers instead of really looking at like a BPO method of we have 25 different clients and everybody has their own plot, has their own form how do we score that? And kind of look through that. So yeah, guys, that's kind of what I got. I mean I don't know if there's any more questions on there. Again, it's really cool, but it's really not that difficult right.
Speaker 2:There's some things in it that are maybe unique, to kind of get through from figuring out the prompt, from figuring out how to get the transcript from. What do you do with the output. But that's kind of the fun part, right, especially that the output and kind of figuring out the prompt, get your team together, have some fun with it. I think that you can really do some really cool things from that standpoint and I think you can get this up quickly, like literally, you can do this in a day, in a day, like you could at least be practicing and testing it on your prompt, and you can get a prompt done in a day for something that's very basic. So, again, it's a whole new world out there. This is maybe the first real piece of AI technology that I think can hammer some type of low hanging fruit, which is the scoring of calls. Right, it's just like resetting passwords. It's like this just give a good data in and it's gonna give you good data out.
Speaker 2:Doesn't need to learn, doesn't need to get crazy into certain things. So again, that's what I got. If you guys have any other questions for me, please let me know. Hit me up on LinkedIn. I'm sure we'll be posting more on this. I love Chris Crosby. He's like, hey, you should do a prompt, have everything, but have the outputs. Be like with Michael Scott from the office, right, I think there's so many cool things that you could probably do even with agents. That could be funny, that could kind of change things.
Speaker 2:And again, if you think of it and I know that's kind of funny, but that's how easy it is to change things drastically just with a couple words, just from a prompt. So learn how to write, learn how to write prompts, learn how to get really good at that. Teach your kids how to do it, like I'm trying to get my kids to use chat GPT all the time, not just for, like cheating on school, right, but understanding how to write things and how to get the proper outputs. We have to be creative and we have to change on how we think as well. So hopefully that helps you guys. That's the secret sauce.
Speaker 2:Again, there's no real secret to it. I mean, I think it's pretty obvious with what we were doing. But I think the cool thing is it can be done. It can be done with maybe if you have a little bit of API knowledge. You don't need to be have a full, robust team of programmers to get this done. I mean, obviously you can take it to different levels, but this base level of scoring your customer's calls if you're a BPO, or scoring your internal calls, it can be done extremely easily. And let me know again if you guys have any questions and I'm more than happy to help. But other than that, if there is nothing else, I'll talk to all you guys.
Speaker 2:Oh hey, by the way, quickly, thursday, or I'm sorry, wednesday, at one o'clock, eastern James Dawkins he's coming on. He is the CX Rockstar. If you've ever seen him at CCW or any of the big contact center shows, he's kind of the Rockstar CX guy, huge influencer, huge into the AI as it relates to the contact center. So we're gonna have a one hour AMA just talking about everything contact center, ai, all the new stuff that's kind of coming on. Let's go, guys. Let's go. Tap the screen, guys, tap the screen. Sure, just need a little love here. Boys and girls, where are you from, man? So I'm from Buffalo, new York. Oh, nice, right down the road in Erie, pennsylvania. Oh, you're in Erie, dude. I love going to Erie, to the Mill Creek Mall. I was so nice to go.