Experience Action

CX Pulse Check - May 2025

• Jeannie Walters, CCXP • Episode 116

Share episode

Artificial intelligence is revolutionizing customer experience—but are businesses applying it with care? In this compelling episode, CX expert Jeannie Walters sits down with Ovetta Sampson, an AI design leader with a decade of experience at IDEO and Google, to explore how companies can harness AI without compromising trust. 

Ovetta challenges common assumptions with a powerful statement: “AI is neither artificial nor intelligent.” From chatbots that fabricate policies to biased algorithms in loan approvals, she shares eye-opening examples of how AI can go wrong—and what to do about it. 

Learn why AI needs continuous testing, feedback loops, and what Ovetta calls “Mike Tyson proofing” to avoid delivering broken experiences. She also offers a practical framework for implementation: focus AI efforts on your most frustrating customer touchpoints, not just where it’s easy. 

Looking to use AI more effectively in your organization? Ovetta is offering listeners an exclusive discount on to our listeners. Start building smarter, safer customer experiences. Use the coupon code "experience" for a 50% discount on one of the following: 

  • 1 hour coaching session ($300 regular price)
  • 1 hour AI Risk Assessment session ($300 regular price)
  • 90-minute CEO lunch ($1500 regular price)

Coupon Code is: experience

Redeem at www.rightainow.com/appointments

About Ovetta Sampson:
Named one of the Top 15 People in Enterprise Artificial Intelligence by Business Insider in 2023, Ovetta Sampson is a tech industry leader who has worked with multiple technologies to help solve some of the biggest problems facing multiple industries. Leading engineers, designers and researchers, at top companies such as Microsoft, IDEO and Capital One, Ovetta has been designing and developing machine learning, artificial intelligence and enterprise software solutions for more than a decade. She left her last job as Director AI and Compute Enablement at Google to found Right AI, a consultancy that helps organizations and businesses minimize the human engagement risks when implementing AI.

Follow Ovetta on...
LinkedIn: https://www.linkedin.com/in/ovettasampson/

Articles Mentioned:

Resources Mentioned:
Experience Investigators -- experienceinvestigators.com

Want to ask a question? Visit askjeannie.vip to leave Jeannie a voicemail! (And don't forget to follow Jeannie on LinkedIn! www.linkedin.com/in/jeanniewalters/)

Jeannie Walters:

It's the Experience Action podcast. I'm Jeannie Walters and I'm so glad that you are here. I am glad that you are here for this episode because we are diving in to all things about AI and CX in the news, and we are so lucky to do this on this CX Pulse Check episode with my very special co-host, Ovetta Sampson. Ovetta, it's so good to see you. I've known Ovetta for a million years and you are one of the first people who really tapped into the power of AI and also the warnings about it, what we can do, some of the things that we need to be aware of. So I'm really excited about our conversation. Will you please tell our audience all about you and your brilliance?

Ovetta Sampson:

Oh, wow, I don't know about that, but I have longevity. I think it would be.

Jeannie Walters:

Oh, that's a good word, that's a good word.

Ovetta Sampson:

So I don't know if we could call it brilliant, but definitely in the game for a while. I started doing dabbling in artificial intelligence back when I was in grad school at DePaul University (go Blue Demons) in 2013. And then went on to start a career in design, artificial intelligence products design, at IDEO Chicago in 2016. And just left Google, where I led a design team that built their machine learning, AI platforms and tooling for generative AI in February, and now I'm the owner and founder of RightAI. It's a consultancy that helps CEOs and enterprise businesses implement AI in a safe and efficient way, and I have been studying and researching and creating AI products for the last oh my Lord 10 years. So, yeah, longevity, I think.

Jeannie Walters:

There you go, I love it. I love it.

Ovetta Sampson:

In AI, which, by the way, is saying something. Of course, now everybody claims to be an AI expert.

Jeannie Walters:

It's true, it's true, and so I am really excited about this conversation because it is everywhere, right, you can barely have a professional conversation without somebody bringing it up, and it's really an interesting time conversation. I feel like we're at the tip of the iceberg for a lot of things, because I feel like people are falling into the camp of, you know, this is so new and exciting and shiny, and look at all the things that it can do, or they're like the robots are coming and we're all doomed, and so I really kind of subscribed to the idea that we are somewhere in the middle of that. And one of the things that you talk about a lot is how important it is to be thoughtful and intentional about how we actually implement these things, and that aligns with what we talk about, with customer experience that we have to be proactive and intentional and thoughtful about it, because it's going to happen whether you talk about it or not, right? So why not own it and why not do it the right way?

Jeannie Walters:

So I pulled a couple of fun articles for us to discuss here. So the first one this is from CX Today and the headline is a contact center chatbot invents company policies, now customers want out. And this article was really compelling because I think this isn't the first kind of hiccup we've seen with chatbots, but this one they were, you know, inventing things like refund and rebate and things that were not available, and so customers were hearing about expectations about promises that essentially weren't going to be fulfilled, and now they're coming back and saying, well, this is a mess and everything else. So I'm just curious, like chatbots are probably one of the first places that we really saw AI implemented, especially in the customer experience world, and so what are some of the things that leaders need to consider as they're introducing AI chatbots, as they're going through this, and what will this be a warning for them about?

Ovetta Sampson:

Yeah, I think. I saw, I read that, and I know that that a lot of your listeners and people who are tuning in have heard about generative AI, I'm assuming. And that's when AI models generate or even create new text content, anything that people can engage with nowadays, and chatbots once the purview of Alexa, Alexa, Alexa have now turned into.

Jeannie Walters:

You just turned on a bunch of speakers. I don't know if you.

Ovetta Sampson:

Right, Right. Like now. I know I turned on a bunch of speakers where you had to repeat yourself. You had to, like frustratingly talk to them because of generative AI. They can have conversations and seem as if they understand us and can engage with us on an almost imperceptible human level. Right, Turing Tests, and for those of you who are not tech savvy, the Blade Runner test, right. So these chatbots seem as if they're human, but the problem is they're not, and the problem is they're not even close to understanding us.

Ovetta Sampson:

What they are are probabilistic systems who base their responses on the probability of what they say is correct.

Ovetta Sampson:

So when you ask a chatbot a question, what that chatbot is doing is going through all the training data that has been inputted into the model system by a human, labeled by a human and described by a human to the chatbot what it is, and then doing a statistical probability model to say what you're asking is once a response in this outcome, and because it's a probabilistic system, it will be wrong.

Ovetta Sampson:

And so chatbots will give you what they're calling the industry hallucinations and what would call lies. It's not because it's intentionally trying to deceive you, but it is because of how it's built and how it works. So companies who decide to use these chatbots if they decide to set it and forget it, they will get problems like this. And what's going to happen a la Google and its first release of Gemini, giving you Indian and black popes right. What's going to happen is you're going to have to go back through your system and figure out why your chatbot made that mistake. The problem is, your customers may not give you that grace period, so you need to install rigorous testing, and even to the point of what I it's called adversarial testing in the industry, but I'm calling it Mike Tyson proof right. It needs to be aggressive. You need to beat up your AI chatbot to ensure that it doesn't give you these wayward outcomes.

Jeannie Walters:

Yep, and I think, with this story in particular, one of the things that struck me was like it's one thing if you hear from one customer right, it's one thing if you hear from like that's a red flag, that's something you can say oh, what's going on.

Jeannie Walters:

It's another one you hear from a bunch, and so that tells me,

Ovetta Sampson:

I also think your example is an example of AI exacerbating an underlying customer experience issue. So if you read some of the comments of the people on that Reddit thread who said that they were leaving and stuff, it wasn't just what the chatbot was saying. It was this overall view that the company was trying to corner the market on them, and then the chatbot's new policy was just another list and examples of that company not treating them well.

Jeannie Walters:

You are absolutely right.

Ovetta Sampson:

So I think if you add AI to bad customer service, you will get outside customer service issues, right. Well, it all goes with an outcome. But adding AI will give you an outside scaled customer service problem.

Jeannie Walters:

And you know what we talk about a lot is that every step of the customer journey is either building or eroding trust.

Ovetta Sampson:

Correct.

Jeannie Walters:

And so if they've already eroded all that trust through other experiences and then this happens, that is absolutely something. But I think one of the things that I wish more customer experience leaders would kind of embrace is this idea that, to your point, you can't set it and forget it. You need to keep oversight on it, you need to keep testing it. Because it evolves. That's the whole idea is that it learns and generates and evolves, and so if it's evolving in the wrong direction, you need to be aware of that pretty quickly, because then you can put a pause on it and figure it out. If it goes completely rogue, like that's hard, you can't put that genie back in the bottle, right?

Ovetta Sampson:

So yeah, and I think I when I speak about AI and especially generative AI, because it seems magical. It does seem as if you have just hired a, a new employee who you do not have to train and you do not have to educate and you do not have to do all the investment that you had to do in another employee who's already ready to take your company, your company policies, your business and deliver that to customers scot-free. The problem is that new employee you hired has the human engagement quotient or the emotional intelligence of a toddler. Right and no offense to toddlers.

Jeannie Walters:

They're little tyrants. Let's be honest.

Ovetta Sampson:

They are right. They have no conscience, right. Their moral code has not been set. We know from research that that comes around age seven, right? So if you're talking about somebody two to five, two to six, they're all about themselves, right, and all about what you give them, right? And generative AI model systems are a lot like that. It's not that they don't perceive their environment, that they don't understand, that they can't learn, all those good things, but they also aren't wielding a lot of emotional intelligence at all, and I won't say a lot, at all. And they have no moral code. So right and wrong means nothing to them, right? And so you can't just unleash that sociopathic machine like toddler onto their customers without a handler, right?

Jeannie Walters:

Somebody is writing a Hollywood script based on that right now,

Ovetta Sampson:

Right, right. You know that movie Baby, right. Like it's just a machine. I mean, WALL-E was cute, right Like it's like oh, that's the robot we want. No, that's not what we're talking about. WALL-E had some emotional intelligence, right?

Jeannie Walters:

I do love WALL-E.

Ovetta Sampson:

Yeah, you can't just unleash that on your customers and be like, oh, hands off, and so what it requires is treating your customers as a partnership in this journey. So that means you need to offer widespread feedback loops in many different ways so that your customers can tell you if your new employee is behaving right.

Jeannie Walters:

That's right. I love that. And I think the other thing is knowing you know, like every so often, have your customer service reps all ask the same question of the bot and see what happens Like are you getting consistent answers, Are you getting the right information? All those little like spot checks, those can help a lot. If people just know that it's not, you can't just release this and hope for the best. So we've talked about hallucinations, which is a big one.

Jeannie Walters:

The other thing that I know you're passionate about is this idea that when we talk about AI and we talk about bots and all these things, I think this is a great segue.

Jeannie Walters:

You know you just talked about they don't have a moral code, they don't know what's right or wrong, and yet sometimes it feels that way because they show bias right. So this headline really caught me. So this is from iotforallcom and the headline is when algorithms deny loans, the fraught fight to purge bias from AI. And what really struck me about this is this is about, you know, the loans that people apply for for homes, for cars, for their education, and the bias was basically that certain marginalized groups were just not getting the loans, even though they might have had the same or similar loan application information. So this one, I think, is something that, to your point, like I'm sure the people involved probably looked at this and thought what? This isn't who we are, this isn't our moral code, this is not who we want to be, and yet this is what was happening. So how do you advise leaders, advise organizations, about this bias that might be in there that they don't even know about?

Ovetta Sampson:

Yeah, and this is really, this is kind of like the heart of my work, right, like this is the heart of why I started my company and why I get up, why I lay awake at night and why I get up in the morning. Right.

Jeannie Walters:

I relate to that.

Ovetta Sampson:

the reason why I'm on this earth, I think, and the reason why I consult with companies. A AI is neither artificial or intelligence. I say that all the time. It's not artificial and it's not intelligent. Despite what you read in the New York Times, let me not go there. On this prediction of what they call artificial general intelligence All BS. It is not even . close. So that's why I like to say Skynet, not yet right. The reason is that until yet, AI starts functioning on its own data, that they're model systems. So these model that it creates by itself not by humans, we are still in control of training the artificial intelligence to give us outcomes, right. So these are model systems. They're not entities. They're not people. They're not personas. They're not any of that. They're are creating these models that are underlying the trained on currently the whole of the internet, so OpenAI, Google Gemini, Meta, Alanda, all of those big tech companies who are generative AI evolution we have right now train these models on, basically a now, souped up search engine of everything that you can find on the internet. The problem with that is who puts stuff on the internet?

Jeannie Walters:

Right.

Ovetta Sampson:

Right, like people with access to computers, people with access to copious amounts of Wi-Fi access, people who spend more time on there, people who all those kinds of things, and the people who don't put stuff on the internet are philosophers, academics or ethicists.

Ovetta Sampson:

You know anyone who are like out there living their lives, and so we have an overproliferation of Real Housewives versus you know somebody talking about Plato. So when you put all that together, not that Plato is not on the internet, but when you put all that together your model system is being trained on the zeitgeist of culture, but also all of the human, what I call trauma, that imbues culture, and that's all the isms, first of all, sexism, racism, homophobia, transphobia, all of those things. The internet follows the zeitgeist because that's what it's trained, that's what is inputted into it, and then the models are trained on that and that is the input for models. So what you have are models. It's kind of like I like to do the visualization of the fifth element, when the young lady looks at she's discovered and then she looks at the TV and then decides I don't want to save humanity because of all the wars and all the bad things that humans can do. Well, all the good things are there, but also all the bad things.

Ovetta Sampson:

And the model system doesn't know the difference. So what it does is it recognizes patterns. And if it recognized that more men have mathematics degrees, degrees in math, or more women are nurses, then when your model system, when you go to train it to recruit employees, it will just decide hey, I'm going to go with the pattern that I've seen, that I've been trained on. All the men with, with all the applications with male sounding voices or male sounding names, then I'll just put them in the to go pile right, the pile of hiring. It doesn't know that that's wrong, it doesn't know that that's discriminatory, it has no sense of that. So you have to teach the model system. So what I say is that you have to detrain these model systems to get biases or mitigate those biases.

Jeannie Walters:

Wow, and it makes me think of the first couple of times I was experimenting with image generation, with AI. I wanted a picture of, I said, a diverse set of American customers waiting in line and it came back with the weirdest combination of people, because diverse meant everybody was black in the line and they looked like they were like african tribal people and I was just like what is happening?

Jeannie Walters:

because I and then I was like.

Ovetta Sampson:

So it's mixing indigenous which first people,

Jeannie Walters:

Yeah

Ovetta Sampson:

with diversity, which is the stereotype of race.

Jeannie Walters:

It was so bizarre

Ovetta Sampson:

Yeah

Jeannie Walters:

I couldn't ever like no matter what I prompted, and then I started feeling like, oh my god, what am I? Am I a part of this? This is awful, and I think that's what you know, that. But it kind of struck me because I'm like you know what the more that humans engage, the more that they'll also recognize this, the more that they'll start seeing this as what it is. And even that article that we talked about, the headline that I talked about, I mean, there's recognition that this is happening, and so my hope is that, to your point, as they develop these tools, as they put these things out, they've already done their homework, they've already taken the time to detrain, as you say, or at least say here's what to look for and here's what we'll do about it, so that people feel empowered to do something.

Ovetta Sampson:

Yeah, this is the bulk of my company, where I work with companies to what I call de-risk their AI implementation, so de-risk the chance that their AI product will have human engagement risks. Mitigate those risks. I will never say eliminate them, because it doesn't matter how you train, how you test, how you produce or how vigilant you are. I think with these AI systems, because they're probabilistic and because they're stochastic, which means they're dynamic, that you're not the only one that owns and controls their learning patterns. It is virtually impossible to extract bias and inequity from these systems, but you can minimize those bias and inequitable outcomes. You can do that. I think anybody who says that they can totally eliminate bias from your model system is selling you a bunch of rocks, because it's not true, because A we actually don't know how biased these systems are until we start testing them right, and so I think it's really impossible to say that you've eliminated this risk.

Jeannie Walters:

Yeah, and I mean frankly, you can't get bias out of humanity, you can't like

Ovetta Sampson:

Right. And we're the blueprint, right. So these models aren't just kind of like coming out of the blue deciding what they want to do, right, Like we're the blueprint for it. Do I think there will come a day that they will? Sure, but I also think that, or I hope, that if that day ever comes, it won't be in my lifetime, but if that day ever comes, they won't make the choices that we make and that we're not the blueprint holistically right. Because holistically the culture of history and who we are, there's a lot of great things, but also, you know, we tend to have some bad outcomes as well in human history.

Jeannie Walters:

Well, I love all this talk about AI because I think it's so important and it's one of those things again that it's happening whether you're thinking about it or not, so it's better to think about it, it's better to be intentional about it and to make those decisions early on. I also know that you love to travel,

Ovetta Sampson:

Yes

Jeannie Walters:

and you just came back from another one of your many adventures and so I wanted to. This is a little bit different for our last few minutes together, but I thought this was a fun way to think about the customer experience too. So this headline is from breaking travel news and the headline is Singapore Airlines elevates customer experience with $45 million investment in Shanghai Airport, and the part that stood out to me about this is it was really an investment in their lounges for their premium customers. And what I like about thinking about this is that a lot of times when we talk to different clients about their customer experience, they kind of narrow in what that journey is. They think it's like okay when they buy the product or when they walk in or this, but when you think about the travel experience and how much that's changed in the last several years. I'm a frequent traveler, I get to the airport earlier now because I can't trust that things are going to go well right, like I just can't trust it.

Jeannie Walters:

I used to. My goal used to be going to O'Hare and never sitting down. Like I had it timed so perfectly that I would walk through security, I would get to the gate and I would board. Now I just can't handle it because things are so different. They're boarding so much earlier. They are, you know, requiring different levels of security sometimes or they don't have things open or whatever. So the airport experience has become more important to travelers, and that's what made me kind of have a light bulb about this, because you think about the journey and what this could do. Well, this might prompt people to say, well, it's a little more expensive, but they have such a great lounge, right. So, as a frequent traveler yourself, as an adventure traveler, what's your take on the airport journey as well?

Ovetta Sampson:

Yeah, you know I don't think I'm the right person to ask, because I love airports and I love hotels and I love airport lounges. I just love to just go into different airport lounges and kind of like compare and I spend so much time at airports that I'm like the opposite. I definitely get to the airport way earlier than I need to. It allows me to kind of like just relax and get in. If I was going to do work or emails or anything, I can do it in the lounge, whatever.

Jeannie Walters:

Uh-huh.

Ovetta Sampson:

I spent most of my time in airport lounges because, uh, I'm a United customer, so I go to their lounge. But I do see, the first time I ever saw kind of like the. This was even before chat GPT came out. I was in the United airport lounge and this little robot was going around collecting dishes, right, and it had like a sound and I even took pictures because people kept looking at it. They were like what's going on? And it was like I was like I'm like when I write the book, this will be the zeitgeist moment where I realized that the thing that I have been studying in the obscure, dark corners of my career has now gone mainstream.

Ovetta Sampson:

Right, there was this robot that was like going around the United lounge and just people kind of like would hesitate and they would like approach it and then like, and then it would kind of like go two steps back, right, cause it has this program, proximity alert, right. And so it was like this hesitation dance like, where like people are like what do I do with this, right? Except for the kids didn't care, they were just like all over the robot, right, and so it was. It was my. I remember distinctly thinking, and this was like two, two or three years ago. I remember distinctly thinking this is is this is an indicator that companies are going to adopt and scale these types of automated technologies in their everyday um customer service offering right and and I and I thought about the workers who, whose job it was to clean up, working with this robot.

Ovetta Sampson:

I thought about us as travelers engaging with this robot and the robot going around in circles because its proximity alert was just going off the rails.

Jeannie Walters:

There was nowhere to go.

Ovetta Sampson:

This is AI, right. They decided to create this robot to make the lounge experience faster, more efficient, all this kind of stuff. The workers don't know what to do with this robot and the robot itself is just getting tripped up and I'm like, yep, this is going to be us for the next five to 10 years. This is what it's going to look like.

Ovetta Sampson:

And then pretty soon we'll have robots taking at TSA and we'll have you know we already have biometrics through clear, like I do clear, and so they just look at my eyes and I'm like I'm there. Now TSA has it, where you can do touchless right. So all of these little things are going to be integrated into our travel experience that you won't notice it.

Jeannie Walters:

Yeah, yeah, yeah,

Ovetta Sampson:

You won't notice it. You'll just be like oh, I just got to my gate in 30 minutes and you could have just been on a part I tried to avoid and that is stepping into the robot or stepping into the airport and then going through security and then getting and doing all that stuff until I just get to the place where I feel stable, peaceful and waiting to get on the plane.

Jeannie Walters:

Yep.

Ovetta Sampson:

Right,

Jeannie Walters:

Here's to more peaceful travel.

Ovetta Sampson:

That's going to be automated. You're not going to have to like worry about that, because companies are going to realize that that's where and to me, this is the key of AI. It's not that whether AI should be there or whether it shouldn't be there. It should be, the question should be is it applicable or desirable? You, and any traveler, would love AI between the door to the airport and getting through security. Wow, because that's the horrible part of the experience. It's the most horrible part of the experience, and if you can use AI to make that experience not horrible, then AI is worth it.

Jeannie Walters:

Yes!

Ovetta Sampson:

But when you start putting AI in things that you shouldn't mess with, like I want to order my own drink, or why do I have to go to a kiosk and then I order and then the order's wrong and then now I got to go talk to a person and they don't know what I ordered. Then I got to tell them what I ordered. Like all of that that's just aggravating travelers or anybody. So you want to use AI in the worst parts of the experience, and if you use it there in the worst part of the experience, I guarantee you will be successful.

Jeannie Walters:

What a conversation. I knew we would have so much fun talking about all of this. Thank you for being here, and if our audience wants to know more about you, reach out. I think you have a special offer, so how can they reach out to you?

Ovetta Sampson:

Yes, so I do what I call the CEO Lunch, and that is where I sit down one-to-one with CEOs to kind of give them a roadmap to how to navigate AI implementation. So I'm running a special on that half off, as well as any half off of any of my one-on-one consulting. That's coaching for designers, researchers and marketers who are wanting to know more about what they can do to add AI to their careers. There's coaching and one-on-one sessions for folks who want to de-risk their AI engagement and, of course, the CEO lunch that I have one-on-one. I'm running that special right now for all of Jeannie's listeners. So, oh wait, I forgot the coupon code for that.

Jeannie Walters:

We'll have it in the show notes. So if you're interested in that, that will be in the show notes for this episode.

Ovetta Sampson:

Make sure that it's for your listeners and your listeners only.

Jeannie Walters:

Awesome. Well, thank you so much for being here. It's always so fun to talk to you. Keep up the great work. I think what you're doing is so meaningful and important right now.

Ovetta Sampson:

I appreciate you.

Jeannie Walters:

We are just at the starting line with all of this stuff, so it's cool to see where we're going and I feel better knowing that you're at the starting line with us.

Ovetta Sampson:

Yes, yes. I appreciate you.

Jeannie Walters:

Thank you so much. All right, and thank you everybody. Thank you for listening and watching and being a part of this Experience Action podcast. I love these CX Pulse Check episodes. If you have suggestions on guests or topics that we should cover, please let us know. And, as always, I am here for your questions. Don't forget you can leave me a voicemail at askjeannievip and I might answer your question on this very podcast. Do not be shy. We want to hear from you. Thanks again to Ovetta, thank you to all of you and we will talk to you soon.

People on this episode

Head Shot

Jeannie Walters, CCXP

Host
Head Shot

Ovetta Sampson

Guest