Charlene Li - Proven Strategies for Integrating AI in the Workplace

Charlene Li, author and analyst, joins us this episode to discuss proven strategies for safely and successfully rolling out AI solutions to employees within an organization. She also compares the roles of generative AI and AI agents in the workplace. 

[0:00] Introduction

• Welcome, Charlene!
• Today’s Topic: Proven Strategies for Integrating AI in the Workplace

[5:43] What are the biggest challenges HR faces with moving Generative AI into the workplace?

• Developing a strategic, rolling 18 month plan for AI implementation
• How HR can avoid stumbling through an AI rollout with a proper governance plan

[15:15] How can HR teams avoid stumbling with AI initiatives?

• Building a minimum viable team to champion the AI integration strategy
• The importance of training everyone in the organization

[31:25] How does generative AI compare to AI agents?

• Managing a team of people and AI agents
• Understanding how AI agents are designed and how to best prompt them

[42:47] Closing

• Thanks for listening!

Quick Quote

“Yes, humans need to be in the loop, but the real power of AI is when you take the human out of the loop because you trust it.”

To schedule a meeting with us: https://salary.com/hrdlconsulting

For more HR Data Labs®, enjoy the HR Data Labs Brown Bag Lunch Hours every Friday at 2:00PM-2:30PM EST. Check it out here: https://hrdatalabs.com/brown-bag-lunch/

Produced by Affogato Media

Transcript

Announcer: 0:03

The world of business is more complex than ever. The world of human resources and compensation is also getting more complex. Welcome to the HR Data Labs podcast, your direct source for the latest trends from experts inside and outside the world of human resources. Listen as we explore the impact that compensation strategy, data and people analytics can have on your organization. This podcast is sponsored by Salary.com Your source for data technology and consulting for compensation and beyond. Now here are your hosts, David Turetsky and Dwight Brown.

David Turetsky: 0:38

Hello and welcome to the HR Data Labs podcast. I'm your host. David Turetsky, alongside my co host, best friend, partner @ Salary.com Dwight Brown. Dwight Brown. How are you?

Dwight Brown: 0:48

I am Well, David, it's a beautiful, sunny day, and we're probably going to be in moving into summer soon.

David Turetsky: 0:58

Well, that's funny, because it's 41 degrees here and raining in Massachusetts, exact opposite. Yeah,

Dwight Brown: 1:04

Yeah thanks Dwight for rubbing that in right?

David Turetsky: 1:07

Yeah that's all right, but it is sunny here. Do you know why it's sunny here? Tell me, because we have with us. Charlene Li, Charlene, how are you?

Charlene Li: 1:16

Great. So glad to be here.

David Turetsky: 1:18

You are our sunshine today, our only sunshine. And for those of you who are following along at home, yes, it makes me happy when skies are gray to be on the HR Data Labs podcast. So Charlene, why don't you tell us a little bit about yourself? Sure,

Charlene Li: 1:35

I'm a long time author and analyst. I have written six books, New York Times bestseller and really focus on,

David Turetsky: 1:43

wait, wait,you can't just gloss over

Dwight Brown: 1:46

Breeze over

David Turetsky: 1:47

you just like, breeze over New York Times bestsellers. Yeah, you know, everybody's got this. Everybody does this. Well, you know, blah, blah, blah. I've written lots of books, but I haven't gotten a New York Times bestseller yet. Charlene,

Charlene Li: 1:58

well, it's fun, you know, it's, it's, you get one. It's fine. All you need is one.

David Turetsky: 2:04

Okay, all right, well, you know, it's like a Grammy or an Emmy or a Oscar, right?

Charlene Li: 2:09

It's, it's great. It's great to have that and the but by focus, I keep writing about disruptive technologies, and how do you create disruptive innovation and disrupt yourself and your organizations, and there just keeps to be new stuff to write about and to talk about. So I been in this space about three decades now, and continue to have new things. And I'm working on my seventh book out, winning with AI the 90 day blueprint for success. So that should be working out later this spring.

David Turetsky: 2:42

Outstanding.

Dwight Brown: 2:43

David and I are big techies and big into how we can integrate that into every aspect of the workplace in some way or another.

Charlene Li: 2:53

yeah and we'll talk about this. But I always believe in looking at this, at all these digital disruptions and transformations, that transformation is never about the technology, and it is always, always about the people. And we make the mistake by focusing on the technologies and not about how people are going to adopt it, how to use it, how it's going to transform them. And so that's why the vast majority of these efforts fail. So it keeps coming back to leadership and culture. And I'm a techie, geeky person that's out there. Yeah, love the tech.

Dwight Brown: 3:30

We can love that

David Turetsky: 3:32

Geeks!

Charlene Li: 3:33

But, if you don't also include the people, yeah, in your effort, managing, your thinking, it will fail

David Turetsky: 3:40

absolutely

Dwight Brown: 3:41

makes sense.

David Turetsky: 3:41

We will get into that. But first, what's one fun thing that no one knows about Charlene Li?

Charlene Li: 3:47

I train cats for fun.

David Turetsky: 3:49

You what

Dwight Brown: 3:50

What?

Charlene Li: 3:51

So

David Turetsky: 3:52

YOU TRAIN CATS?

Charlene Li: 3:54

Other people like, teach old dogs new tricks. I'm playing a completely different game. I am training my cat how to de tricks? Yeah!

Dwight Brown: 4:02

okay, so clue me in here. What kind of tricks people?

David Turetsky: 4:06

By the way, you're talking to two dog people

Dwight Brown: 4:08

yeah, with dogs that never listen to us, but

Charlene Li: 4:12

who's snoozing over there, it can do about a dozen tricks. So he can give me high fives both paws. He can turn around in circles. He'll lie down, which is really hard for him to do. He will, like, kind of do this funny Boppy thing where he just kind of bobs up on my hand. He'll put his chin on the table. He'll come and sit on my lap when I call. But the best trick is he will jump over my arm and through a hoop in my hold a hoop, and he'll jump through it.

Dwight Brown: 4:39

Get out, really?

David Turetsky: 4:41

But let me ask. Let me ask anything useful, like sitting on the toilet and going potty in the toilet,

Charlene Li: 4:48

He'll pee in the sink.

Dwight Brown: 4:51

Okay, listen, because you wanted to, or No, I mean one or two way it.

Charlene Li: 5:00

Think when his little box is not, oh, I have a little robot too as well.

David Turetsky: 5:05

So oh, okay all right, there you go. Yeah. Well, I'm not going to get as graphic for my next question as I as everybody who's listening might imagine, but what we should do is now transition to our topic so we don't get into trouble because we have no explicit ratings on any of our episodes,

Dwight Brown: 5:22

And we're a little punchy,

David Turetsky: 5:23

Yes, as you can tell, as you can tell. So Charlene, our topic for today is how generative AI will transform in HR in the workplace. And we've had lots of conversations about this, so we are extremely fascinated. So Charlene, our first question is, really, what are the biggest challenges HR face with generative AI and moving that into the workplace?

Unknown: 5:53

I think the biggest one is that we don't have a lot of good strategies that tie AI to either HR specific goals or even your business, your strategy goals as an organization, typically the AI strategy I'm putting quotes up here is a list of use cases. Use Cases are not a strategy. And more importantly, what I'm finding is that HR simply isn't at the table in the strategy discussions. They are either shut out because technologists take over and say, This is a technology issue. We don't need HR in here, or HR says, I don't need to be involved in this. This is technology. It doesn't involve people. And as we started at the top of this podcast, it's always about the people. And so what I find is that if you treat AI as a technology versus a transformational force, you're looking at it the wrong way. If you look at it as an enterprise technology that's going to be rolled out in the traditional way it's you're going to fail, because that is way too long, way too slow of an impact to have. The space is changing literally every day, and you've got to get a strategy out there that immediately drives impact and value against your top strategic goals. I

David Turetsky: 7:16

think one of the worries that a lot of us have in HR is the current strategy and the current thinking is, reduce cost, reduce headcount, and we'll you know, that's it. That's a strategy for implementing AI and HR, but it's not a good one, and it's certainly not one thought through with those use cases.

Charlene Li: 7:35

Well, I think you're missing the transformative value of AI and especially in the context of HR, because if you only look at it as efficiency and productivity, and those can be very transformative, you're missing the ways that you can engage with people that's both your customers and your employees, and completely different ways. I mean, think about training and Leadership and Development, the scale and the speed of which you can create customized, personalized development for each person based on their learning styles and their development plan is incredibly exponential in terms of the impact. So this goes beyond just saving a little bit of automation about generating training scripts. That's thinking very small. I'm thinking, you can think big, start small and then scale fast with AI. The problem is that we're not thinking big. We're thinking very small. And so as a result, we don't take these big swings. We don't start with our biggest strategic goals and say, how, what are the biggest problems we have in addressing these big strategic outcomes. And unless you think about it from a strategy perspective, thinking big, you're going to start small and stay small.

Dwight Brown: 8:52

Do you think part of it is people just don't understand AI and what the capabilities are?

Charlene Li: 8:59

Think they don't understand their strategy.

David Turetsky: 9:01

Well, yeah, I mean, that's, that's a problem we've always seen in HR. The strategy has always been administration. The strategy has always been keep the lights on and then fill in gaps, like the story of the little Dutch Boy, you know, putting their finger in the dike. It's always been a just try and be everything to everybody. And what you're saying is think more transformatively about how we can offer better, more personalized service that doesn't scale with people, it scales with the available technology. Is that correct?

Charlene Li: 9:32

Right? Well, again, this is a very interesting time where you can scale and grow without adding new people, without a huge amount more expense in a fraction of the time. You've never had this opportunity before. It is completely scale level difference in the way we operate. And so you do have to step back and be more strategic. And this is the transformation that HR has been going through for the past 15 years. I. We were so bogged down by the administrative aspects of recruiting people, engaging people, retaining people, again, just trans, just like all that administrative aspects of it. And now technology can remove a lot of that administrative overhead and allow you to think much more strategically. But we haven't trained ourselves as HR professionals on how to even ask strategic questions.

David Turetsky: 10:25

Well, so to Dwight's point before, though, isn't that? The bigger problem is we need more help building that strategy. We can't really go to anybody else inside the organization. They're going to be like, Are you shitting me? Pardon my French. You didn't know how to develop a strategy? Why are you coming to us now? This should have been done years ago. You know what I mean?

Charlene Li: 10:46

Yeah. Again, better to acknowledge it that, yeah. So let's go do it there and say, I don't have a strategy. And again, a strategy is fairly straightforward. A strategy is just a set of integrated choices that you make to achieve your business objectives and to win. So align them against your overall core strategy as a business. And here's the thing, most businesses cannot tell you what their strategy is. So it's not just HR, it has a problem. It's the entire executive team and the board. The hardest part about AI and strategy, it causes you to make you focus on what is most important to us, integrated Choices, choices, what we will do when we won't do achieve our goals. If you are not aligned on what those goals are, you can't even begin to make decisions. Which is why we default to the efficiency and productivity, because that's within my purview. I can look at these small, little problems. I don't have to deal with alignment and harmony and making sure we're all on the same page. Fundamental problem with just overall strategy inside of our organizations.

David Turetsky: 11:54

So one of the things that I wanted to ask was a lot of us did a lot of us do in the HR world, build a strat plan. You know, every year we have a strategic plan for what we're going to try and accomplish next year. I don't think what you're talking about is developing anything broader than that. You're just saying, Is there a way of looking at that strategic plan and being able to accomplish that a different way, more efficiently, without having to throw money at it without having to throw necessarily people at it. I mean, you might have to throw some money at it to begin with, but you're gonna get a much better expectation out of it if you're using potentially Gen AI instead of just powering through it with people.

Charlene Li: 12:37

Yeah, again, I think that's a very efficiency and productivity right? To look at things and frankly, that strategy plan for the next 12 months, it's an execution plan and a budget, right? And as you get closer to the end of those 12 months, your plan shrinks and shrinks and shrinks. What I really advocate is a strategy plan that helps you achieve your overall business objectives as an organization. So it's longer term, long term, whatever that looks like. It can think about three to five year horizon, but I encourage people to think about it as 18 months, six quarters, and each quarter you're laying out what is the impact and the value we are going to deliver every single quarter. And this is more than just administrative things. What is the change and the impact you're going to create in order to help us achieve our long term goals? So if you are creating that change, then how can AI help you accelerate that change or achieve that change? Because there are a lot of barriers in the way to make that transformation happen. And the key is, at each at the end of each quarter, you're evaluating how far you've come. You're making adjustments to the next five quarters of what the impact is going to be, and then add on another quarter. It's a rolling 18 month plan, and you shouldn't have to redo it every single quarter. It's an extension of what that plan looks like, and it's far enough in the future that it is actually strategic.

Dwight Brown: 14:05

So, the strategic plan you're talking about is that a strategic plan specific to AI, or are you saying look at your business strategy and work backwards from there and figure out where AI can help you achieve that strategy,

Charlene Li: 14:23

Absolutely, because the your C suite and your board won't care unless it helps you create your business it helps you achieve your business objectives. Creates competitive advantage. They won't care. It's like, yeah, go to go deal with it in your size, your department. But if you really want to have an impact and to transform the organization, then help us achieve our big, audacious goals that we have set for ourselves, identify the biggest problems that lie in the way of us achieving it. What are our biggest challenges, and help us understand how HR and AI are going to help us get. To those goals, better, faster, cheaper,

Announcer: 15:04

Like what you hear so far? Make sure you never miss a show by clicking subscribe. This podcast is made possible by Salary.com. Now back to the show...

David Turetsky: 15:14

Charlene. One of the things that troubles me though about adding AI into the mix is that HR typically stumbles when it comes to these initiatives anyways, but because AI is still in its infancy, it still has its training wheels on in in reality, we also don't have these private clouds or these private instances yet enough in organizations that will enable us to have these types of strategies well thought through, without it leaking to others by saying, Hey, tell me some examples of other companies that have gone and improved their HR strategy this way. And all of a sudden, chatgpt just spits out our company name. Oh, well, salary.com did, by the way, salary.com just an example of a company name. Should it just? Probably just a company name. But you know what I'm saying is, to me, there are so many barriers from the technology being nascent. Is there in your mind as you're thinking through this and as you're espousing this, is there still a problem of this kind of going up the maturity curve yet. Or are we? Are we ready for this now?

Charlene Li: 16:26

Well, first of all, one of the first things we encourage people to do is to get your a core team of people trained on how to use AI in a safe and secure way. And that's you can do that simply with chat GPT teams, if you put that in place automatically, all settings to feed the model, any of your data is off, just automatically. It's all private, then it's 100% private. Okay, so that's the system I use. You know, can you trust them? Well, if they can't do this securely, then the whole entire business model is is put but trust them. And then the second thing is, you want to put in place the governance models for responsible and ethical AI. You build what I call it an AI trust pyramid of safety, security, fairness, all those things, all the way up to transparency. Then make sure everyone understands this is what responsible and ethical AI looks like. It doesn't have to be complicated. It's built on the foundations of existing data policies and security policies you have in the company. And then third most important is to give people access broader across the entire organization. Right there are off the shelf solutions you can get from your existing cloud providers that can just turn it on and it's completely locked off. I talked to a global organization. They have 65,000 people, and they run AI for everybody for a couple $100,000 a year. Yeah, we're talking like this is like a small little sludge budget within the IT department.

David Turetsky: 17:59

Sure. I guess I'm a little more concerned about those people, each of those employees who have not heard about that yet, go to 1min.ai which is a off the shelf aggregator of of lots of different Gen AI models, and you buy, literally, can buy a lifetime license for $69 and have access to a bunch of credits per month, but that is completely public. I mean, you can set all the settings to be private, but Charlene, the thing I'm worried about is that that thing you mentioned about rolling it out to all your employees, if you don't do that and you haven't done it yet, believe me, each one of your employees has created their own private little cloud. They have not made those privacy settings private, and all of the things that they're asking about Gen AI, Gen AI in your company are being not necessarily broadcast, but they're being absorbed.

Charlene Li: 18:55

This is why it's so important to lock it down right, and the more you can catch it with your IP nets. And anytime they go to these things, they get redirected to your authorized, safe and secure way that you're locking down that technology. It's logging things, but it's removing anything from that log within an hour. So nothing is keyed in secure. It's all software compliant. So even the transmission lines are secure, everything is locked down. So even if somebody inadvertently uploads something into your private cloud, it

David Turetsky: 19:28

What about copilot or Apple intelligence or doesn't stay there. Gemini or whatever it's called these days? What about the ones that are more ubiquitously available within the applications we use every day, either our phones or Microsoft Excel has or Word has co pilot built in.

Unknown: 19:46

Yeah, those are all locked down again. It's in agreements that you have, if you have copilot, if you have any of the Microsoft tools, again, on your systems, those are locked down so they're in the MSA so you can be secure by that. So you want to work with your vendors that they understand you're responsible in AI ethics and guidelines. So it's like your entire ecosystem is on the same page. Want to make sure that all the vendors are applying and abiding by your safety and security privacy practices. You do that anyways. You do that with AI.

David Turetsky: 20:19

Well, they may not have had that done yet after listening to this, hopefully they will.

Charlene Li: 20:25

The thing here is that safe and secure access to AI and that you have you can proceed with it with confidence, with trust, is vital to your use of AI, and if you cannot have that as a foundational building block, then you cannot go forward. We are very, very keen. My co author and I are very keen on making AI readily available, expose it as much as possible. So because once you have that safe and secure platform, you can blow the doors off of this. And the CIO we were talking to said, I don't care if they use it, if they use it, if they don't, I really don't care. And we're just going to make this like electricity. It's going to be ubiquitous, and they can have it, and I don't have to worry about ROI because it's so cheap. But I know that there's impact, because if I try to take it away from them, they're like, over my dead code, fingers will you again? I he can't measure the value, but in an organization of 665,000 people globally, to spend $300,000 a year on AI for your biggest safe access to everybody, nobody cares about how much it costs, all those questions around ROI, just go out the window, and then you can focus on, what can we do with this when everyone has access my goodness of things that come out? And then it's a focus of saying, efficiency, productivity, go, go, do it. Just no brainer. Just do these things, right? We don't really need to really think too much about them. If it's helping you do your job better, just do it. Stay within these guidelines that we have and go for it. But what really has to happen is we need to know strategically, how are we going to use this across our business so that it drives the biggest bang for the buck to help us achieve our switching goals. That is a strategy question that your C suite and your board level should be talking about, and HR needs to be a part of that mix, because you don't get that level of transformation unless you're thinking about the people aspect of things.

Dwight Brown: 22:36

So does it start with HR developing a strategy and move up the chain? Or how do you see this working best?

Charlene Li: 22:45

There's a minimally viable team, minimum MVT, of somebody who's thinking strategically, the highest level strategy person you can get in the organization. You get somebody from HR who can think about people and their adoption and use of these things. You have somebody who is thinking from the digital and technology side. It's not your traditional technology department, it's somebody in digital and then you get somebody who understands the customer, who has that perspective. It could be somebody in marketing and product in the commercial side, but somebody who can encourage the team to be customer centric, whoever that customer is, and so that's a very small team. And buy in from your legal team to like, Hey, we're going to abide by all these rules, but outside of that, and inside with those guardrails, we will stay within that, but they don't belong in the room, because they're the departments of No, they're not the ones who think deeply and imagine what the work could look like, and if they promise to never say no, the minute they say no, they get

Dwight Brown: 23:51

terms and conditions that you have them signed before they can walk in the door. You know,

Charlene Li: 23:55

Like, do you really want to be a part of this conversation? Yeah, exactly careful what you wish for. But you know, we're not going to implement anything until we run it by you anyways. So you don't need to be here to say no to something that we haven't figured out how to do yet. So by having this really small team that is going to be very strategy based, you have a winning you have a fighting chance to really focus on the big problems that you can solve with AI.

David Turetsky: 24:23

One of the things that I worry about, or that I'm encouraged by, is, if this tool is so ubiquitous and everybody's got access to it, the one thing that we're going to need to do, I think you mentioned this the beginning, is train people, is to give them guidelines, is to tell them how to write a prompt or to what tools will you use? How will you use it? You know which ones are the right tools for you, even if it's just one, even if there's one product that they use, making sure they don't go to other products don't use their iPhone, those are all important things to make sure they understand. And and just sending email won't work. So training is going to need to be key here, and making sure the trainers spend some time with that team, the MDT, to be able to roll out the appropriate level of communications, change management, whatever it is, webinar, because everybody wants to try and use it. Everybody is waiting for this to, like, come out so they can go, Oh, I'm going to put an email in it to see if I can make it better. Okay. Is that the right use of your time? That's what this training should answer as well.

Charlene Li: 25:33

Yeah, I do believe people need to have a foundational level of understanding, and yet they may have it right content for them in the beginning, and I really believe in training the top two levels of your leadership as soon as possible. These are the people who understand the strategy of the organization, or at least they should, but they are more likely to think strategically about the use of this, this tools. So the more they can transform their own work with AI, the more likely to see the transformational power and potential of AI for the organization, but they have to experience it themselves. I was recently at an HR conference, and we did a poll to say, how many of you, how often do you use these things, and most people were using it a couple times a month, a couple times a week. And 6% of them said, I couldn't live without it. Just 6% about 500 people, just six that's 6% Wow, yeah. So I encourage, first of all, for those people to raise their hand, stand up, so other people could see them and go go talk to the rest of the conference. But that's where you need to be. Maybe not at that level, but you need to be using it to do the work that you do every day. So there's a huge hump to get over again. Trust is a big part of it. Example, prompts. I do believe in having prompt libraries and the also, I just really train people to do three things, to create content, to do research and to use it as a thought partner. And the higher up you are in the organization, the ability to just use it as a thought partner, as a sounding board, becomes more and more important. So habit review something, give it a problem that you're wrestling with, give it a situation with a an employee, a tough conversation. You have to come up and ask it to help you do a script or do a role play. And what I encourage them to do is ask to add this one line at the end of any prompt that they write. Because most people, we're not very good at writing prompts. So you ask the AI to prompt you by asking this question, you add at the end, before you start ask me any clarifying questions you may have interesting.

Dwight Brown: 27:51

I could really use that one.

Charlene Li: 27:53

Yes. What happened the table? The AI then starts asking you all the questions that it needs to make to do that task that you've given it.

David Turetsky: 28:04

Why Charlene? Why wouldn't that be natural? Because when we have conversation, we're doing it right now we if I don't understand something, I'm asking you, you know reality, I'm asking you so I can clarify and understand, so I can then the next thing coming out of my mouth, which usually should be somewhat smart, but usually isn't, especially in this conversation, would be a better output. Wouldn't normally that be the kind of...

Dwight Brown: 28:33

But the system, the system doesn't know what the system doesn't know, I think, is, the

Charlene Li: 28:38

problem is we're treating AI like a Google search. We put it in and we get an answer. We don't like it. We put in another question and try to get another search in. AI is intelligent. It has a ton intelligent. And so you can treat it almost like a person. And so I treat it like a research assistant. So I'm like, here's a task, I want you to go do it. And what do you say to a person? Do you just give them the task and then, like, shoo them away to go do it, push a button, tap and go out the door. You go, Are you clear about my instructions? You have any questions for me? That's what you would normally do. So treat the AI like a really smart assistant who needs more clarifications. And the most natural thing you would do is like, here's what I want you to do. Got any questions for me?

David Turetsky: 29:23

But that goes back to your point before, of being able to understand how to make prompts appropriately.

Dwight Brown: 29:30

Prompt engineering, yeah,

David Turetsky: 29:32

Yeah. Prompt engineering classes

Dwight Brown: 29:35

on,

David Turetsky: 29:36

yeah,

Charlene Li: 29:37

yeah. It's again I usually will include an example that says these are the key components to include in here. Here's your role. Like your role is to be a business copywriter who's an expert on HR communications. Here's the instructions I need you to write a 10 minute script for a presentation I'm doing a college recruiting event. Here's some context. This is who the audience is. There are a bunch of engineering students in this club, and they're curious about what it's like to work at your company. And here's some information about the company, about how we have college internships, how what early careers look like at the company. And then very clear output. Give me a 10 minute script divided to three sections that I can do and, you know, go to the races and then ask me any clarifying questions you can have. But even giving people that level of prompt, and by the way, all these engines have prompting FAQs in them, and they're getting more and more sophisticated, but at a basic level, these are the kind of core things you want to include in a prompt. And we also have these beautiful things called AI agents coming along now, because prompting is

David Turetsky: 30:52

now we're going to make a left hand turn and talk about agents. Well, that's a bigger one. Hey, are you listening to this and thinking to yourself, Man, I wish I could talk to David about this. Well, you're in luck. We have a special offer for listeners of the HR Data Labs podcast, a free half hour call with me about any of the topics we cover on the podcast or whatever is on your mind. Go to salary.com/hrdlconsulting to schedule your FREE 30 minute call today. And Charlene, I know we've gone a little off script, but, but let's now talk about AI agents, because that is kind of the next we were talking about from Gen AI, which is more of a to your point before a research partner. And now let's talk about agents which are, think of them as your administrative assistant, right, or administrative partner, right?

Charlene Li: 31:46

Think about them as peers. Or again, one of the things I just did was completed a course on, how do you manage a team made of a people and AI agents like, how do you think about this? It's no longer this passive technology. You have a bunch of things that has to be done your team to create some output and value. You have people who can do it. You have AI agents who can do it, and you have a combination of the hybrid of the two of them that could do it. So how do you figure out which is the right combination to use? And the reality is, you're going to have to constantly adjust for it, because these agents are going to do a lot of that automation. They, in combination with a person, would do a lot of this prompting for you. Basically, agents are, think of them as orchestrators. They will take and understand a particular task you want done, and it'll go and look for the right information, the right data. It'll feed it to the AI engine, the generative AI engine. It'll come back evaluate the quality of it, and if it's not up to snuff with the quality, it'll send it back and say, do it over again. And then finally, give you an answer. So all those steps we've had to do in the past ourselves, manually with generative AI agents make it a lot easier, and they can learn over time. What does good quality answers, what do they look like? And just improve and learn alongside humans on a team.

David Turetsky: 33:14

But there has to be a step for humans in there to check it, because we've heard, and I've especially we've had some guests on the podcast, Dwight, and I've said, Well, you know, AI lies. It doesn't really understand crap up. So don't we have to have a check in there alongside the agents to make sure that the agents are being checked?

Charlene Li: 33:36

Yes, that's what these agents do. And frankly, that that description of it lying is putting some sort of judgment on what AI is doing. You could think about it as making things up, lying again, but treating it as a Google search engine, that's not what it's really good for. If you want an answer, go to Google search if you're looking for somebody who can be creative, right? When you're writing something, you wanted to understand your style, you want to understand the facts, and you can tell it to be very careful, don't make things up. This is very important to my career, and it won't so there are ways, again, you think about it as an over eager intern who wants to make you happy. They will do everything possible to make you happy, including make things up until you tell them exactly, so that, again, they need instructions. And what the AI agents do, it gives them very detailed instructions. Go and look only at this data for this particular problem, and it understands your problem even like are you asking about just how many vacation days we have, or are we looking at and evaluating what makes a great employee and understanding something beyond what's in resume to understand that full person? Those are two very different activities, completely different types of data. You want to be looking at different reasoning. So you would approach. It the same way. And that's what these agents do. They understand your request, route it the same way. And then has these, what I call judging, or critic AI, that sits on top of it and says, Is that a good answer or no? And so that critic is trained by humans to say, Yeah, act on my behalf. Get rid of all those things that don't match what was quality, send it back, and then train the foundational engines to be better, because that's, in the end, how you train and over your intern to be a great employee. Same thing with agents, same thing with AI. You've got to train it.

Dwight Brown: 35:39

Now I'm just dying to go back to chat GPT and start figuring out the fun tools that I can try.

David Turetsky: 35:46

Well, yeah, and I think Dwight, that's the problem. Is you have to actually be able to exercise these muscles to learn how it all fits together. And then how does the agent fit in the use cases you're talking about, on top of it, if it was the strategy, then, to me, one of the best things about the AI agents would be to help notify you or inform you when decisions have been made or situations have occurred that seem like they could be trending you away from your strategic goals. And I mean not to tell tale on my peers or on my my friends in my group, but then to show me the examples of where this is so that I can go correct it.

Charlene Li: 36:35

Well, I know some people are starting to use these agents as true co pilots and true leaders alongside them, because there's too much happening. There are too many reports to look at. I have too many team members and employees, especially the higher up you go in organization. So they're using AI to build those notifications for them to say, I will routinely look for these things. Look for these things. If anything is out of order. Give me those exceptions, and give me recommendations on actions I can take. And then over time, you just go like, Hey, I trust you, because you keep recommending the right actions. Just take them. Just do it. And that's how AI you begin to automate, is that it's showing its work, it's giving you the logic. And over time, you're like, you don't need to ask me anymore. Just go do it. And the things that you begin to trust AI to do comes through uses. You just don't say, Yeah, I trust that you as a vendor, can do all these things. Go for it, just turn it on. No, that's not that's never the way it works, right, right? You wouldn't do that people. Why would you do that with tech? So, yes, humans need to be in the loop, but the willpower of AI is when you take the human out of the loop, because you trust it, and you trust it to do the job it's been meant to do, and you trusted that it could probably do it better than a human could. And that's where things get really interesting, because it calls into question this existential issue of, what does it mean to be human? Because if AI can do all these things, then what are we? Who are we? If AI can do all these things,

Dwight Brown: 38:10

the million dollar question, yeah,

Charlene Li: 38:12

and AI still needs us ask the questions. It needs us to imagine the future. It can't do those things.

David Turetsky: 38:21

I think Charlene, this goes back to when we started using computers and the same. I mean, back in the early 90s, late 80s, we were still in the working world. We hadn't really fully bought into computers in the workplace yet, but it started to get into many jobs. You can't even imagine them not using computers today. And they there were some thoughts of, well, you know, this is going to take the place of so many jobs. Well, yeah, it did. How many administrative assistants do we see? How many receptionists or secretaries or mail what do they? Call them? The mail room. We don't see mail rooms anymore, really. You know, you have an inbox at the front desk, and people kind of pick their stuff up, but there's a lot more capability and functionality we have now because we use computers. And I think it's the same paradigm, isn't it? It's, it's kind of, we're at the nascent stage. We're back to the 80s and 90s, trying to figure out how this is going to improve what we do, not necessarily just replace us, but how it will improve what we do.

Charlene Li: 39:27

Yeah, there's a saying that's been out there for a while that it's not AI that's going to replace you. It's somebody using AI who will

David Turetsky: 39:34

Yeah!

Charlene Li: 39:35

So learn how to use AI, and the people who most strenuously object to AI, my first question to them is, have you used it?

David Turetsky: 39:45

Yeah,

Dwight Brown: 39:45

right.

Charlene Li: 39:46

And havent, I'm like, if you have then I can have a rational discussion with you. But if you have it, then let's get you on it to see really what it does. And I understand these are big concerns, and it's. Gary, and it's AI, it's Skynet coming to take us all over and everything. I get it, but the best way to understand it is to use it, and then you can understand what it's capable of doing and also what it's not capable of doing, and understanding the technology can strip away a lot of the fear. And I'm not saying that AI is fantastic, automatically, a great thing. I'm optimistic about its uses, but it requires people to put it against those good uses. And in the absence of that positive, optimistic view of AI, we have the people who are using for nefarious reasons. And the only way to overcome the negative effects of AI is we use it for good. So we need more people using it for good. So if you're worried about the impact of AI, then find ways to mitigate the bad and make sure it's being used for good.

David Turetsky: 40:49

And I think that's the mic drop moment right there, Charlene, so we're gonna leave it at that. There's so much more we could cover. And I feel Dwight. I don't know what you think, but we could talk to you about this all day.

Dwight Brown: 40:59

Oh, for sure, yeah.

David Turetsky: 41:02

I mean, there's so many things that we can go into on this Charlene, right?

Dwight Brown: 41:06

You can see the wheels turn in our heads right now

David Turetsky: 41:09

Dwight and I are both thinking, you know, we should go back and I've got mine, my, as I said, I have 1min.ai over here, and it's up. And it's normally up on a daily basis. And I do ask a questions. But you know, to your point, I am at my nascent stages of being able to be a prompt developer. Because, right, I don't get it yet. I gotta get I've got to do training on it.

Dwight Brown: 41:31

Yeah, I mean, you go to Coursera or edX, and they're all kinds of prompt engineering courses out there. I remember the first time I saw that. I'm like, what?!?

Charlene Li: 41:43

Yeah. And the thing is, I encourage you to take those courses, but literally, go look at open AI's guide to prompt engineering. They'll lay out like these are the six things to make better prompts. Cloud has Gemini has errors. Read through those, and you're good to go. And if you want more than go and look at the other things. But it's pretty basic, and it's kind of like we know how to optimize for Google search. We had to learn that over time. You have to do AI. I do believe that little question at the end you train you like this is how to ask me. This is you need to give to me, and instead of trying to write the perfect prompt, prepare yourself to have a really good conversation with AI.

David Turetsky: 42:26

So just so we're clear, especially for the for the transcription, what was that one question

Charlene Li: 42:34

Before you start, ask me any clarifying questions you may have.

David Turetsky: 42:47

Charlene, thank you for being on the HR data labs podcast.

Charlene Li: 42:50

It was fun.

David Turetsky: 42:51

And Dwight, thank you. Thank

Dwight Brown: 42:52

you. The wheels are turning.

David Turetsky: 42:56

Smoke coming out of both of our ears. See the video because you had all kinds of ideas at work. So thank you very much for listening. Take care and stay safe.

Announcer: 43:08

That was the HR Data Labs podcast. If you liked the episode, please subscribe. And if you know anyone that might like to hear it, please send it their way. Thank you for joining us this week, and stay tuned for our next episode, stay safe. Y