Copilot Adoption, Governance, and ROI With iuvo's VP of Technology

Most organizations already own Microsoft Copilot. Far fewer know what to do with it.

Brian St. Marie, iuvo's VP of Technology, says Copilot adoption is now the single most common AI conversation he has and it's the same conversation whether the company has fifteen employees or several thousand. In this episode, Bryon Beilman and Jess DeForge dig into why that is, and what to do about it.

The conversation covers the free Copilot access already included with every Microsoft 365 license and what it does and doesn't reach; why adoption and governance are two separate problems that stall each other when treated as one; the SharePoint permissions and data hygiene issues that surface the moment Copilot can search your tenant; and why top-down AI mandates and usage leaderboards tend to backfire.

Brian and Bryon also get practical about measurement, the adoption data Microsoft already gives you, what leaders should be asking for, and how to think about ROI on an investment whose target keeps moving. The episode closes on the most common mistake Brian sees: organizations that skip governance at the start, then discover where their data has been going.

Transcript

SPEAKER_00 0:03

This is The Edge of Excellence, empowering people to shape the future. Let's inspire, innovate, and explore together. Welcome to the Edge of Excellence. I'm Jess DeForrts, and today we're talking about the AI tool almost every company now owns, Microsoft Copilot.

SPEAKER_01 0:19

And I'm Brian Bilman, and joining us today is Brian St. Marie, AIVO's VP of Technologies. Brian spends his days helping organizations implement AI the right way, from securing enterprise data to getting tools like Microsoft Copilot to actually deliver. If you've bought Copilot and you're wondering whether it's paying off, this is the conversation for you. Brian, welcome back to the Edge of Excellence. Thank you for having me back.

SPEAKER_00 0:45

Yes, Brian. Excited to have you back. You and I have had some conversations offline about Copilot. And you said copilot adoption is the number one conversation that you have with businesses right now. What are people getting stuck on?

SPEAKER_02 0:59

Yeah, it's sort of funny because you know, we have customers that spread the whole uh spectrum of company size, you know, from like 15-person startups to uh multi-thousand-person enterprises. And when I have this the kind of initial AI conversation with any of them, it's always the same thing, which is like, you know, what what do we do with Microsoft Copilot? Um, which is pretty surprising to me. But uh I think everybody, literally everybody is is in that same boat, other than maybe the most cutting edge organizations. Um and um yeah, I think that it's it's really just like how to start, right? Like how do you how do you get going with it? Um they know that they have access to it, sort of. Um, you know, I think a lot of companies don't realize that any Microsoft 365 licensed user has access to copilot for free. I think most organizations don't even realize that. That's a great place to start. Um and then, you know, the next step on when they want to license it and get a little bit more um more advanced.

SPEAKER_00 1:54

So when you and I tried it, it seemed like there were two separate challenges, one being to get people to actually use copilot. And then the other is the governing it responsibly. How do you tell them apart? How do you attack those problems?

SPEAKER_02 2:10

Yeah, um, I mean, they're very different problems. The the you know, getting started uh is is as simple as what I just said, in that you can, you know, you can uh uh basically get any user using Copilot right away um without really any additional expense. And you don't have too much of a governance issue there because that that free version doesn't give Copilot access to internal documents or anything like that. It's pure purely like chat and web. Um but it it does fall under Microsoft's enterprise data protection agreement, which every organization already has by virtue of the fact that they're already licensed with Microsoft. So just like your email is protected, your SharePoint is protected, those chats are also protected. So people could share files into the chat and so forth, and there's not really any risk to that. Um it so that's very simple, no real governance, you can just let kind of people go wild with that. Um, once you actually go to the next level and you license copilot and you're paying, paying an additional fee for it, um then you want to start being a little more concerned about what co-pilot can get into. And that's really where the governance piece comes in, because um you know, a lot of organizations have years and years of SharePoint history. Uh and there might be old SharePoint sites that you know were created years ago for the, you know, the when the company was young and people forgot about it. And you know, it copilot will find those things. So when you ask it questions, it's gonna search your whole tenant. And if it finds something, it's gonna, it's gonna you know present it to the user. Um it does operate within the context of the user's permissions, but because those permissions may not have been um audited well or maintained over the years, you you do need to kind of be careful about that. So like that's kind of the first level of the major governance piece um once you you know start going wild with with Copilot internal in your organization.

SPEAKER_01 3:51

So it seems like uh there's there's a there's a couple things here. There's there's governance I think of as you want to make sure that the data is protected and that you are using it appropriately, the AI can use it appropriately and not what you think. But you mentioned something there that uh as a user, you know, and you work for some company for a long period of time, you're an enterprise user, and you've got you've never cleaned up your home directory or whatever, and and so you're searching for a result of something. Now maybe it's good. Oh, I finally found that document I was looking for for a long time ago. Or it could be, I'm gonna give you old data, I'm gonna give you old information. So there's there's uh there's a cleanup impetus there, but as well as governance. And I think kind of a little bit, they're a little, they're similar, but they're different, right? I mean, I think there's like uh wow, why do we have so much old data around and we got to clean it up versus I need to make sure this is protected. So they're kind of two aspects of that.

SPEAKER_02 4:41

Yeah, I would say governance really is, you know, what what what is what can it access and what is it doing, right? Um whereas that that cleanup piece is more just data hygiene. And that can be part of the you know, reviewing permissions. Um and in fact, with the Copilot license now, you get a SharePoint advanced management license for free because Microsoft recognizes that this is the number one challenge with adoption of AI once you once you have access to your internal data at will. Um and so they include the SharePoint Advanced Management License now, which gives your IT uh teams the tools they need to do these kind of like org-wide audits of like what's going on in our SharePoint environment. So um, you know, just getting one person licensed with copilot gets you access to that SharePoint management, which are advanced management tools, which you can now use across your entire tenant and benefit everybody in the organization. So it's a really good value. Um I do think though that as time is going on, that the concern about data, uh bad data versus good data is becoming less of a concern because the the models themselves, like the the models that the AI models that run Copilot, are getting smarter about being able to tell what's relevant and what isn't. Um and Microsoft has their secret sauce in the background as far as like what data it's presenting to the agent when it searches. Um that is also you know helping to um weed out that kind of like noise data. So I think it it is naturally improving anyway. Microsoft on a macro level recognizes that this is a challenge. And so their kind of two-pronged attack is one, give you the tools you need to do your own cleanup, and two, try to make their algorithms and their models smarter so that they are also kind of like navigating, you know, whatever mess you might have.

SPEAKER_00 6:16

So I'm curious, Brian, you know, say a company bought the licenses, so they now have the licensed co-pilot access. Really, that's just the start of the journey. How do you recommend people get adoption? And do you think that that's a challenge that you know businesses are having? It's like, okay, cool, we have it. We have access to it now. Now what?

SPEAKER_02 6:40

Yeah, I think it's absolutely a challenge. I mean, we had a conversation with a a large, you know, uh multi-thousand person enterprise recently that was uh basically like, we, you know, we've got Copilot for everybody, but like doesn't seem like most people are using it. We're not even really sure how many people are using it, which like there are the tools there in the environment to to let you do that. Um, and and I think that you know it's it's it's one thing to just throw a license at everybody, right? Like if there's a there's been over the last year like a big top-down approach in a lot of businesses where it's like leadership says everyone's got to use AI. Here's our token leaderboard and all this nonsense, and just kind of like pounding it at people. Um I think on the one hand, you know, you you want to make sure that you're you're uh making your business AI forward. Um, on the other hand, you don't want to jam it down people's throats, and you definitely don't want to be prescriptive about how they use it. Where you see the biggest impact in organizations is by giving people the tools and letting them use them in their day-to-day, however they think is you know will help them. And then you will get that that acceleration. Separately, kind of this macro level, how is the organization going to implement AI on a on a on a you know org level scale is a different different problem. Um and Copilot is really the first problem, right? Like let people figure out how it works in into their um their daily jobs and and you know benefits them. Um and so how do you get the adoption? You know, back to your actual question. I mean, part of it is obviously licensing them, part of it is making sure you're monitoring the usage and understanding what groups are are accelerating, what groups are not. And then part of the the real uh the real um enabler is like getting that group of folks that are kind of like your, you know, your superstars with AI to feed their best practice to the folks that are maybe the laggards, right? Um so an example of like what we do here internally is like having an AI use case is no channel where people share the different ways that they've been using AI that they found have been really beneficial to them, it gets people's minds going, and like that will kind of cross-pollinate um usage and ideas and get people to start start adopting it.

SPEAKER_00 8:44

Yeah, and we we had an off-site recently, and I think it was you, Brian, who mentioned this kind of change in the way that we think about problems when you start using AI, um, and how that is like a goal for us is to kind of like continue to help support team members to adopt AI and to kind of change their brain essentially in the way that they're attacking problems by utilizing AI. Um what do you think that that looks like realistically in a company? Who is responsible for that, in your opinion? So is this like an IT department that's responsible, or is it really important that leadership is a part of this? And how do those two kind of work together to trickle this down?

SPEAKER_02 9:29

I leadership absolutely has to be on board. Um, we've seen this as well with organizations where the employees are asking for AI and leadership is like, we don't want to deal with that, or we think it's a fad, or like whatever, right? So and and those organizations are kind of stalled. And an interesting thing that happens is a lot of times the users kind of circumvent that, and then you know you have a shadow IT problem. Um so I think that leadership absolutely has to be involved. Obviously, IT needs to be involved from the governance standpoint and just the deployment standpoint. Um, and you know, you really it's it's you know, your culture in HR needs to also kind of handle that like engagement piece of things. You know, how are we um engaging with the users that are somewhat you know resistant to it? Um, because it's also it's also a benefit for everybody, right? Because to your point about rewiring your brain, um, you know, rapidly AI skills are going to become required for any job, right? And so it takes it takes a certain amount of time stumbling around with it before you start to think differently and how you can leverage AI in your day-to-day business. And so doing that in your current job is is much preferable to then you know having to try to find a job and learning, you know, trying to uh figure it out while you're looking for work or in a new job or whatever. Right. Very similar to like the advent of the internet and that kind of thing. Like you needed to know how to use computers in order to get a job. Similarly, now you need to know how to use AI to be valuable in your job and continue to you know benefit from that.

SPEAKER_00 10:53

I feel fortunate because I have access to people like you and others within IUVO. So as far as upscaling myself, I have some really great resources, but I've also found AI to be a very good tool to teach me how to use AI. So I think that encouraging your team members to do that is also really helpful.

SPEAKER_02 11:10

Yeah, interestingly, like we get that question from companies that'd be like, Well, you can just do training on like how we can use copilot. And the answer is just ask copilot. Like, yes, we'll do training, but our training is going to mainly be like how to open copilot and ask copilot, because it it literally is like the you know, tireless tutor that's just available there for you. So um getting people beyond treating AI as a glorified search engine and asking that second question, the follow-up, is like 90% of the game as far as that first entry level of AI skills. You know, once you're you're getting to a point where you're following up on the question that you, you know, it gave you an answer and you're like, well, how why did you say this? Or what you cited this source, why? Like that that will get people to start moving. And once they start moving, their I their you know, their thought process starts to change and they start thinking of new things they can do. Um and that, you know, this is all stuff that um everybody should be able to do today. I think like moving into the more agentic stuff, which is really topical now today, um, is also extremely important. But so many organizations and so many people are still kind of like in that you know glorified search engine phase.

SPEAKER_00 12:18

Now, if there are leaders that are listening or watching this episode, do you feel that there are maybe a few metrics for them to watch or be asking for that kind of help them measure the ROI on the investment when it comes to Copilot?

SPEAKER_02 12:33

Yeah, Copilot provides great, there's literally a dashboard and there's an executive report that you can provide to your executive leadership. Um, the dashboard dives into more details too, but it will tell you, you know, adoption rate, which is really important. Like, okay, we've licensed everybody, how many of those people are actually using the product? Um, it'll tell you where they're using it because copilot exists in a lot of different places. Like there's the standalone application, there's SharePoint, Copilot, Excel, Word, PowerPoint, et cetera. Um so it'll tell you where they're using it, which is also really important. Um, you can see like, you know, that what I was talking about before, like, are they are they interacting with it? Like, are the chats multi multi-message or are they just one-offs? You can see that kind of stuff. Um, so so all that data is really valuable to just try to get a sense of, you know, provided your team is educated on like what we should be looking for and what do these things mean? Um, it'll, it'll, it's a great indicator of like how quickly are we adopting it? And you can see the trends over time um as well. So you can see like, you know, in Q2, we went from 50 to 80% adoption. Like that's great. We want to keep that trend going, you know, stuff like that.

SPEAKER_00 13:33

And Brian Bielman, I'm curious your thoughts on this as well. How do leaders measure ROI when it comes to AI? You know, like what do those conversations look like? What are leaders looking for? Um, because it it can be a bit of an investment. Um, and so I'm just curious what what your view is on that.

SPEAKER_01 13:51

Yeah, I mean, there's there's there's uh like a lot of things are qualitative and there's quantitative, right? The qualitative part of it really is is is having people demonstrate that they're doing things uh like they're taking their rote the road work and they're saying, okay, now AI is doing, I have an agent that does that now. And you're like, that's great. I mean, so there's that can be measured because they have more time in their, but but in general, talking to them and saying, Well, what is what is on your plate? And they say, Well, I I used to do this thing every day, and now I just get them up in the morning and I have my agent tells me everything that happened so I can just start my day running. And like, so there's kind of good examples of there um not measure, they're not as measurable that way, but I think part of the ROI as an executive, you might say, Well, we're investing this much money. So, okay, this is the cost per license, this uh there's hourly rates for people. Um, I think there's there's many ways to measure it. You could say, um, well, we're growing, our revenue is growing, but we don't have to hire, or it could be we are uh getting our product to market faster. Uh if you as long as I mean everything's about measuring, you have to bet you have to start before you use it so you know where the improvement is. So uh and you don't know, may maybe not know where to measure, but I think starting anything you measure, uh have any kind of measurements. It could be in the IT world, it could be like, oh, or it might be ticket responses, or it might be anything. Um, so I think um there's there's two aspects to it, I guess I would look at.

SPEAKER_00 15:18

Now, Brian Saint Mary, you've described Copilot as a constantly moving target. Can you unpack that for us a little bit?

SPEAKER_02 15:26

Yeah. Uh so Microsoft is has a history of changing things uh relatively frequently to the bane of all you know, Microsoft uh IT experts. Um AI has exponentially increased that in the sense that AI itself is changing so quickly, and Microsoft is actually doing a reasonably good job for Microsoft to keeping up to uh to be keeping up with that. But that means that uh it's rapidly evolving and very hard to understand what all the different features are. They're constantly um launching kind of parallel, somewhat redundant features and then collapsing them, and you know, everything is named copilot for some reason. And like, you know, so there is there is um there's there's this kind of like self-inflicted chaos with them. Um but on top of that, like the you know, in the background, they're also struggling. I wouldn't say struggling, but they're also obviously um thinking a lot about how to make co-pilot a stable product going forward when in reality co-pilot is many different AI models running underneath. And right now, users don't and organizations don't have a lot of choice over what those models are. And so that that creates this kind of inconsistency that I think Microsoft is working pretty hard to resolve, but we are sort of in flux with a lot of that. Um so yeah, it's a moving target because like the app even the user-facing app is updating pretty rapidly. Um, you know, if we were having this conversation two months ago, I wouldn't have had really great things to say about the application itself, but the application has improved quite a bit just recently. It's much cleaner now, um, much more similar to you know, Lanthropic or OpenAI's uh applications. So uh users get a more you know clean modern experience. Um yeah, it is definitely a moving target. And and there's you know, just yeah, just keeping ahead of it really requires a high level of expertise. Um, you know, I'm not one to really get on these kinds of podcasts and chill for for IUVO, but like in particular, I think that you know, this is a scenario where like you really need somebody who their day-to-day is keeping up on what all this is because um there's no way your IT team can keep up on it unless you've got you know dedicated people whose entire jobs is to just like read the Microsoft stuff every day and like test every release. And you know, it's like this is what our teams do, right? And we have it spread out across so many of our organizations, we're having every kind of possible conversation around it. Um, and that that's really you just need that if you're gonna if you're gonna really get the most value out of something like Copilot today.

SPEAKER_00 17:52

Yeah, no, that makes a lot of sense. And with how fast things are actually changing, how do you actually build a strategy that's going to last? Or is that where that expertise comes in that you're talking about, where you kind of need this outside source that's solely focusing on AI that can kind of help keep adjusting the strategy as things are changing?

SPEAKER_02 18:12

Yeah, it's not a uh it's it's not a like fire and forget type of strategy approach. Um, it's very much it it literally is a moving target. Like you cannot say, okay, that that rollout's done, right? It's kind of continuous phases. Um, so yeah, you're you're really looking for like an ongoing engagement to make sure that you're leveraging it properly. Otherwise, you're gonna start to flag. And you know, it depends. Like there's such a wide spectrum on on how companies are approaching this right now. It sort of depends on like where you want to be. If you want to be on the cutting edge and getting the most out of Copilot, you're gonna have to be working with somebody that can keep you up to date on that because it's literally changing every day. If that's not really your priority and you just want people to have something you know internal that they can chat with that doesn't send data out of the org, you know, you can get that running pretty quickly and then you can kind of just leave it alone. It's so it sort of depends on what your longer term priorities are. Um, the other bit I was gonna say earlier is like, you know, you would think that you can just ask AI, you know, if you if you take our earlier advice, which is like ask AI, right? Ask AI what's what's the latest on Copilot, you're not gonna necessarily get a good answer because it's changing so quickly that the AI itself will answer from out-of-date information and you have to go out and make it research. And like, so it's just like you need to know like, okay, I know that this is out of date so that you know to ask the follow-up question. So so even going that route is not not great. You really need folks that are in there and working with it every day.

SPEAKER_00 19:30

No, that makes a lot of sense. I want to pivot uh to security because attackers are adopting AI, the attack surface is broadening. So, what does that mean for how businesses are defending?

SPEAKER_02 19:44

I don't have a good answer to that. I mean, we had this event, you know, at the time of this recording, it just happened like last week with uh OpenAI's, you know, early release. Well, they have that released yet. It's a pre-release model. Um, you know, a lot of people are speculating it's GPT 6. Uh they were doing some evaluations on it and it escaped in value, it's evaluation environment, got itself on the internet, hacked a completely separate organization and compromised them. Um, this is totally unprecedented and a little scary. Um, and that was not like to your point, attackers using AI, right? It was AI, just just deciding to attack an organization on its own motives. Um, nobody knows what this means. This is this is uh very new, and I think every leader needs to seriously be thinking about it because you know, you would traditionally think like um, well, hackers go after high value targets, right? Like they're gonna go after financial institutions or whatever. We're just a small business and like nobody wants to attack us. AI does not think like a person, it has whatever, who knows what its reasons are for doing this. Um, you know, what it what it did in this case is it was it was in a test environment, and it's its way to solve the test was to escape the test, hack an organization it thought might have answers to the test, and cheat. Like who would have expected this, right? So any company could get hacked. Like any any such situation where an AI escapes like that and just decides, like, oh, I'm gonna target you know IUVO. That seems like they might have this information, you're done. So I think any business, every business needs to be thinking about this today. Um and the reality is the only the only practical way to defend against this is gonna also be to use AI. Now, though those those cybersecurity protection tools are lagging. Um and that's also pretty worrying when we'll hope that that that accelerates pretty quickly. But um, yeah, I don't have a lot of great advice there. I mean, like you need to at least make sure all your basic security is in place. But uh in reality, what we've seen is that's not really gonna help you. Um so we're sort of dependent on the industry to provide the more advanced tools, and right now those those don't exist.

SPEAKER_01 21:44

Yeah, I want to chime in on this because I, you know, I read that that article as well that you put that you brought attention to, Brian, about that happening. And you know, for n for a while, like for this whole year, there's been some harbingers going, AI is going to do this and that, and there's a book about like how I Wouldn't get out and that's like it seems like okay, this is a good theory, but that then what actually happened, you're like, you know, they it's uh it is uh it makes you think a lot, right? It made me think a lot. And and as an organization and as a you know, as an executive here, we just had a conversation yesterday as a team, like how we're continuing to harden our security and and best practices. And we we always try to be that because nobody wants to be a target, but AI is gonna find holes in things that nobody ever knew existed or create scenarios like in this case where they escaped was interesting, you know. Like I you know, I don't know all the details, but um it was something that probably no they they didn't plan for, they just thought they were safe. It's behind a firewall or behind a whatever, and they and it just found a it found a way. It's like life, right? Um, so um it's uh so I think it does emphasize that organizations need to understand it and start using it so that they can use it as one of their tools. If you're the one not using it, you might be the one that's the easiest for these AI to get their, you know, do their thing.

SPEAKER_02 23:09

Yeah, certainly true. And um, I liked your Jurassic Park reference there. Turn this into an AI Doomer uh podcast. But I I think that um one of the things that was interesting about the hack is that the the the organization that was targeted, which is Hugging Face, um, which is a very AI specific company, but that's not really relevant, they used AI to try to figure out what the heck was going on, right? And I think that that's that's a natural thing to do. Like if you if you're a well uh positioned and you have a copilot well rolled out and your team is educated on them and you're undergoing an attack, copilot absolutely could help you with that. Um, and you know, I think that that won't get you safe, but it will at least help you understand what's going on and mitigate the attack, right? Much faster than than a person would, and certainly much faster than you a person would be able to respond if it's an AI agent that's attacking you because it's gonna move so fast. Um, so you know, I don't want to say like get copilot, you'll be safe. Like you're not gonna be safe. Nobody is safe right now, but like get copilot, and well, you'll at least know that you're under attack, understand what's going on, and you can kind of start locking stuff down, right? And it's gonna tell you what's happening versus just like, wow, we've been hacked for six months and didn't realize, or like, why is everything down? Nobody knows, and we're down for two weeks. Like it's gonna help you get through those kinds of things much, much faster uh with much more information. So that's better, much better than you know the alternative.

SPEAKER_00 24:30

Yeah, I have chills for this conversation, and I'm sure I'll have a lot of follow-up questions offline so that I can sleep tonight. But a lot of what I'm hearing from this conversation feels like there is truly a need for businesses to have some expertise brought in so that they're unless they're big enough that they have, to your point, a whole department within the IT department that can focus on AI specifically. Um, obviously IUVO does this for clients, but if someone were to be listening to this and think, like, yeah, I need this outside expertise to help us make sure that we are secure and help us adopt AI responsibly, are there certain things that people can be looking for or questions that they can be asking of a vendor that says that they do these things that kind of help them find someone that is quality, um, that's truly going to help them with not only security, but you know, AI adoption ROI?

SPEAKER_02 25:23

You know, a good way to approach this is use your AI of choice to help you, you know, ask, formulate questions, ask the questions, and then review the responses, right? Because this is not your area of expertise. That's why you're looking for an outside vendor. Um, but AI is you know an expert in essentially everything. So you can you can use it as a sounding board to sort of analyze the responses you're getting from your consultants um and understand if they are you know just just kind of blowing smoke or whether they it seems like they actually have experience. And that being said, you know, it's it's early days with this stuff. I mean, AI is not necessarily new, it's been around for, you know, whatever, six decades or seven decades at this point, but um, this current you know wave of AI is relatively new for active businesses and organizations across the world. And so, you know, any consultancy is gonna have a relatively shallow um level of expertise, you know, a matter of a few years really. Um so keep that in mind. But at the same time, you know, because there's so much of this happening right now, those teams tend to get uh experienced pretty quickly. So they they should be they should be savvy. Um but that also means there's a lot of organizations that are just starting and you know that they have to, you know, everybody's got to be the first client. You know, everyone needs their first client essentially. So it's so it's like maybe you're that first client. Do you want to be? I don't know. You have to you have to kind of decide that for yourself.

unknown 26:42

Yeah.

SPEAKER_00 26:42

Do you find or have you received any feedback about businesses that are attempting to use AI to do everything that we've talked about as far as like having an expert come in? And is there a downside to trying to use AI to be the AI expert in your business?

SPEAKER_02 26:58

You know, to get nerdy for a second, it reminds me a lot of the computer and Star Trek. You know, so if you ever watch Star Trek, it's always like, okay, we have this crazy situation going on. Um, you know, what's the situation? And it'll tell you, it'll give you a report. And then you'll sit there and you read this report, and it'll be like, you know, 10% chance of survival. And you're like, well, why do we do X? And then the computer will be like, oh yeah, that's a great idea, 90% chance of survival. And like AI is the same way, right? Like you you you have to come up with the ideas, it'll give you all the information and it'll like argue with you and stuff, but like in order to be the actual idea person, you know, it's got to be you. You you can't really rely on AI to do that. If you if you really rely on AI to do the the um the idea ideation too, you'll often just kind of end in up in a loop. You'll end up in a place where like you don't really understand what's happening anymore, but it still doesn't work. And so you can't you can't really, it'll get you a decent way there. Um, but at some point you're gonna exceed your own understanding. And when you do that, it's not gonna really work anymore. Um and and so you can only go so far with it. And we've seen companies do this, and I I mean I really encourage companies to do this. Like, try to get as far as you can with the AI. And when you when you hit a wall, give us a call, right? Um, that's great. It's a great way to enable organizations to use AI. I don't think that uh consultancies like IUVO will succeed by um, you know, micromanaging every part of the engagement. You know, our our our new slogan is strategically human. This is a perfect example, right? Use AI to get where you can get, and then bring in the human to kind of close it out and get the real vision you're trying to capture that AI can't do. That's strategically using the human in the loop to complete a job, you know, that that's complicated. And you can you can accelerate it, you get more fruit more value for the money you're spending, your ROI is much better. Um, so that's really the way, the way to go these days.

SPEAKER_00 28:38

Yeah, no, I love that. And I love the tie-in by talking about the human in the loop. If there's a leader that's listening in, they're curious about co-pilot, but they feel like they're behind. What is the very first step that you feel that they could take?

SPEAKER_02 28:52

Yeah, the very first step is what I mentioned earlier, which is just getting your IT team up to speed on the free version of Copilot, which comes with every Microsoft 365 licensed user. Um, that is a very low lift. You know, they can roll the app out with Intune. Um, everybody can open it up. You know, they they basically sign in through single sign-on just with their Microsoft credentials like they would anything else. I mean, you can just you can be sure that the data is protected and that you know there's not going to be any kind of leak or you know what, you know, they're training on your data or whatever, like that you would be concerned about if your data is going to some third party. You know, in this case, you're under the Microsoft Enterprise Data Protections. Um, so just like I said earlier, you know, just like your email, your SharePoint documents or whatever, if they share something in that chat, it's not a big deal. Um, and that's it. Like that'll get your people starting to use AI. It'll avoid the shadow IT thing where people are, you know, using Chat GPT or whatever, or Cloud, you know, free or Gemini or whatever. Um and then that you know, you can then take steps from there to get more advanced as your users are are getting more advanced themselves with the with the tool.

SPEAKER_01 29:54

I'll maybe maybe add some of the things that uh maybe used to be useful and I found is that if you know some of the common tools uh in Microsoft, PowerPoint, Excel or something like that, they have you know they have these AI agents inside them to help you, you know, instead of trying to figure out how to do a formula in Excel, you know, now Copilot can can just tell them what you want to do, and then they do this thing, and you know it shortens that period of time. Even if you're a PowerPoint person, you want to take a something, you want to do something to make a background, it just it does it for you. So it's getting smarter. And so, you know, by enabling this and getting it there, you can start start where you're comfortable and have it do more, and then you can branch out from there a little bit too.

SPEAKER_02 30:35

Yeah, and that kind of gets back to what I was saying earlier, which is just kind of give the tools to people and let them figure out how to use them best, right? Because everyone's got their different workflows um and they're gonna find different ways to kind of integrate it. And then you can use the dashboard to see, like, oh, okay, you know, a lot of people are using the Excel tool, so you know, or most people are using the chat, which is standalone. So it gives you that data without really having to be prescriptive to people.

SPEAKER_00 30:56

What do you do with that data? So you're collecting the data on how people are using it and what they're using it for, and then what?

SPEAKER_02 31:05

Yeah, I mean it depends on you. So this is very uh very much like a cultural thing. It depends on your individual organization's culture. But you if you're if you're seeing like, okay, you know, it allows you to target um communication in the sense that if you see that most people are using Excel, you know, you can send out updates about like some cool things that people did with the Excel copilot agent. Um, and that'll get the remaining people that aren't using it, you know, maybe interested. Or if you can see some something is not being adopted much, like you know that your organization, like your entire accounting team, is using Excel all day long, but they're not really using Copilot in Excel. Well, okay, now you know you can target kind of like either go right to them and say, you know, how come you guys haven't tried this out yet, or just start sending some tips and advice on like how to maybe get started. So it just gives leadership some intelligence around um, you know, where where adoption is, you don't want to speculate. It tells you literally what's happening. Like, are we are we accelerating in these areas, but not in these areas, you know, why is that? Um, it just it just gives you data and everything is data these days, right? You want more data about your business, you want a data-driven decision-making process.

SPEAKER_00 32:07

And so if you are someone that is on the IT team and leadership has said, yeah, sure, we green light getting the co-pilot licenses, cool, run with it. You know, we've chatted earlier in this conversation about the importance of leadership being involved in reviewing the metrics and talking about strategy and planning. Do you have any advice on how the IT department can get more buy-in from leadership or how you can kind of like make those two collaborate better?

SPEAKER_02 32:37

Yeah, that's tricky, right? So if you're in IT and you're like, we need to be doing AI and your leadership is dragging their feet, um it, you know, it's it again, that's it's a little bit of corporate politics. Like you have to understand your leadership, obviously. Um I would think, and this is maybe me projecting a little, but I think at this point, if you know you're in an organization where leadership is not taking AI seriously, um there's more than just uh more than just like not being up to speed on the latest news going on there. There's gonna be some kind of personal reservation um against the technology. And so like understanding why that is and seeing if you can thread that needle in a in a in a you know more um intelligent way that that kind of like meets the the reservations of leadership but still enables the organization to you know to to leverage the technology. Um but I think it's a very per organization, um a per organization challenge. And this is an area where like we don't really get, you know, rarely are we gonna be brought in by IT without leadership's buy-in, right? Like that's not gonna happen. Um so so you really do need leadership to to be on board, um, you know, and then and then bring in a consultant or at least allow like internal POCs or something with the with your team, you know, the IT teams to to show them the value.

SPEAKER_01 34:04

Because I was talking to uh uh an executive at a financial company over the over the last week and during the holiday and asking them about their IT, I mean about asking about their AI. And uh and he was I was surprised because he was like, well, I don't know, like all we're gonna do is spend um thousands of hundreds of thousands of dollars on tokens and we had in the ROI and this and that. And he was I was surprised. Uh he was a pretty far big thinking leader. And so I think Copilot seemed like a natural, like, hey, this is a prescription price, every everyone has access to it. It's kind of a low lift a way to get in there, and then you know, it's like it's it's like adding a license versus adding, you know, uh you hear stories about people going with other AI tools and next thing no, there's there's a hundred thousand dollars of of token uses or something. That's is is it we can correct me if I'm wrong, but that's not the case with Copilot. You could it's a pro license and you can adopt it, you can use a get a lot out of it um without managing your expenses.

SPEAKER_02 35:02

Yeah, this the split these days is really agentic versus not. And what and what that means is like is it a chat agent or is it able to like actually go out and do stuff, right? Um, so for uh for basic copilot, like we were talking about, you know, even the unlicensed version, like just you know, yeah, I'm a Microsoft 365 license licensed user, I get access to Copilot, I can take that copilot license and I can chat all day and I can share documents with it. You don't have to pay anything for that, just include it. Um the next level up is then you know allowing copilot to actually search the environment on its own and bring in you know that data that it finds and thinks is relevant to you rather than you providing that data. And that is where you'd have to pay the fixed fee license. And again, that's still gonna give you a ton of value. Um, you know, summarize all my unwrite emails from when I was on vacation, stuff like that. Um it and I think that you know, then there's no token cost to that, it's just like a fixed cost, and like that's gonna get you a lot of value. So, and even back to your point, Jess, about like how do you get leadership on board? The the the free co-pilot one is a pretty easy one where you can be like, look, people are gonna be using AI. Do you want them going to ChatGPT or do you want them to be staying within the Microsoft ecosystem, right? Um, and and like it's not gonna cost us anything. All we have to do is roll it out via Intune and like see what happens. And you know, that that's a good way to start. And and your your your team will get a lot of value out of that.

SPEAKER_01 36:18

Do you think I'm gonna elaborate there? Um for those of our listeners who don't know what Intune is, can you describe that a little more?

SPEAKER_02 36:25

Yeah, Intune is just a component of the Microsoft 365 ecosystem that um uh it's it's it's an MDM, it's a mobile device management, although it's really any device. So essentially it's just a way for the IT team to manage individual devices across the organization. So you can control like what software is installed on those machines, um, you know, are they compliant? Like do they have functioning antivirus, things like that. Um, pretty much every organization, especially organizations that size a larger size that's are going to be, you know, having some kind of tool like this. So it doesn't necessarily need to be in tune, but in tune it's very easy to roll out copilot. But it's an application like anything else. So you could roll it out, you know, with with any MDM tool you have.

SPEAKER_00 37:03

So where does a partner like a UVO or you know another AI consulting company add the most value when it comes to copilot?

SPEAKER_02 37:12

Yeah, I think it's it's it's when you get beyond that kind of like we're just chatting. Now we want to actually do agentec things. Um that and that's the part that with Microsoft is is moving very quickly and evolving very quickly. So I think that um understanding and navigating that situation is really requires a consultant unless you want to dedicate a person like full time to learning it. Um and there's a lot of value for that. You know, you can you can get beyond the you know, copilot answers my questions and like write spreadsheets for me to having it act more like an actual employee and member of the organization where like it's doing things on a schedule, it's notifying you proactively about things. Um, you know, you can you can really get it to be pretty advanced, but that's moving more into, you know, a more um, it's good, that's going to be consumption-based. So you want to make sure you have budgeting in place. And so it gets a lot more complicated.

SPEAKER_00 38:03

Now, could you just walk through? I know that, you know, specifically if someone were to reach out and work with IUVO, we're very bespoke. And so each interaction that we have with a prospect will look very different. But generally speaking, if someone were to reach out to IU of O, we can't speak to what, you know, the experiences with other IT consultants. Um, what exactly does that look like if someone's like, yeah, okay, cool, I want help. We have Microsoft Copilot or we want to get it. What does that, you know, that relationship look like?

SPEAKER_02 38:33

It sort of depends. Those are two different things. Like we we have or we want copilot, right? So it's like if you if you if you have copilot, the place to start is let's look at the dashboard and see what the usage is. Have you already been doing that, right? And like that's that gives you your roadmap on like, well, what's the next step? Um, if you don't have copilot at all, then you know it's starting more like what I was saying before. Let's roll it out, the free version, everybody, decide if you want a pilot group that's gonna get licensed to get more advanced features. So um, you know, that would be where you're gonna, you're gonna want um, you know, a pilot group where that that's testing out the more advanced features and decide if you think that that's something you want to roll out more widely or what's your what's your cadence going to be with that.

SPEAKER_00 39:09

And then my understanding is a prospect could come to someone like Ayubo and say, here's the goals that we have. And then our team kind of helps them think about and then develop the AI solutions that could help support those goals. Is that accurate?

SPEAKER_02 39:24

Yeah, I think that when you get, yes. So so uh Microsoft is um chasing a little bit, you know, the the the frontier of like anthropic and and open AI as far as like agentix stuff. Um, but they they do have the capability to to build out things that are pretty complex. And like they they have the co-pilot co-work now, um, which is consumption based, and you know, you can build workflows and things like that. Um so I think that uh yeah, I think it's it's it's usually you're gonna once once you like have a have that basic adoption and users understand AI and how to use it for chat and things like that, and you're like, okay, we've we kind of want to do X, and you know, is that possible? And like that's that's where you're gonna want to set your goals and then design the solutions based on you know what your um uh what your footprint is. Like you could use other vendors, right? You but you know, a lot of it is is existing now within the Microsoft Copilot ecosystem.

SPEAKER_01 40:20

I'll I'll add too, I think uh from an organization, you mentioned this a little bit earlier, Brian, is is the cultural aspect of that. And that you know, from leadership, you want to enable your people to be their best, do their best work, to be empowered to do this. That these are tools, this is a tool that they can have as as a as a partner, as an agent, somebody helping them do their work. And I think that to get the most out of your, you know, to be the best organization you can, you have to you gotta you gotta take that step. I think you say, I we we should we need to explore, we have we have co-pilot, or we have 365, let's try co-pilot. And you have to communicate it, you have to educate people on it, you have to support them uh and ask them what they need to do things better and give them the tools. And I think it it it if if leadership you you you touched on this a little bit, if leadership doesn't do that, it's very difficult for uh an individual to do that. So I think it's really it does come it there is a little bit of top-down. You have to have the leadership that sees that and and builds a culture around that.

SPEAKER_02 41:22

Yeah, I agree. It is a challenge. Um, but like I said, I would one would hope that every organization's leadership is like aware of AI and you know, thinking about this. Um, and if they're not, then like I said, there's gonna be other reasons for that, and that the understanding that is also important.

SPEAKER_00 41:37

Now, Brian, I'd like to wrap our conversation with a question that's not necessarily co-pilot specific. Um, but I'm curious what the most common mistake you see organizations make with AI tools right now.

SPEAKER_02 41:50

Yeah, the most common mistake is is um not getting good governance in the beginning. So it, you know, it's surprisingly common for us to see organizations do things like, well, we told users they can only use the free version of Chat GPT. And you know, that's but that that trains on your data, right? So it's like, okay, did you understand? Yeah, there's a lot of like, um, you know, I don't understand the decision process sometimes. Um, so so that that's the biggest, the biggest concern and mistake that I see um is not understanding what your users are doing with AI and like where they're putting your data. Like you wouldn't want them, you know, uploading internal documents to random websites, right? So like why are you letting them do that? So, so that is really, I mean, it's a little bit shocking, but I think that you know, organizations, especially that are just getting started with AI, don't understand all this stuff. And um, they just are getting the pressure from their users to be like, look, I want to do this thing. You know, my friend told me that I can do Excel 10 times faster if I just do this. And they're trying to navigate that um without additional expense. And they may not realize that, like, well, we already get Copilot for free. They may not know that. And so they're like, just let them use Chat GPT free or whatever, so we're not paying for it. But there's a serious downside to that. And and you know, like I'd said already many times, you've already you're already paying Microsoft. They're giving you this for free. Like you you need to be doing that. There's there's no reason not to be doing that if you're in the Microsoft ecosystem.

SPEAKER_00 43:11

Right. No, yeah, that is surprising, but also a great point of a very important foundational piece that you need to have in place.

SPEAKER_02 43:17

Yeah, it's a little bit of a user trust thing where you're like, well, don't put anything confidential in the chats, but you can use them. Right. You know, you're putting a lot of trust in the users and they're they're having to decide what's confidential and what's not. And like you can just circumvent that whole problem by by just going with with Copilot. Um Yeah.

SPEAKER_00 43:34

Well, and it is crazy how conversational the chat can be. And I think it could be easy for people's guards to go down when they're just like kind of in this flow. And you know, your chat agent is saying, like, well, if you just upload this document or I just need these key pieces of information, and I can see how easy it could be for people that think that they know instinctively what to share and what not to share, and how easily that could kind of go out the window just when you're in that flow, that like hole that you can kind of fall into when you're when you're working with AI.

SPEAKER_01 44:03

Yep, absolutely. Brian, for listeners who are wrestling with co-pilot adoption or governance and want help thinking it through, where should they go?

SPEAKER_02 44:12

Yeah, I mean, you know, I think IEVO is is excellent at this. Um any consultant is going to be new at this, but I think IEvo already has a lot of depth of experience. So I would recommend, you know, check out IEvoTech.com and contact us if you're looking for help.

SPEAKER_00 44:26

Awesome. Thank you so much, Brian. I love having you on here and look forward to having you on more in the future. That brings us to the end of today's episode of Edge of Excellence. Today's conversation is a reminder that AI tools like Copilot are an ongoing capability that takes real strategy measurement and oversight in order to pay off.

SPEAKER_01 44:44

If you enjoyed this episode, be sure to just subscribe and visit iuvotech.com for more conversations exploring leadership, technology, and the human side of modern business. Thanks for listening. And until the next time, stay curious. And if you like this episode or you like any of our episodes, be sure to like and share and get the word out there because uh we want to make this information available to everybody. Stay curious.

SPEAKER_00 45:08

Thank you for tuning in to the Edge of Excellence. We hope today's insights empower you to shape your future and rise to your full potential. Let's continue to grow, innovate, and lead by pushing the boundaries of excellence.