SPEAKER_03

An AI agent isn't going to take your job, but someone who knows how to use AI better than you may take your job.

SPEAKER_01

Somebody listening to this podcast just bought AI licenses for the whole company. What would you tell them to make sure that they're getting the most impact the quickest from those hundred plus licenses or whatever it is that they just bought for the company?

SPEAKER_03

So you've got to train people on how to use it with the policy, you've got to roll that out, you've got to give them some basic information about prompting and about the aspect of hallucination and how to avoid it and how to overcome that.

SPEAKER_01

Despite all of the talk about it, has hallucination been an issue for you guys?

SPEAKER_03

No, it's really not. You have to verify the data, you ask for references and links, you go confirm it. The value add far exceeds the risk of some bad information. AI is not going to help you solve something that humans don't already know how to do effectively.

SPEAKER_00

Darren Ward is an executive leader and chief strategic innovation officer who helps organizations apply AI to real business problems, improve productivity, and build a lasting competitive advantage.

SPEAKER_01

Welcome to Using AI at Work. I'm your host, Chris Daigle. Each week we'll be learning how today's business owners, entrepreneurs, and ambitious professionals are getting more done with smart use of tomorrow's tech. Let's get started. Right now, every business leader is asking the same question. What are we going to do about AI? If this is you, ChiefAIOfficer.com has the answer. We give you a simple path forward where we provide executive and team training so your people know exactly how to safely use generative AI in their day-to-day. We also manage the deployment and implementation to make sure tools actually get adopted and deliver results. And we'll also guide company-wide transformation so AI becomes part of your operating system, not just another shiny object. The companies that act now will increase productivity, cut costs, and grow faster than their competitors. Those that wait will get left behind. So if you want to make AI work in your business, visit chiefaiofficer.com and see how we're helping companies of all sizes finally get results from AI. Ladies and gentlemen, welcome to another episode of Using AI at Work. My name is Chris Daigle, and I'm the host of the podcast. And today I have a friend and client, Darren Ward, who will be joining us. And this is an episode that we don't normally get to do something like this. And when I say like this, what I mean is that Darren is an executive. We met at a Vistage Executive Summit about a year ago. And his company was, it's in the manufacturing space. His company was already experimenting for sure. They were uh, you know, he was mentioning some things, and I was like, okay, wait a minute. This company's a little more advanced than the rest of the companies coming up to me. And it was um, and that kind of led into us exploring like, well, how could we support the continued growth of your organization when it comes to AI? Uh, we engaged, and now we've been working with them for about a year. But the transformation that I've seen has been pretty incredible. And this is something that if you're listening to this and you're an executive or you're a decision maker at your company, I really want you to pay attention to the path that Darren has been on. He's gone from being, you know, uh the president at the organization, really like the in the like in the operations, making sure that things were happening, evaluating AI, how it could impact all their hundreds of employees, to now becoming the person who's really the chief AI officer at their organization. Now, is that the path for everybody? No. However, nobody knows your company better than you do. So as you hear Darren talk about his journey today, I want you to ask yourself should you be the person leading AI in your organization? And maybe the answer is yes. So, Darren, before we jump into kind of like um what that's been like over the past year plus of your organization's uh journey on AI, anything in particular you think that might be relevant for the audience to really understand like why you are qualified to um give an executive's perspective on AI's impact on business?

SPEAKER_03

Sure. Yeah, absolutely, Chris. Yeah, I I it really goes back to the beginning. I got uh degrees in engineering and I started my career in management consulting and initially did some consulting in IT strategy work. Uh after consulting, I ended up working for an enterprise software company for about nine years in various roles product management, implementation consulting, uh, and sales consulting. And from there, I transitioned into middle management in mid-market companies. My first job outside of the software industry was actually the vice president of IT for a small mid-market distribution company, uh private equity-owned. And from there, I began my journey in middle management, working up to senior executive management, ultimately became the CEO of a distribution company in Houston, where I met my current boss in Vistage over 12 years ago. So I've been a Vistage uh member for over 12 years. And so I was working private equity. Her parents had recently transferred the company to her as a third-generation family-owned business, and she asked me to get out of private equity and come help help her build that company. So uh for over 10 years, I was working uh building this privately owned manufacturing company, and we were very interested in keeping the company at the forefront of AI. So when I met you, I was still actively in the president role, but I was basically spearheading the initiative that was sponsored by our owners. And um, I was able to beginning to transition. So we hired a replacement as the president of the company in January, and I live out of town now, work mostly remote as I'm uh supporting my my parents, and I'm continuing in the company as our uh full-time chief AI officer. So that's a a quick recap of the background how I got here.

SPEAKER_01

So so Darren, when we met, was was your plan to like retire?

SPEAKER_03

Uh my plan was to slowly retire. I was I I I'm uh too active to just disconnect altogether, but I have to spend a lot more time now with my parents. So I had always planned to potentially do some consulting on the side. And when I was asked to stay engaged at that company, that was a perfect win-win, right?

SPEAKER_01

So that ongoing engagement, though, was specifically a reflection of the work you were doing in AI for the organization. Is that right?

SPEAKER_03

Correct.

SPEAKER_01

Okay. So what is the official title? Uh, do you do you hold the role of chief AI officer, or is there an official title that you've got?

SPEAKER_03

Yeah, my my official title is chief strategic innovation officer.

SPEAKER_01

Okay. So but a big part of what you do is is AI.

SPEAKER_03

Absolutely. Yes. I mean, I still support the company in terms of looking at uh strategic projects and things like that as an advisor, as an executive advisor, but the core of my day-to-day activities are really driving AI transformation at the company and continuing to stay on top of that.

SPEAKER_01

So here's what I can tell, dear listener. Um, if if this is something, if if an AI career is a pursuit that you want, right? You've you've kind of found like I really enjoy this. I think I can enjoy my career more if I was more focused on AI. I would tell you to, if you can, follow the path that Darren did, go from whatever role you're in, embrace AI, start to introduce it into the organization, and stick around your own company, but more in that AI role. And I could tell you, there not that you didn't have fun before, but are you having a lot of fun as the chief AI officer?

SPEAKER_03

Absolutely. Yes. Yeah. I I can I can probably provide a lot more focus in an area that I'm really passionate about, which is technology applied to real business problems.

SPEAKER_01

So let's talk about that focus. Would would you have been a because because you kind of were, you were president of the organization and you were really the person that was the champion of AI internally, right? Correct. Um could you do could you do both equally effectively?

SPEAKER_03

Uh it's challenging, right? But I think every company is gonna need you definitely need an executive sponsor to drive this initiative. You can't just hand it off to IT. So from that perspective, the owner of the company and myself both agreed that we needed if it if it wasn't me staying at the company, there would be another senior executive with the leadership team that would be assigned to oversee and drive this.

SPEAKER_01

So my advice to companies is that AI is too important for it to be somebody's side job, right? Because they're gonna get distracted with their primary stuff and go, oh, I'll get to the AI stuff later. That's right. And later never comes, right? Um, and you mentioned something that I also am in total agreement with. It's not something that that IT needs to lead. Now, now give me your perspective on on why you hold that position.

SPEAKER_03

Yes. For AI to be effective, it's got to be applied to real business problems. And we we've got a fantastic IT team, but without understanding what they're building the technology for, they don't really have the desire the or the necessary context to know where it best applies. So when we can go to them with specific problems we want to solve, and then they actually get really excited about it because they want to be solving real problems. They don't want to be building toys, right, that don't get used. So it's it's really uh rewarding when I can bring to them a, hey, this thing is really great, but you know what would really make it outstanding is if we could add this and this bell and whistle, and that would help us accelerate this process. And they just light up because they love doing that, but they just don't know where to begin.

SPEAKER_01

Yeah, I I and I totally agree with that perspective. So for the listener, um, we kind of see two paths of AI. We see the analytical side, which is the machine learning and the data science, and the the PhD and you know, from MIT kind of person. But that's not what most companies need, in my opinion. And I think Darren is a perfect example of that. Although we had a technology background, it wasn't necessarily being the lead technologist, it was the business person who understood technology, right?

unknown

Correct.

SPEAKER_01

So the path that we see on AI, we call that uh applied or operational AI. It's introducing generative AI into the operations of the organization. IT doesn't quite understand how HR or sales or the guys on the factory floor are doing their thing. The president of the organization, somebody with an operational mindset, definitely does. And I think that that's why Darren was such a perfect fit. Um, here's one. What mistakes did you make early on that you think that other leaders and and not uh everybody's gonna make mistakes, and especially when it's something brand new and you don't have a uh a playbook to follow, right? What would you tell the person who's gonna maybe follow in your path and their organization? What would you tell them to so that they don't um make those same mistakes?

SPEAKER_03

Absolutely. When we started out, our initial thought was let's just get basic capabilities for how to run a chat bot out to as many people as possible. And so we were looking for training, we just try to invite uh a very broad audience, and then we started trying to push it out with uh you know regular weekly AI calls, and we set time that, hey, everybody should come on here and try to do something with AI. What we found is a push model isn't near as effective as a pull model. So we had a handful of really engaged power users, and then we had a bunch of people that just thought this was a distraction from their day-to-day job. And so what I would say now is identify the small group of power users and champions that are interested in it, that uh can get engaged and train them to be really effective. And when they find uh business problems in their departments, in our case, we've got power users in engineering, in sales, in marketing, in accounting, then other people begin to see, oh wow, that's actually really cool. How are you doing that? And then it begins to grow some traction, and people then come and start asking, well, hey, how do I learn how to do that? Nice. And so the other thing we realized is you can't do one size fits all training. So we used to try to have, we began doing quarterly sessions, training sessions that were all day. And we would have some kind of beginner training for the beginners and then some advanced training for the power users. But then the beginners get lost when you start going into more detail and you're talking about multiple things, and the power users get bored hearing what they already know. So the model we've transitioned to is we're gonna have quarterly full-day workshops that are focused on just the power users and let them go deep and help resolve issues they're encountering and learning new things and learning new workflows and capabilities. And then we started doing uh twice a month a fundamentals of AI session that are recorded sessions, and that's and then we're inviting everybody to come participate in that, and then we can address more basic topics, we can answer their questions. The power users don't even need to come on that if they don't want to. But what I'm finding is actually some of them are sitting in listening in anyway, and then uh, you know, just because they're interested in more formal structured training, but uh I'm so far that's seems to be a lot more effective.

SPEAKER_01

So the people who are attending the fundamentals classes, I know you said you've got some of the power users that still check it out. Are those people that maybe they've had a license but they uh from the company, but they haven't necessarily been doing much with it? Are they new employees? Who's who's attending that?

SPEAKER_03

Yes, uh a little of both. So we've got new employees that have come in and they've heard about this stuff, but they haven't had an opportunity to go to any training. So we had an example of that uh this last you know, two weeks ago. We had some new users and then they were immediately excited to it's like, oh wow, I never knew this. Uh and then similarly, like our engineering department, we rolled out Chat GPT license as the engineers, but there was only two or three that actually knew how to use it well, and the rest were confused, right? I mean, not that they weren't, they just didn't use it, right? So it was kind of a waste of a resource. So for them, it's like and what I realized was in talking with them, some of them had misconceptions around AI because they hadn't been trained. Yep. And so when I said, you know, that's that's interesting, but did you realize XYZ? And I'd have a conversation with them, and and I realized, well, you you ought to come to this fundamentals training. And then they got excited about it when they realized, oh yeah, that's actually could be useful. And now they're starting to talk to their friends and say, hey, did you realize AI could help us do this and that? And then they're coming into the training. So that pull model seems to be a lot more effective.

SPEAKER_01

One of the things that I I talk to executives, uh, mid-market, it's maybe not so much the case. Um, but even so, they're they're concerned about well, what about the skeptics? What about the those who resist it? What about those who are concerned about it taking their job? What kinds of resistance did you see in the company when you started it to beat the drum on AI? And and I'm not just talking about employee level, also at the executive level, was there resistance?

SPEAKER_03

Um, we have had it's been a mixed bag. Some of the executives are fully supportive and engaged and actually using it themselves, power users themselves. Uh, and others uh just see AI as a tool. To them, it's nothing different than a spreadsheet or um email, right? And so they question the you know ROI and the investment and the emphasis. So but there's enough traction in the company realizing that this is a strategic advantage in the long run that you know they're okay as long as it's not the the executives that are doubting whether uh this is an effective investment, uh they're okay with that as long as they don't see it as a uh a drag on what they're trying to do. So we've had to be careful around not making this mandatory. This is a this is we're looking for opportunities where this can help the business and not just be a distraction and looking for those areas where uh we can actually make people's jobs easier. I I honestly, we haven't really heard any concern at all from the employee levels around losing jobs. Yeah. The company is growing so fast that everyone has way too much work to do. So anything nice, anything that they can do to save time and get more done and go home earlier, uh, they're embracing. So that hadn't been a challenge for us.

SPEAKER_01

So has the ability for the individuals who were trained in using AI them to be able to do more, has that allowed for some of that growth to be less painful?

SPEAKER_03

I yes, I think it I think it definitely will. I I hesitate just because we're still a little bit early. I mean, right now we're uh we're saving time on specific processes. I can walk through some of those, some of those examples. Yeah. I think the in the long run, it will enable us, enable the company to continue to grow uh without as much strain, if you will. Sure. We're able to automate some of the mundane tasks and keep humans focused on things that are much more value added.

SPEAKER_01

So for the listeners, I want you to pay attention. And I don't know if you caught that, but Darren said that we're still new, it's still early. They've been doing this for a year and a half. If your company has not gotten started yet, um, and this is somebody like who's been you know doing it for a year and a half and still considers that that they're early in the gains that are going to happen, the the transformation of the culture and everything like that. So uh like today is the day to get started for sure.

SPEAKER_03

That's a great point, Chris. Yeah, that's that's a great point. But it's it's true, right? Oh, absolutely. Because you know, you hear all the the talk in the news about agents and uh jobs getting replaced, and the reality is it's not easy um to actually get measurable wins, right? You've got to be very focused on identifying very specific problems. It's gotta be a process and a problem that's already well defined because AI is not gonna help you solve something that humans don't already know how to do effectively. Um so yeah, it's it's it's been a journey.

SPEAKER_01

You know, it's it's interesting you you mentioned like you know in your gut that if you've got people using the tools, there's an ROI. But the measurement of that ROI is a bit elusive, and I'll give you a perfect example. So Aton, who teaches our you know, uh AI developer stuff, he just got back from this amazing uh event in Napa Valley with operating partners from private equity. He said there was a trillion dollars in assets under management represented in that room. He said he ran into one other person, so that means there was two people in that room that knew much about AI, at his level at least, right? And he said, but everybody was talking about it. But then the number one thing that was consistent across all conversations is how are you measuring ROI?

SPEAKER_03

Correct.

SPEAKER_01

So I I don't know, like, I mean, have you cracked the code on that?

SPEAKER_03

Honestly, Chris, it it is a challenge. We are measuring time savings where we can.

SPEAKER_01

So what we're doing is we we do think how much of the time savings do you think you're actually capturing?

SPEAKER_03

I think we're capturing a minuscule amount.

SPEAKER_01

I think and you guys are actively trying to.

SPEAKER_03

We are actively trying to capture the time savings in multiple formats. So we did a survey of our AI users and we asked them how much time you think you're saving each day. Uh and then we're also we have an AI council that meets weekly where we're documenting wins or trying to document them. And then when we get departments that have developed a tool or a process that's you know a consistent Specific thing that they're solving with AI, we're getting estimates from them as to how much time we they think they're saving. But I know that there's a lot more that people are doing. I mean, personally, I don't know how I would get by without AI. I use it every day, multiple times a day, all day long. Right. How do I quantify that? Yeah. Now, you know, the skeptic on our executive team would say, well, okay, Darren, where is that hitting my PL? So you can count hours.

unknown

Yeah.

SPEAKER_03

You can, yeah, you can sum up hours that people are saying they save, and you can apply a labor rate to that and calculate a dollar figure. But how do you measure that on the PL? And the answer is I we can't directly. Now, when we get to a point that we are beginning to grow and we're realizing that our accounting department, for example, we can double revenue without doubling headcount to process invoices and accounts payable. That's a metric, but that that's kind of slow, that's slow go over time, unless you're a really large company. Uh, the other thing we're doing is trying to identify uh business process metrics. For example, engineering prints created or new orders processed per FTE. So we're trying to document a baseline and say, okay, in 2025, our sales order processing department had X number of people and we processed Y number of orders. Yeah. Therefore, it's roughly, you know, X number of orders per person per day. And now that we've implemented, you know, five new AI-assisted processes, what's our productivity for that department? Can we go from you know 20 orders a person to 30 orders a person or 25 orders? So same thing on engineering, engineering prints per per drafter created. Yeah, right. So those are the types of things we're trying to look at. So we can quantify that, but it's challenging.

SPEAKER_01

So, listener, if you're having trouble with this as well, you're not alone. We know that there's more is happening in the company at probably a higher quality than without it. Um, we know that there are things that were sitting on the whiteboard that we never had time to get to that were now starting to cross off because AI is opening up bandwidth. But how do I turn that into dollars and cents and and give it to somebody who is very focused on the PL? I don't think there's an answer for that. Like, and and this is like seems like that would be the number one thing that everybody would be trying to measure. But I don't know that the uh I think that on the adoption curve of where people are, they're just they're just happy to have like have people using it now. Then they're gonna figure out the ROI. Now you you've mentioned engineering a number of times, your manufacturing company, that's an important element to the there was, you know, we talked about the the resistance or uh I asked about resistance earlier on. And there's a specific use case or or case of that from the engineering team that we've talked about with Ross. And I'd love for you to share with the listeners, like what they can expect, even from the skeptics, um, what that looks like.

SPEAKER_03

Yes, yeah, that's a great point. So uh early on, you would think engineering would be one of the early adopters. I mean, they're typically technical, uh, that's a great use case because you can use AI to do research and analysis and reach into your databases for and documentation, but our uh head engineer was a skeptic. And his concern was well, it hallucinates, I can't trust it. Engineering is precise, I'm accountable for my results and for my math, so I don't trust it. And um, our owner and CEO basically had to have a heart-to-heart with it and say, you better get on board with this and figure it out, because this is where we're going as a company, and this is uh key to our strategy. And he did. He began to research it and read it and understand how to address the FUD of quote hallucination and how do you manage that effectively to generate value? How do you put guardrails on that? How do you implement human in the loop reviews? And he quickly went from being a skeptic to being one of our power users and building custom GPTs and rolling it out to his department uh and to the sales engineers to help streamline information before it can't come in, came into product engineering. So, yes, that that's a great uh great case there, Chris. Thanks for reminding me of that.

SPEAKER_01

Yeah, so and you brought up something else. So one heart to heart between an executive and a team lead is one thing. But how does leadership, how would you suggest they communicate that AI is no longer optional across the whole company?

SPEAKER_03

Yes, there's there's multiple ways. Um it could be as simple as, you know, some some leaders send out periodic, you know, note from the CEO or note from the executive team, emails out to the company uh talking about the strategy. Uh it could be in all hands meetings, highlighting that as examples of things we're doing and what different departments are doing and how they're uh using this or saving time. Um, I've I've read about companies, we actually considered this, but we decided a little bit premature, but you know, some software companies or uh companies that are primarily uh white-collar actually make it mandatory for their uh performance reviews. So that actually becomes part of their performance reviews, is what they've done with AI. Now we're a manufacturing company, so the majority of our employees are actually on the technicians on the line, so it's not as applicable for them. But um the way we have driven that is tying performance objectives for for employees to business results, but then showing them where using AI can help them achieve those results.

SPEAKER_02

Yeah.

SPEAKER_03

So that's the link that we have done. So all of our pilots, we have a strategic playbook for the company, which are the business objectives that we are trying to achieve. And those are drilled down to individual metrics for every employee. We've done it for every salaried employee, and just clarify. But every salaried employee, the company has what we call individual performance objectives, which are tied to these strategic objectives.

SPEAKER_02

Yes.

SPEAKER_03

And they have their performance evaluation at the end of the year is tied to how well they met those metrics. So when we are screening pilots for AI, we're literally linking those to one of those strategic objectives and saying, okay, how is this going to help you achieve those objectives? And when they can do that, and then they begin to tell other people, oh, hey, you know how I'm meeting my metric? They built this custom GPT for me. Yeah. Um, a great example I I like to use are uh one of our safety advisors, he has one of his objectives is to roll out a toolbox topic every week, right? And we often tie it to incidents that have occurred. You know, if someone cuts their hand, okay, let's do a toolbox topic on gloves and hand safety or whatever. And it used to take him a couple hours a week to go research the topic, look at the official websites, find videos that supported it, create it in our LNS system. And uh, we helped him build a GPT that understands the exact formatting he needs to use, what sources to go reference, uh, where to go find the YouTube videos. And he said it cut it from a couple hours a week to 30 minutes.

SPEAKER_02

Yes.

SPEAKER_03

He just says, Hey, I need a toolbox topic on this, and it spits it all out for him. He just copies and pastes into the LNS and he's good to go.

SPEAKER_01

You know, and I think that so my my my guess is that that wasn't the highlight of his day, having to spend the time. He's got a role that he likes, and now he's got to go do this research and put this stuff together. And in every role, there's something happening like that where I got to do this thing, it's important, but I really hate it. And here comes AI to save the day. And now I would imagine from that one experience, he's probably like, hey, this AI stuff, it's pretty good, right?

SPEAKER_03

Exactly.

SPEAKER_01

Yes. Yeah. So one of the things, and I know that that you've just recently done a very deep dive into what has been the impact of AI in the business. Like you had a presentation to the executive leadership team on specific metrics, and part of that was surveying usage. Yes. What did you learn from that survey that that maybe surprised you both good and bad?

SPEAKER_03

So let me caveat this. We only sent surveys to users that had a GPT license. So we sent out that survey to everybody at the time that had a GPT license. Even with that, I you know, you can't really track with a GPT team account, you can't track individual usage. We had no idea whether using the license or not. Uh, so I was pleasantly surprised in how many of them were actually using the tool and reported savings. Uh, the other thing that was surprising, you you always hear a lot about quote, shadow usage of companies using uh tools that aren't authorized. And I think we've done a really good job of uh rolling that out and communicating that. Uh and we asked it, we asked the question in ways that wouldn't be threatening, but there was very little, because we've promoted so heavily what the approved and preferred tools are, which is primarily Gemini and and Chat GBT with a handful of users on Claude, um, very, very little shadow IT usage was reported. And I really haven't had any um reports or concerns about that. Because we've messaged and managed it and said, um instead of saying don't do this, we've said do do this other thing, right? Use these tools. And by giving them tools, they don't have to go outside and try to figure out something else.

SPEAKER_01

So one of the things that you said, and I caught it, was that you were surprised by how many were using, but it wasn't 100%.

SPEAKER_03

It wasn't 100%. No. Yes.

SPEAKER_01

Do you do you know why the people that weren't using it did did you get did you capture any reasons why they weren't using their GPT license?

SPEAKER_03

So we did we did capture comments. I didn't review that before this call, but I'm trying to recall some of them. Some of them, like you said before, just don't they just don't know what to do with it. Yeah, it was it was rolled out to them because their manager requested it for them, but nobody really gave them training. They didn't really have time to do anything with it, they're just focused on doing their day-to-day job the way they know how to do it already. Yeah, others reported concerns about hallucination or um bad results. So that is really a training matter. So you can go back, we can go back to those users and say, hey, I noticed that you're concerned about this. Can we have you participated in one of our training sessions on prompting so you can understand how to reduce hallucinations and then how to identify them? And so yeah, I would say that primarily it was those two, just either nobody showed me how to use it yet, and I don't, I've got I'm too busy doing my stuff I know how to do already. So just a typical change management. People don't like change. And then the other issue was just being afraid of it because uh of these negative perceptions, oh, AI just hallucinates.

SPEAKER_01

Okay. So the reality is somebody listening to this podcast just bought AI licenses for the whole company. Right. I guarantee it. What what would you tell them to make sure that that they're getting the the most impact the quickest from those hundred plus licenses or whatever it is that they just bought for the company?

SPEAKER_03

You've got to train people on how to use it.

SPEAKER_01

Yeah.

SPEAKER_03

Uh the basics. So one of the first things we did was define an AI policy, an AI usage policy.

SPEAKER_01

Yes.

SPEAKER_03

So you've got to train people on how to use it with the policy. You've got to roll that out. You've got to give them some basic information about uh prompting and about the aspect of hallucination and how to avoid it and how to overcome that. So they don't just have a negative perception.

SPEAKER_01

Now, companies can do too much training, not enough. I don't know about too much, but it could they could do so much that it starts to interfere with with job performance or becomes like another job for everybody. How would you strike that balance? What would be the frequency? What would be the the duration of the training for let's say the CEO who just bought those hundred licenses?

SPEAKER_03

One of the things we one of the mistakes we made was inviting a large group to an all-day training session right off the bat. I think if you roll it out to power users in the and then you schedule follow-ups with that power user that now has some training, they can be your champion. And then you schedule follow-ups on a department level and then do that basic fundamental training. And then what we've kind of got dialed in now, which I think is going to work well, is about uh is one hour every other week. So, you know, one hour every other week, that's not onerous, it's not even mandatory. But that way, if people have heard about AI, they're not sure about it, or you've got new employees that come on board, uh, they can they can join that. Now, if a company has a formal training department and a learning management system, this is where we're going next. Uh, where we're gonna be implementing AI learning paths and actually putting courses in our learning management system. We haven't got there yet, but that's the next step.

SPEAKER_01

Yeah. So I would imagine that the trainings that you're conducting, you're gonna be productizing that for internal consumption.

SPEAKER_03

Correct.

SPEAKER_01

Very cool. So new and existing employees, as as part of their expectations for their quarterly review or their you know, annual performance review, will have been to complete X number of hours or X number of classes in the internal system. Correct, exactly. Yes. Okay, nice. Um, so uh I can tell you that you're one of the we work with we work with a lot of companies. Unfortunately, for AI at least, I mean maybe not for operations, but unfortunately for the AI side of things, most of them are are on Microsoft. Right. And I mean, I'll we got a call just for the listeners. We if you don't know, Microsoft's not doing such a great job with their copilot tool. It's getting better, but uh I would say that for as far as an office environment, Google has done a much better job for their like business suite integrating uh a capable model into the day-to-day tools. If you use Google, you know that you can it can help you with in your inbox, on your spreadsheets, and all that sort of thing. Yet, even though you guys are a Google shop, you still have chat GPT licenses, as well as there's others, some that are using clawed code, maybe, and some other things. What is the what does the stack look like? And then how did you decide who gets what?

SPEAKER_03

Sure. Yeah, this is something that's evolved over time as well. So we are, as you mentioned, we're not a Microsoft shop, we're a Google shop. So we've already got uh Google Workspace Enterprise licenses for every employee. When they become an employee, they get a Gmail account, obviously the company domain. So with that, we get access to a pretty broad suite of the Gemini tools. Initially, our Gemini VAR, uh, you know, our Google VAR, this is two years ago now, when they first started rolling it out, Google was gonna charge extra for the AI. And so we're forward thinking. So we we were like, okay, we'll we'll go ahead and pay for that because it's gonna be integrated into all the stuff we use every day. Well, Gemini was not ready for prime time at that point, and it just didn't work. I mean, most of the questions you would ask it, it would say, I can't do that. Yeah, that was a standard.

SPEAKER_02

I remember that. Yeah.

SPEAKER_03

And so that's what that's what drove us to Chat GPT. Now, fortunately, Google realized that because we had actually signed up and we're paying extra for these capabilities. And we about the time we went back to go try to get a refund just a few months in, Google basically dropped the extra charge and started including it for all their workspace enterprise clients. So um, but that was what drove us to Chat GPT initially was the promise of having Google integrated in was great. It just wasn't uh wasn't very effective at the time. And at that point in time, this was two years ago, you know, Claude was still primarily being used by developer companies. Yeah. Right? It wasn't really mainstream yet for business users. So Chat GPT was the clear choice. So we started getting, we got a Chat GPT team account, we started rolling that out, training people on custom GPTs, and so we got traction there. Meanwhile, over the past year, Gemini has come leaps and downs. So Gemini has really excelled. So uh it's interesting, Chris. We actually had part of that AI survey, we were actually assessing which users needed to keep Chat GPT licenses and which ones were perfectly fine using Gemini. So we were actually able to save some money because a lot of the users were like, yeah, Gemini works fine for what I do. And so we actually reduced our Chat GBT license count and redefined our AI stack because Gemini is now much more capable. So we now have a three-tier approach. Tier one, everyone in the company automatically has Gemini. So we're training everyone on the Gemini tools, right? Tier two are those power users that are doing things like custom GPTs and building projects where they need memory retained for static processes. And so those are the power users that are on Chat GPT, and then a sub tier of that tier two is the technical users that use Claude. So our IT team, a handful of our really technical users that are writing queries and codes and app scripts use claude. And then the third tier we haven't really talked about yet, is something we just rolled out. And this is our own private LLM for the purpose of accessing our internal data. Yeah, yeah. So uh I'd I'd like to, if I can, let me talk about the evolution of that because that was actually a big question. Uh so when we started down this journey, in fact, we were talking about this with you when we first started, uh, when was that June of 2025? Yeah. At that point, you know, all there was a lot of talk about data security and data privacy. So I had in my head, well, we're gonna segment things like engineering, proprietary data, we're gonna build a private LLM. And so I had directed our IT team to work on that, get a private LLM up and running so uh we could segment that out. Uh, what we found was it's cost prohibitive to download these open source models, run them on your own servers. Uh it's a lot of maintenance internally, and it's you're gonna pay for all those cycles, uh server cycles for that private model. So for some companies, that might be relevant, uh, but in our case, we weren't the data wasn't that sensitive. It's not like we're dealing with you know government data or highly sensitive data from our clients. So we became comfortable with the data privacy provided by the commercial agreements that you have with Gemini and Claude and OpenAI. I mean, Jim and I already had all of our data, right? And we're doing everything with Google Drive, email, calendar. It's already got all of our data. So, make a long story short, our IT team switched approaches and built a front-end LLM where we can swap out the model so we can plug in, you can even select which one you want to use. Okay, OpenAI, anthropic, or Gemini models, but built the RAG infrastructure, the vector search infrastructure, so that we can actually query our internal data in our ERP systems and our internal databases. And so, and just using APIs. So now we've got this three-tier approach. We've got Gemini for everyone for general purpose. We've got ChatGPT and Claude for building out these more sophisticated uh use cases at a task level where we're building projects and skills and GPTs. And then for accessing our uh proprietary systems data, we now have this third tier of this private LLM. And that has been a real game changer. Even the skeptics in the company that they can now, instead of having to know which form to Go run in an ERP system or writing some custom report and not having the column that they need to answer the question they're trying to answer out of our ERP system, all that. Now they can just go ask an LLM interface and it pulls data directly out of our ERP system and they could it could even tell them where to go find that if you need to go validate that. So that has been a huge win in terms of gaining over even the skeptics because now they've got access to that and they they see huge value.

SPEAKER_01

And I verified this a few times, but one of the first stats I heard when I got involved in generative AI was from a guy named Vishen Lacchiani from Mind Valley. And he was presenting, you know, whatever AI course they were selling. But he said the average knowledge worker spends, I can't remember how much time it was, but it was shocking, spends like 40% of their day looking for the knowledge to work with, right? Yes. And and having this um proprietary environment, that natural language interface, I don't have to go and hunt hunt it down or or merge cells or anything like that. I'm just able to get it from has got to be a huge win. The feedback from that, how long have you guys rolled that out? How long has it been?

SPEAKER_03

Uh we've literally just started rolling it out. We just did a scaled rollout over the past six weeks, and we've had a huge success.

SPEAKER_01

So the adoption of that or the usage of that has probably been higher than just Gemini or Chat GPT alone.

SPEAKER_03

Yes, because people it yeah, it's just so easy for them to just ask a question and get an answer that, like you said, it was a lot more challenging to go find the information before. Now they can just ask a natural language question. And for new users coming on, we're growing quickly. So we bring new employees on instead of having to train them on all the details of the ERP system and which form to go look up what in. And uh we have multiple sites, and so you've got oh, if it's not in the site one, you gotta log out of that, log into site two, and it's like you don't even know where it is, and now you can just go to one place and ask a question, and it gives you the answer. And then if you need more detailed information, it can tell you where to go look. Oh, it's on this form in this site. Great, I'll go there.

SPEAKER_01

So, for the listeners, what I would suggest, what Darren has done with that is you you're gonna want that, but I wouldn't make it either or and I wouldn't start there. I would suggest that you have, if you're on on Google, get people using Gemini. If you and if you're in Microsoft, get people using Copilot for sure. But either of those alone may not be the total gener the operational AI solution that you need. So be open to exploring Chat GPT or Claude. Which one do you use? Well, you know what? Like Claude was ahead of the game for a few months, then GPT five, like just pick one, right? They're both capable and they're both they're both very good at the the business security, addressing the business security environments now. Agreed. But while that's happening, if you can get your team to start to build what Darren has built at his organization. Now, Darren, some advice on that. If you had to start today to do that, um, what would you do differently? It sounds like you wouldn't try to get a proprietary, like an LLM hosted locally.

SPEAKER_03

Correct. Yeah, if I was to start today, we wouldn't worry about trying to use open source models. We would just sign up right away with those primary models, Gemini, Anthropic, or OpenAI, connect APIs, put a LLM any interface on the front. And um, you know, the the hardest thing is doing the back end connection to the uh database. So spend the time on doing the data connections. Don't spend resources on trying to host a private LLM model unless you just the company just really dictates that you've got to have that extra layer of data security, which we just wasn't a business requirement for us.

SPEAKER_01

Yeah. You know, one of the things that that I talk to companies about regularly is I tell them maybe you don't go out to the world and advertise, hey, we're on the AI journey. Maybe you you you know you do your experimentation unintentionally, but you do it in stealth mode just because there's a lot to figure out and you don't necessarily want to alarm potential customers that, oh my gosh, these guys are using AI? Because your customers have heard some of the same things that your employees have heard, which is hallucination and that sort of thing. But now let me let me address that real quick. Despite all of the talk about it, has hallucination been an issue for you guys?

SPEAKER_03

No, it's really not.

SPEAKER_01

It's not, yeah.

SPEAKER_03

It's yes, you have to verify the data, you ask for uh references and links, you go confirm it. But we you once we learned how to train users to structure the prompts correctly and then do the proper validations, it's it's really the value add is much far exceeds the risk of some bad information.

SPEAKER_01

Yeah. In your opinion, would you say that being AI forward is that uh an advantage in your marketing or your messaging with clients or anything like that?

SPEAKER_03

We haven't really used it as an advantage in our clients, our particular industry is not really a you know technology.

SPEAKER_01

They're not looking for that.

SPEAKER_03

Yeah, tech for industry, so it's really not looking for it. So, like as you said, we're really operating in stealth mode. That's why we're not even talking about a specific company. Yeah, um it's not something we need from a marketing perspective, and frankly, we'd rather our competitors not know what we're doing because we think it is going to be a competitive advantage for us just from a uh a productivity and innovation standpoint as we go forward.

SPEAKER_01

Yeah. And so for the listener, I want you to consider that. Um finding AI capable talent, it's not easy. You really do need to home grow the people that are on your team that already work for you. And here's what's gonna happen. A, yes, they're gonna be doing a lot better, like we've talked about, they're gonna be going from I'm skeptical to wow, I'm building tools that my entire department can use. That's great. But what's also gonna happen is as your business starts to grow, like the growth that Darren's company is going through right now, um you're not having to scramble to go and find talent and then you know, groom them into an AI capable contributor to the organization, because the people you've got, they're able to do more, do it faster, and do it at a higher quality because they're AI literate or AI fluent, even in a lot of cases, right? So it it's there the the competitive advantage just continues to compound the more you dig into this stuff about why you should be doing it. Now, if you're a listener that that has been on the fence, I mean uh hopefully today's conversation, and this has been the fastest hour on any podcast that I've done in so long, I've been wanting to have this conversation with Darren for a long time because I've been a witness to where they were a year ago and what they've become now. And it's it's absolutely impressive who Darren has become as somebody who was, you know, figuring it out, but but like doing the thing, committed to it, to now being, you know, going through chief AI officer certification and actually being somebody who is extremely competent. Like you could go and lecture at any industry conference now on that, and you would be bringing not theory, but boy, we tried this, it didn't work, but we tried this and it did work. Like there's not that many people out there that are doing this stuff. So what would be that that last bit of advice? Maybe not necessarily um something for uh, well, maybe we do a little bit of both. What would be your advice for the executive, somebody who's in your role, but they haven't gotten where you are? And then give me a little bit of advice for the the listener who is, I mean, they're listening to this podcast, they're AI interested, but maybe they don't have the influence within their organization. What would you recommend they do?

SPEAKER_03

Yes. Um, so for the executives, I would say, you know, don't delay. Yeah. Start doing something now because it's changing so quickly. Um, you know, I've heard an argument that, oh, well, what are we doing training on this? Because it just changes so fast it'll be obsolete tomorrow. And a lot of these skills that people are learning are not that tool specific. They're general skills. Uh, Chris, you you taught us it's learning to think in AI. And so when you think in AI and the technical aspects of what's changing, all you got to do is ask AI and it will tell you how to use the new tool or widget or a process. And so it's a matter of training a new mindset to be agile in thinking about how we do things because uh processes are gonna change and evolve so quickly that you don't have time to wait. So that would be my number one thing to executives is you know, start small, but start now. Start something right away. You can't put this off. Uh, for the person that is in the organization, maybe doesn't have the influence. Uh, I would say just start learning it on your own. You know, watch YouTube videos, uh, find out what your company will allow you to do. Uh, ask IT, ask HR, ask your manager, hey, what am I allowed to use here uh if they don't have any tools like Copilot or Gemini that have the security, you know, ask if you can get a paid GPT license that has the company's security turned on. But, you know, take it upon yourself. Um, I've often heard this said as well. Uh people are concerned about AI taking jobs. Um, an AI agent isn't going to take your job, but someone who knows how to use AI better than you very well may take your job because they're going to be more productive and more innovative and be able to adapt with the times. So that's the that's the advice I would have for both the executive and the just the operator within a company.

SPEAKER_01

Yeah. So uh for the listener, I just I want to make sure you understand like what we had a chance to do today is talk to somebody who he's got no products to sell that are AI related. He's not an AI consultant out there, he's not trying to be a thought leader. This is somebody who saw the opportunity that this would give to his organization to just be more efficient, operate better, blah, blah, blah. And he dug in over the past year and a half, now, you know, even more now, um, and figured it out and made the mistakes and took the risk of the investment and of people's time and that sort of thing. And now it's starting to pay off. So if you're sitting on the fence and you or you've got friends in your peer group or your vestige group or whatever that are kind of dabbling in it, because just because they're writing emails with AI does not mean that they're they're ready for the future, right? Forward this to them and let them listen to the the journey. Because would you say that the the AI effort has been successful? Yes.

SPEAKER_03

Yeah, absolutely.

SPEAKER_01

Absolutely. So, like this is a message that needs to be heard by more executives, but uh Darren's first, his last bit of advice to the executives was don't delay. Like it is time for you guys to do something. So, Darren, I just I know how busy that that you are over there running uh uh the AI efforts at a large company like that. And I just appreciate the time and the the honesty and transparency that you've shared with everybody on the call today. So thanks so much, man.

SPEAKER_03

Yeah, thank you, Chris.

SPEAKER_01

It's been a pleasure. All right, everybody. So um again, we'll be back next week with another uh amazing episode of Using AI at work. And I look forward to uh to having you join us then. If you know people who are on the journey uh or just getting started or they're experts, we'd love for you to um support the podcast by, you know, letting them know that we're out here, we're doing the good work, making sure that that this is accessible to all strata of the business, from the executive all the way down to the frontline employees. So thanks again for being a listener. Take care, everybody. Thanks for tuning in to using AI at work. Don't forget to subscribe for more conversations about how to use AI at work. And a special thank you to our sponsor, Chief AI Officer, for empowering businesses with AI education and training. Visit their website for free AI readiness assessment and AI strategy guide to help you get started using AI at work. That's www.chiefaiofficer.com. Follow us on Twitter at the handle UsingAI at work and visit www.usingai at work.com for free resources to help you harness AI in your role.