How AI Agents Are Transforming IT Ops

AI agents are moving from hype to the heart of enterprise IT. In this episode of the AI Proving Ground Podcast, Eric Jones and Ruben Ambrose — two leading AI experts — explore how intelligent, human-guided systems are transforming IT service management, incident response and operational scale to deliver faster resolutions, stronger security and smarter decisions across the enterprise.

For more about this week's guests:

Ruben Ambrose is a Chief Technology Advisor at World Wide Technology with over 25 years of experience spanning application development, IT operations, infrastructure, and enterprise architecture. He now leads teams focused on prototyping AI-driven solutions that accelerate innovation for global enterprises.

Ruben's top pick: Beyond Chatbots: How Digital Humans Are Transforming Enterprise AI Experiences

Eric Jones is an Area Director of Strategy and Innovation at World Wide Technology. He has been in the software industry for over twelve years. For the last two years, he has been focused on Generative AI and its application to enterprise workflows. This research helps World Wide Technology provide industry-leading services in building Generative AI solutions for customers around the globe.

Eric's top pick: Part 1: Transforming IT Operations with Large Language Models

The AI Proving Ground Podcast leverages the deep AI technical and business expertise from within World Wide Technology's one-of-a-kind AI Proving Ground, which provides unrivaled access to the world's leading AI technologies. This unique lab environment accelerates your ability to learn about, test, train and implement AI solutions.

Learn more about WWT's AI Proving Ground.

The AI Proving Ground is a composable lab environment that features the latest high-performance infrastructure and reference architectures from the world's leading AI companies, such as NVIDIA, Cisco, Dell, F5, AMD, Intel and others.

Developed within our Advanced Technology Center (ATC), this one-of-a-kind lab environment empowers IT teams to evaluate and test AI infrastructure, software and solutions for efficacy, scalability and flexibility — all under one roof. The AI Proving Ground provides visibility into data flows across the entire development pipeline, enabling more informed decision-making while safeguarding production environments.

Chapters

• 0:00 Setting The Stakes In IT Ops
• 2:26 Inside Service Management And Data Sprawl
• 6:01 Why Agents Beat Alert Fatigue
• 10:22 Defining Agents And Human Oversight
• 13:54 Under The Hood: HPE Private Cloud AI
• 18:10 Modular Hardware Choices And GPU Roadmap
• 21:12 Software Stack: NVIDIA NeMo And RAG
• 24:36 Safety, Privacy, And On‑Prem Control
• 28:01 Guardrails, Evaluation, And Human In The Loop
• 31:05 Practical Use Cases Across Incidents And Changes
• 35:05 ROI: MTTR, Avoided Outages, And Scale
• 39:03 CapEx Benefits Versus Token Costs
• 41:00 The Future Of IT Interaction
• 44:10 Key Lessons And Closing

Transcript

SPEAKER_00: 0:00

From Worldwide Technology, this is the AI Proving Ground Podcast. When something breaks in IT, a system goes down, a ticket floods in, a critical application stalls, it's easy to see the process as simple. You report the problem, they fix it. But behind the scenes, that fix is often buried under mountains of data scattered across countless platforms and handled by overworked teams trying to connect dots faster than the problems multiply. And that's where a new kind of AI is stepping in, a Gentec AI, built not just to answer questions, but to think, reason, and act with complex enterprise environments. On today's show, we're talking with Eric Jones and Ruben Ambrose, two deeply technical IT practitioners who are actually building and deploying AI in production and optimizing AI-powered systems that solve real-world challenges. We'll dive deep into how AI agents are reshaping IT operations, from spotting patterns humans might miss to reducing resolution times to transforming how we think about data risk and scale. We'll explore how companies are deploying these systems with Nvidia's Agent Toolkit and HPE's private cloud AI stack. Why keeping humans in the loop still matters and how this technology could redefine the way every employee interacts with IT. It's not just about futuristic sci-fi bots, it's about solving the problems that slow your business down right now. So stick with us because the way we manage IT won't be the same way we manage it tomorrow. And these two are already building what's next. So let's jump in. Ruben, Eric, welcome back to the Ad Proven Ground Podcast. I think this is your second time appearing. How are the two of you doing today?

SPEAKER_01: 1:44

Hey, doing good, Brian. How about you?

SPEAKER_00: 1:46

Doing good. Eric, how you doing?

SPEAKER_02: 1:48

Doing well.

SPEAKER_00: 1:49

Happy to dive in here. Absolutely. We're talking about AI agents for IT ops. I I will admit, you know, Eric, I'm going to start with you. Um, when I think of IT, I think of quick fixes. You know, my Wi-Fi goes down. I call IT, uh, something happens on my laptop, I submit a ticket. I get the sense that IT sees my tickets as just, you know, a mountain of requests that they have to get to. So before we dive into why AI is going to be helpful, um, can you level set for us on just kind of the state of IT ops and you know what's unfolding right now in terms of you know not only pressure and number of requests, but also from a talent perspective?

SPEAKER_01: 2:26

Uh sure, Brian. Uh, I think we can talk about that a little bit just to maybe level set the conversation. So most IT shops, you know, they have very large environments they're managing. They have very large user bases. Um, they have a lot of different systems they use to manage their work, both technical systems that watch what's going on in the data center and on people's machines and devices. Then they have all the systems that use just internally to organize their work. Uh, service management platforms typically are what they use to do these kinds of things and to coordinate the different teams within IT and to interact with their customer base like yourself, right? Um, when I say service management, think of things like ServiceNow. It's a very popular example of a service management platform that probably most people who've worked in large enterprises might be familiar with. When you as a user need to get something done, you need your password change, you need to get a new laptop, something's broken, you need it fixed. You know, we're all familiar with going out and opening a ticket, right? Uh put that request in. That's typically a service management platform, is what you're working with there. Uh, you're on the customer-facing side of it, but then folks who actually like fulfill the request or try to resolve the problem in the IT team, they're on the fulfiller side of it. So there's really large systems involved just in getting requests managed, handled, making sure that nothing's slipping through the cracks, people get quick responses to things. And you know, these systems also store knowledge for the IT organization, right? We fixed this problem last week. Let's record that somewhere. So if we see this problem again, we know how to fix it again. Or if you see this issue, run these commands on this system and that should resolve it or give you more information. That kind of thing, right? These systems are used not just for fulfilling requests, they're used for IT to keep track of changes they're making to the environment as they upgrade things, patch things. Um, you name it, they manage their projects in there, they manage their financials in there. And that's just a service management platform. There are other systems that system administrators use all the time that watch the infrastructure, what's resource utilization look like? Are we running out of storage on disks, those kinds of things? And all these systems generate, you know, data and information. Uh, and they're these piles of things grow, right, week to week, year to year, as everything is up and running. So teams in working through their daily uh environment, dealing with their daily requests, work in these systems that have, you know, think of it as a haystack getting bigger and bigger and bigger and bigger. And a lot of its stuff is kept for posterity, a lot of it's very pointed information that needs to be remembered and retrieved later on for specific reasons, like the example I just gave where we're trying to solve a problem we've seen before. We wrote that down somewhere. We need to find that. Um, but that's kind of what they're dealing with, right? You need these systems, they generate a lot of data, but after a while, the challenge can be that you have a lot of data, but it's hard to find the information in that tile of data that you need at a given point in time.

SPEAKER_00: 5:41

Yeah. Eric, so uh build on that. We kind of have this perfect storm of data demand. There's certainly a little bit of human limitation in on this mix as well. So why is AI ripe to solve for the some of those challenges? Specifically, why are um agents uh ripe to solve some of these problems?

SPEAKER_02: 6:01

Yeah, and I'll I'll use an example, uh, Brian, and it's timely in my mind because we actually had a a situation like this that happened in our environment just yesterday where uh you know there's a in a third-party system, let's imagine a situation where that that third party system ends up having an outage. You know, I think a lot of us uh who were around at the time when AWS went out several years ago and it was you know ended up being a root cause of an intern running a script in an environment that shouldn't have had production access and did, right? But that type of small situation where it ended up taking out large portions of the internet. You think about an IT environment where uh they're not necessarily seeing that root cause at the time, but for large organizations, they may be seeing a ton of tickets coming in, a ton of information coming in that says, you know, this mission critical site is down, this site is down, uh, another internal thing is down. And the uh on the outside, all of those tickets do not appear to have commonalities between them. But at the end of the day, they boil down to some root causes, right? In that example, it's AWS is down, powering large parts of the internet. Um, and then of course there's root cause analysis on their side. But the reason that you know AI is so well poised to tackle this problem is that um, you know, human individuals only scale so far. So for uh large organizations, lots of uh individuals inside of the IT centers, whether it's the call centers, whether it's you know the actual individuals taking tickets that have been created either automatically or by users, those are getting routed to different parts of the system. And those individual humans may not be able to talk to each other in real time, right? Maybe there's some chats going on behind the scenes of, hey, I'm seeing this and this person is seeing that. Whereas an AI system and particularly an agentic-powered system is able to be a little bit of a forefront into that uh noise coming in to see patterns and to recognize those types of uh incidents that may not look similar to the end user, but there's a there's a root cause, right? And so some of the things that uh we're looking at tackling as a part of this entire uh program is to take a look at how do you get broad visuals across the entire uh system of an IT organization and try and correlate uh situations that you know may be the same, or how do you help uh mitigate those situations faster.

SPEAKER_01: 8:52

I think one key thing to tack on to what uh Eric just said there, um, Brian, is you know, specifically, yeah, a lot of these systems have databases in them, right? At the end of the day, and we're all used to those. You have databases and you store big data's informed fields, and things are very discrete. But in the kind of scenarios we're talking about, a lot of the data being captured is literally just conversation going on back and forth. Think about the last time you opened a ticket, it'll be service desk, and then they said something, and you said something, and then they said something, and so it's kind of unstructured text data. Yeah, and the whole conversation is kept in that system, and now multiply that across hundreds of incidents that are going on every day, hundreds of change tickets, etc. Right. Uh what LLMs and this these technologies bring to the table is the ability to parse through and quote unquote understand that text, that human language, and read it and summarize it and gain insights from it and surface information from that kind of like just big field of words, right? To think of it that way.

SPEAKER_00: 10:17

Agents, I feel like these days get a wildly different definition based on who you're asking or what they're doing. As it relates to IT ops, Eric, can you I can you kind of just define a little bit about what types of agents or what we mean by agent?

SPEAKER_02: 10:32

Yeah, and that's one thing, Brian, that we're really uh striving for with this entire project is to really take agents into the true meaning of what uh we envision them to be and to bring that power to table, right? To be uh one step further than just a series of automation steps, but to actually have you know portions of your system that that's where we'll refer to those as agents that can uh take you know the individual information with a specific goal, that they can analyze different portions of data with that specific goal in mind and kind of report back to the core orchestration layer. Uh, something that I'm sure we'll get into later on in the podcast is that we're leveraging NVIDIA's Nemo agent toolkit uh very heavily from the ground up in this system, which is something that uh is helping us to kind of piecemeal those agents out into those specific roles and eventually attaching those uh additional tools for not just that agent with a very specific uh outcome and goal in mind, but also to have that group of tools that it can uh identify what actions should be taken, which is something that uh Ruben and I actually have we've actually had many conversations as we've been uh designing a system as to whether or not we want our agents to be able to take that action. Uh, I think at the moment we are leaning more towards the area of caution of uh the human in the loop system where you know an agent will identify the tools and identify the the right path forward. But Ruben, I think your uh years of experience in in IT apps have been weighing in on the decision that maybe we don't want to just let the agents loose to let that action take place.

SPEAKER_01: 12:27

Yeah, uh that that's definitely the case. I mean, especially as people need to get used to these systems and what they offer, I think you definitely still want a human in the loop. And you want the AI acting more of an advisor and a guide, more so than an autonomous thing that's just gonna go do something in your data center uh without you necessarily saying, yep, for sure I want to do that, or without you even just applying the proper change protocols that need to happen before a change actually goes into the environment, right? That most IT shops want to do. They want to have everything audited and recorded and make sure tests are run and stuff like that before changes get put in to the actual production data center.

SPEAKER_02: 13:05

Yeah. And I and I think that being said, you know, going back to what our goal for our agents inside of the system, we do want them to be going basically up and to the point of taking action. So I've you know analyzed the data, I've looked at the knowledge bases, I've looked at previous incidents and how those were solved, doing all of that legwork or mental work for you, and then having a pretty robust solution of, hey, this is where I think we should go, and then passing that information along to the human and making it um, you know, as easily understood as we as we can in these systems with making sure that the proper change protocols stay in place and that that human in the loop is ultimately who's making the choice.

SPEAKER_00: 13:53

Yeah, absolutely. Let's dive under the hood a little bit from an architecture standpoint. Um, you know, what types of technology is powering this? Eric, you alluded to uh some of it already, but what what is powering this project, this solution, um, knowing that we have to have these these agents work together across many systems and in some cases a lot, a lot of systems. What is uh powering in here?

SPEAKER_02: 14:16

Yeah, and I think the way that um Brian, that we're designing this and that Ruben and I have been thinking through this, is that there's very much so a hardware component to these systems, and there's a software component to the systems. And uh in that vein, you know, the hardware perspective, we've really been focused on how uh enterprises would want to be bringing in uh particular hardware to be able to solve for uh and and host the software infrastructure on top of that hardware. And so we've partnered very closely with uh HPE and specifically focused in on the HPE private cloud AI stack and platform uh for our enterprise and the solution that we're building out. And there's been quite a few reasons why we we chose to go that route from a hardware perspective. I think one of the biggest is that you know, as we have worked with many customers in this space, which is a lot of what um made us want to work on this type of uh platform, is that we've noticed that there's a desire to have uh modular hardware that's well understood, that's easy to configure and build out upon, that's also um ready for your enterprise today. And so what I mean by those types of things are you know, is it a uh an air gappable system? Can this system run inside of an air gap situation where some of our customers, uh whether it's the financial space or utilities or anything that's that's regulated, are gonna need to deploy this completely air-gapped check. Uh we're looking at other things like is your data center ready for liquid cooling? I think we've seen that the industry at large is going to be moving more and more towards liquid cooled data centers, but we also understand that that's a large challenge, right? And so uh looking at a system that uh can be completely ran on an air-cooled system and ready for your data center today, and just having all of the software infrastructure that goes around that, right? So the Kubernetes uh layer that is inside of the HPE private cloud system and the data fabric and the analytics that come along with that really lended itself well to building out the solution and kind of what's going to be sitting under the hood. And of course, as we're talking about AI and LLMs, uh the NVIDIA GPUs that go in and power that system is also very, very important. And so the HPE private cloud system starts with your L40S GPUs, multiple different sizes that you can configure those into. That's what our team is developing on today. Uh and they also get into the H series on the larger side of that. And then looking forward, there's that Gen 2 uh nature of this hardware platform that gets into the new uh A6000 Pro Blackwell architecture from a GPU perspective, something that I'm personally really excited about and something that we're uh bringing into the AI proven ground very shortly for this engagement. And then getting into higher parts of that uh H series of GPUs. So I'm really excited to be able to host that much uh GPU and compute power that's going to be powering the various uh LLMs and AI infrastructure that we're gonna need for uh the software stack to sit on top of that. And then I don't know, do you want to talk to the software stack a little bit?

SPEAKER_01: 18:01

Well, actually, before we get Derek, I wanted to uh see if you wanted to dig in a little more. Um, you know, and that I had when we talked about with regards to like overchoice and UAD HP platform is kind of almost t-shirt size, and it makes that whole selection just straightforward because you have like a fixed set of options instead of kind of skies at the limit. You want to maybe dig into that a little bit? Because we've heard feedback from customers on that as well.

SPEAKER_02: 18:25

Yeah, no, that's a really good call out, Ruben, and something I I forgot to circle on is that we have seen that from customers as we get into uh you know various engagements where there is just a lot of options when it comes to hardware, which is it can be a good thing, especially if you're trying to specifically design a system that fits very well into your existing IT or data center. But also having uh pre-configured or or modularized uh systems that you can more or less choose from in that more small, medium, and large t-shirt sizes is something that we we've seen from our customers is a desire for that. And a lot of that conversation comes down to also understanding okay, if I buy this, the small system with two L40S GPUs in it, what can I deploy on that, right? How many tokens per second am I going to be getting? How many concurrent users can I support on that system? And as those questions come in, it really helps to focus on that modularized, okay, here's what you can do on this system or this or that system. Uh, but then also having the ability to expand on those and grow them once you've made a choice is something that uh the HPE platform brings as well. So it's kind of the best of both worlds where it simplifies some of those choices, but also has some of the configurations that are going to fit nicely into your data center. And then in addition to that, it can continue to grow and you can continue to expand uh onto that system to make sure that you know, once you start leveraging the GPUs and once you start seeing the value out of it, that you can continue to expand that module out.

SPEAKER_01: 20:10

Right. From a software perspective, like you were asking about, Eric, um, at the heart of the solutions that we've been walking through designing at this point, typically as you would expect, there's a larger language model at the heart of it. Uh, and in some cases, we're going to couple that with retrieval augmented generation. Uh, we're building everything, as Eric mentioned earlier, using the Nvidia uh agents toolkit. Uh NAT is what they kind of call that for short. Um, it's a nice development framework that kind of allows you to accelerate your developments, the easiest way to describe it. Um, and a lot of the models we're deploying, you know, we're deploying them as an end, which is another in kind of NVIDIA kind of framework technology. Again, just all to speed up development, speed up how long it takes to deploy things, how easily you can pull this one out and switch something else in if you want to try a different model, for example, to see what kind of results you get and that kind of thing.

SPEAKER_02: 21:09

All right. I think some of the things that that we've seen, some of even early on, Ruben, the the return on that, those decisions is as our team dives into different use cases and we try something out, maybe it doesn't work well at the very first time that we try it out. And I know even that first use case that we were doing, Ruben, we kind of got stuck a couple of times where the results were not quite what we wanted to be getting. Right. And yeah, leveraging that the common framework in the in the Nemo Agent Toolkit helped us, you know, I think uh our engineers uh were able to kind of turn on a on a dime overnight, even and uh one day came back with a whole new solution to our first approach, and it turned out to be a much, much better system that um that we were happy with, right? And having that that common uh software foundation was something that allowed us to make that pivot quickly.

SPEAKER_01: 22:11

Yep, absolutely. That and the um kind of the add-ons that NVIDIA seems to be adding into that whole environment, that whole framework, you know, feature things you can add into test the efficacy of your prompting and how well it's working and how well you tune those uh things to help you figure out uh evaluate the content of the output and how does it change from output to output to output as you change different parts of the system, right? There's an evaluation framework they've added in to it as well, and they're adding more pieces like that basically to help the development experience.

SPEAKER_00: 22:52

Okay, so we're we're talking about very powerful models, very powerful GPUs and infrastructure. Um, but as we know, you know, these the performance can only be as you know powerful as it is safe. So, what are we doing to make sure that this is secure and take into consider you know privacy and things of that nature?

SPEAKER_01: 23:14

So that's kind of one of the benefits and the beauty of the DHPE private cloud AI platform. Uh everything's local, right? Everything sits in your four walls. It's a piece of infrastructure sitting in your data center. So your data doesn't leave it and it's always within your control, and you're not sharing anything with any external parties. Uh, they can't infer anything from what you're sending to it, what prompts you're doing, anything like that. Um, so the privacy and the data security is pretty much as good as you can expect because you're running everything on premise.

SPEAKER_02: 23:51

Yeah, and I I think in this context too, Ruben, um there's an added layer of importance when you're talking about the particular uh nature of the data that we're moving in and out of the system. And uh one thing that, you know, I I was not even thinking about early on in the development is that uh it was pointed out, you know, we have data coming through the system down to the IP address range, right? And so when you think about your security posture and from a cyber security layer, some of the worst things to be leaking out there is, you know, exactly what are your IP ranges on internal devices? How would you be getting from one device to another device, uh knowing that one device has been compromised, which could be a part of the longs inside of the system? So you're really talking about a different even classification of data versus you know somebody trying to summarize an email that might have a customer name in it. That is something that you would want to make sure is secure, but maybe isn't going to bring down your entire system or expose you to cybersecurity threats. Whereas some of the data that's flowing through this system, you absolutely would not want to be leaving your four walls just because it's it's almost like your uh your plan of attack all on all on a single map.

SPEAKER_01: 25:19

Yeah, so it's IP addresses, it's things like usernames, even maybe admin login usernames that are being used, uh commands that are run on systems to do things that an administrator would do. Yeah, you're right. It's all that kind of data that's involved with some of these things, and uh it's a little higher degree of sensitivity for sure.

SPEAKER_02: 25:39

Yeah, and even um even firmware uh versions.

SPEAKER_01: 25:43

Firmware versions, yep, what patches are applied to a system or not applied to a system, or oops, we found out we need to apply this patch because it fixes this vulnerability and we can't put it in for another week till the next maintenance window. Well, okay, so now we have a week that there's a known vulnerability on a server. You don't necessarily want to advertise stuff like that.

SPEAKER_02: 26:02

Yeah, yeah. I think to to try and translate that to um out of IT speak, maybe it would be you know putting inside of the local paper which one of your window locks is broken and uh that you need to replace next week.

SPEAKER_00: 26:18

Yeah, well, I always appreciate it when uh you can help me uh decipher the IT speak. So thank you for that, uh Eric. What about um agent performance? Um, you know, what about the risk of an agent um coming up with the wrong assumption or reporting back the wrong uh level of detail or context? What what are you what what are we doing to safeguard um from that perspective?

SPEAKER_01: 26:40

Yeah, this is why we want to make sure there's a human in the loop, right? Uh when it comes down to it, a lot of these models operate on a statistical basis, and it's not deterministic programming in the old days where these inputs will always give exactly these outputs, and you always know that it's going to be the same every time. Because there's a statistical component to how a lot of the technology operates, you can't guarantee everything will be exact from time to time. And so, because of that, we can't remove the human element to make sure that what you're getting is valid. It's no different than all these different chatbots we're all used to using now throughout all the different tools we use. And you always see a little warning that says, please make sure and check your results, right? AI can sometimes make a mistake. You need to make sure what you're seeing is what makes sense to you.

SPEAKER_02: 27:36

Yeah, and I do think that the there's a delicate balance there of making sure that you have that right level of security. And in our um system that we're building, that measure of security is basically just a hard stop, that an ability for the agents to carry out those actions, right? There's the ability to propose that to the human in the loop, but they don't have the tools or the ability to move farther than that. Um, but also making sure that you don't make that hard stop so early on in the process that the suggestions that come back from the individual agents are not necessarily things that a user is going to be saving time on. Because that's at the end of the day, what we're really trying to do is reduce your mean time to remediation on these various tickets. We're trying to get systems uh up and running as fast as we possibly can and helping organizations to scale. So you have to make sure that you do strike that balance with your agents of hey, there's a there's a safety protocol, if you will, and you have to define that up front for whatever system you're developing. We've done that for ours. But then how do you allow your agents to run pretty much right up to that protocol to give you as much value as you can possibly squeeze out of that system?

SPEAKER_01: 28:58

And I think there is another layer that kind of protects us given the use cases we're talking about, Brian. So to give you an example, we put an agent together that helps uh somebody in your service desk work incidents quickly. Okay. So service desk person is working on incident, the incidence is, you know, it's any notes. This is the problem I'm having, the usual shows they're having, and this is the error message they're seeing. Well, we want the AI to go and look through every single incident that's in the system to see if there's anything that matches. We're gonna bring back the top three that we think match and then present those to the service destination to say, hey, here's three that look like the same. Go look at how those were closed because you probably want to repeat. If one of those three actually isn't a good fit, uh that's okay, right? We've still helped the user. The first two are good fits, and they can easily by parsing through the three, figure out which one to pay attention to and which one to not, even though the third one may not be that great of a match. And we've saved the user from having to like, you know, there's no real way to parse through thousands and thousands. incidents, right? So they would never even have gotten to those two that are actually useful. Does that make sense?

SPEAKER_00: 30:06

Yeah. No, absolutely.

SPEAKER_01: 30:07

Maybe offer up a coup a couple more um use cases here, or maybe even if we've uh you know if we've had any real uh world uh case studies uh that we can kind of detail here yeah so I think the incident case is a good one um another example would be hey I have this problem um are there any knowledge base articles that address this specific issue so now you're not looking in the incidents at all you're looking in a totally different place which is where work instructions and things like that are kept uh some of which are exposed to you as an end user Brian you've probably seen one where hey if you want to go change your password read this knowledge base or because we'll tell you exactly how to do it right services people have work instructions like that too um for the stuff they do but another example on top of that I would say is okay uh you're somebody who is a manager in an IT operations department and you just want to understand generally the pulse of what's going on this week in the environment right across multiple things that are happening against incidents that are being worked because people need things resolve against changes that are happening this week that we might need to pay special attention to um trends and kind of higher level insights. So in a change example let me give you one there. We have a bunch of changes recorded in the system are going to happen in the next coming week. Have any of systems that we plan to touch had issues when we've done similar changes to them in the past and cause outages right that would be something I would love an AI to be able to look through all the past changes, look at what I'm doing this week and kind of bring those to my attention so when we do the change on system X, whatever it is, uh we don't fall into the same gotcha from the last time uh or we take precautions right in advance like completely avoid the problem or if it happens we know what happened and why it happened last time and we know in advance what we have to do to like resolve it before we cause a bigger outfit you'll have a bigger downtime you are expected.

SPEAKER_02: 32:18

Well Eric let's pull on that value uh word a little bit more here we've got the systems in place we've got the software the infrastructure um the safeguards everything in place um but you know the it all hits the road when it goes into production so what types of metrics or um ROI signals should organizations be looking for uh to to not only help you know make sure this is working but to help build momentum for further investment yeah so I I think there's a there's a couple of broad uh variables that I would look at there the one that I mentioned earlier right the mean time to resolution I think is the easiest and most straightforward metric that you'd be looking at um which is simply a ticket comes in how long does it take before that ticket gets resolved and is that uh time span getting smaller and smaller I think that's a really good metric to be looking at because it does indicate real ROI. I think it is also um you know just kind of a a tip of the iceberg if you will as far as things that that companies should be looking at and that that we're looking at with the system. I would say some of the the bigger ones that I'm hoping that we can continue to tease out over time that is not necessarily tracked is what are the what are the errors that um didn't happen or what are the things that didn't happen because this system was in place. Whether it it caught you know a pattern early on and it stopped that pattern from propagating into other users or you know whether it uh was able to actually predict something that was going to be happening in the future based on things that had happened in the past kind of like what Ruben was saying those types of uh return on investment metrics are really hard to be tracking over time. But I will say that uh when you see one and I'm I'm hopeful of the system that we will be seeing them as time goes on, everybody can point at it and they can say oof like that just saved us a week's worth of downtime or something like that. And what I'll say is it really only takes one or two of those every single year to make that ROI very tangible for your stakeholders all at all levels of the organization. And these are things that we've seen with some of our internal tools. I know uh as an organization we talk about the RFP assistant all of the time but that is something that uh this metric is kind of a a hop away from a metric that we've seen in that tool that is you know what are some of the things contracts or projects that we've won that we previously would not have even been able to put in for right and in this case it's kind of that similar um diamond in the rough type metric of where are the events that that could have gone catastrophically wrong and they didn't because of the system finding it earlier, synthesizing it earlier. That's one of the biggest things that I'm hopeful that we can uh begin to tease out. But you know the most tangible one that you're going to see is that I mean kind of resolution. And then you know I think as that begins to also scale out which is the other variable that we see uh as enterprises really wanting to hone in on is how does my organization scale? Right. And I think that when you think about IT operations, that problem does not just become you know a people problem, right? I think you can scale by hiring more people. But what we've seen consistently across these uh organizations and in this space is that how long does it take you to train that person once you've hired them on right and if you have you know people who have been working in these systems for five 10 15 years they have a lot of knowledge right and that um that knowledge doesn't scale with newer employees uh as well as maybe if you can make those individuals more efficient you can have that same level of expertise now augmented with this AI system so that you can continue to do more and more work and and also help those individuals who are experts maybe not get stuck up on some of the monotonous tasks that uh AI could do for them.

SPEAKER_01: 36:48

And Eric we've worked with specific customers who have tried to build specific AI chatbots that are knowledge based right like expert system knowledge for their employees to work with and query basically the manual right on how to manage a certain system or piece of gear and stuff like that. I know we've done work specifically with customers and deployed it for them for real real world use cases.

SPEAKER_02: 37:11

Oh yeah absolutely and and we had um we had one uh I was having a conversation not too long ago uh around you know they wanted the knowledge base to basically be uh this person right like when something goes wrong they always go talk to this person how do we create a knowledge base that is just around the questions that are asked of this individual uh so that you know we can free them up to do other things um is is one of the more interesting ones that I've seen. But yeah we we build out those knowledge bases kind of all the time for customers and I think this is just continuing to evolve on that pattern of uh instead of just one knowledge base or two, it's it's really trying to take a holistic view on the IT operation plane.

SPEAKER_01: 37:58

Yep. Brian you were asking about return and I know you meant that more for the solution but there is a return aspect to talk about from the infrastructure perspective as well. Like we said a few times the HPE environment it's a hosted environment it's in your data center. The other plus that comes with that is it's a capital expenditure you have your gear and then it's fixed. And you can contrast that with a model where you're using models that are hosted externally and you pay by the volume of tokens you're using right when you send it prompts and how many questions you ask it and how big the questions are and stuff like that. And you know there's variability in that what your costs month to month are going to be are going to fluctuate you can run out of tokens if you don't have enough in your account that are billed every month you have to find yourself increasing those on the fly if you need to um you kind of avoid that kind of variable unknown expense let's call it uh in a system like this where you bought your hardware it's there you can use the crap out of it every day 24-7 and the cost isn't gonna change you know it is what it is it's a known quantity.

SPEAKER_00: 39:04

Yeah no it's speaking to a very practical um solution that's gonna drive value um but at the same time it's also pointing to um a larger transformation so to speak and you know we're getting to the bottom of the episode so we'll end on this uh this one you know this seems to be not just um a solution or a technology that's gonna change how IT ops works but how um everybody uh works and interacts with IT or interacts with their own systems from an IT perspective to the both of you maybe Eric start with you you know where do you see the puck uh moving in terms of how humans interact with AI, AI agents um as it relates to the to IT um you know and how businesses work yeah I think one of the downstream effects of what I'm hoping a a system like this can be helping with is that you know going back to some of the practical use cases and things that I run through every day uh I had a situation uh over the past couple of days that I was trying to get access into a particular system.

SPEAKER_02: 40:04

Um and it was even something that uh when Ruben mentioned it to me, I said, oh yeah I've always had problems with this system because historically speaking I've needed access in different ways. And uh I I ended up putting in a a ticket for the that system. And then the ticket kind of got returned to me that said hey there's an automated form that you can fill out fill out the automated form. Of course that did not end up working for me because of kind of a historical context had to submit another ticket saying hey this thing did not work had to kind of bug somebody inside of IT uh even though I I know that this should be easier than it is it just isn't for one reason or another. And I think it's those types of interactions uh on both ends that I would hope that we can start to help with um on the user end in that case being me. Like I know that getting access to this system I basically know exactly what I need but I don't have the ability to go and get it for myself. And so I have to go through a couple of back and forths um which is time spent that I'm not spending doing other things. And then on the other side the IT operations perspective having to get uh that human in the loop and saying yeah I know I did this I did that I mean I think we've all been on that um call before where you went through the call center and you did all of the options and you finally got to the person and they start the run book all over on you. That's the type of stuff that I would I would really like to help out with on this. And then I think the other thing that I'd really like to help out with is some of those nitty-gritty details um going into kind of network logs and actually system logs and understanding connectivity problems. I think that one of the hardest things from an end user perspective is when your tools just aren't working, whether it's something as basic as your internet connection in the office, connecting to a printer, something of that nature that it's a silly problem that ends up taking up half of your day. But then by the time you get connected with somebody inside of your IT operations group the problem is either resolved your internet's back in back in play or you were able to print or whatever it was from a silly perspective. But then being able to uh analyze those longs and actually understand in in more of a real time how to fix those problems I think that's a really interesting space that this could go as well.

SPEAKER_00: 42:34

Yeah no absolutely I mean connecting to a printer bane of my existence so I'm happy to hear that you'll be tacking tackling that one in the future. Ruben any uh any yeah any final thoughts on kind of you know where uh the few what the future holds yeah I think I want to double down on Eric's first example the one he just gave where he tried to resolve a problem went back and forth and didn't fix it.

SPEAKER_01: 42:54

We also had other folks on the team who went through the same thing a few weeks ago and all the stuff that is needed to know how to fix it and what the actual process is is definitely recorded in the system somewhere right from one of those earlier conversations. So if when Eric logged us to him we'd just been able to service that up to the person dealing with it on the IT side his experience would have been totally different.

SPEAKER_00: 43:19

Yeah. Well Eric and Rubin thanks again for for joining us on this episode I know it was a meaningful conversation I took away a lot from it and I know this is something that you know our clients organizations listeners out there a real tangible use case that they can use AI right now to benefit their organization. So thanks to the two of you for joining in an otherwise busy schedule.

SPEAKER_01: 43:39

Hey thanks Brian thanks for having us thanks for having us so what did we learn today?

SPEAKER_00: 43:43

Three key lessons to consider first AI agents are more than automated scripts they're intelligent collaborators that can analyze unstructured data surface hidden insights and recommend next steps all while working alongside humans. Second building trust into these systems isn't optional that means keeping sensitive data secure through on-prem architectures enforcing human-in-the-loop safeguards and designing for explainability and auditability from day one. And third, the value is measurable from inducing mean time to resolution and preventing costly outages to scaling institutional knowledge across teams and unlocking new efficiencies at enterprise scale. The bottom line agentic AI isn't just transforming IT operations, it's setting the stage for how humans and machines will collaborate across the entire enterprise and those who embrace it now won't just keep the lights on, they'll light the path forward. If you like this episode of the AI Proving Ground podcast, please follow, rate or review us wherever you listen and join us next time as we continue exploring how AI is reshaping the enterprise. And you can always catch additional episodes or related content to this episode on WWT.com. This episode was co produced by Nas Baker and Kara Kuhn, our audio and video engineers John Knobloch my name is Brian Felt. We'll see you next time