Your AI Is Only as Good as Your Data. Here's Why.

Nearly 90% of enterprise data is unstructured. That's a daunting statistic until you realize it may also be your biggest competitive advantage.

In this episode hosted at Cisco Live, Brian Feldt sits down with WWT's Jeff Fonke, NVIDIA's Jason Schroedl and NetApp's Mike Hommer to explore why data has become the defining factor in enterprise AI. From token economics and GPU utilization to hybrid AI infrastructure, the conversation keeps coming back to one idea. AI is only as valuable as the data behind it.

If your organization is moving beyond AI experiments and into production, this episode offers a practical look at the decisions that matter most and why bringing AI closer to your data may be the smartest investment you make.

More about this week's guests:

Jeff Fonke leads WWT's High-Performance Architectures team, where he helps customers build the AI infrastructure they'll rely on for years to come. With more than 25 years of experience, he brings a practical perspective on what works, what doesn't and where enterprise AI is really headed.

Jason Schroedl is Director of Product Marketing at NVIDIA, where he helps shape how enterprise AI moves from innovation to production. With more than two decades in enterprise technology, he brings a practical perspective on AI infrastructure, data readiness and what it takes to scale AI successfully.

Mike Hommer is a Field CTO at NetApp, where he helps organizations make smarter decisions about AI, data and infrastructure. With more than two decades in enterprise technology, he focuses on separating lasting strategy from short-lived hype and helping customers build for what's next.

The AI Proving Ground Podcast leverages the deep AI technical and business expertise from within World Wide Technology's one-of-a-kind AI Proving Ground, which provides unrivaled access to the world's leading AI technologies. This unique lab environment accelerates your ability to learn about, test, train and implement AI solutions. 

Learn more about WWT's AI Proving Ground.

The AI Proving Ground is a composable lab environment that features the latest high-performance infrastructure and reference architectures from the world's leading AI companies, such as NVIDIA, Cisco, Dell, F5, AMD, Intel and others.

Developed within our Advanced Technology Center (ATC), this one-of-a-kind lab environment empowers IT teams to evaluate and test AI infrastructure, software and solutions for efficacy, scalability and flexibility — all under one roof. The AI Proving Ground provides visibility into data flows across the entire development pipeline, enabling more informed decision-making while safeguarding production environments. 

Transcript

:00:00 - AI Just Changed the Rules

SPEAKER_02 0:00

Agentec AI is placing a new kind of pressure on enterprise infrastructure. GPUs need to stay busy. Models need constant context. Token consumption is rising. But the biggest obstacle to getting a handle on all of that may be the scattered, unstructured, and poorly governed data these systems depend on. So how do you bring AI closer to the data without multiplying cost, complexity, and risk? That's where this episode kicks off. I'll be talking with Jeff Funkey, who leads new and growth solutions here at WWT, including high-performance architectures that support AI and data. Mike Hammer, a field CIO with NetApp, and Jason Strodel, NVIDIA's Director of Product Marketing, to explore GPU economics, hybrid AI factories, and why data readiness is quickly becoming a competitive advantage. Quick note, we recorded this episode live on the show floor at Cisco Live. And as you'll hear, the lesson is rather simple. The companies that understand their data, control their economics, and build an infrastructure that can keep adapting as AI changes will be the ones that thrive in this era of incredible innovation. But executing that strategy as many are learning right now in real time is anything but easy. This is the AI Proving Ground Podcast from Worldwide Technology. Let's jump in.

The New Math Behind Enterprise AI

SPEAKER_02 1:36

We're talking about GPU optimization, but a lot of times that's not strictly the cost center here. A lot of times, uh at least as we've talked about it, you know, data can be that real bottleneck. What are you seeing in terms of GPU utilization, data optimization, and where the cost sits right now?

SPEAKER_00 1:51

Yeah, it's a good question, Brian. And I think when you looked at the one of the slides G2 had up today, it was you know, before the chat bot, you know, the utilization of those GPUs was very sporadic. Then the the second half of that slide was, and I think Jensen's got one that's very similar, but it's just a solid strain of workforce or workload, if you will, going across the GPU infrastructure. What's driving that? Of course, the data behind it, but ultimately the agentic tokenomics that come into play and the amount of agents that are actually leveraging the LLMs to go do the work for us, I think that's where it becomes a real interesting TCO play for any organization that's trying to be optimized in this space, right? I think there's right ways to do things with your tokens and there's ways that can be less efficient. And uh we as technology providers need to help customers navigate those uh conversations in a way that is you know secure, but also uh reliable and and in a way that they're not wasting money where they shouldn't be.

SPEAKER_02 2:45

So yeah, Jason, maybe build on that a little bit. I think for a long time we've thought about GPU utilization as more of a technical metric, but it's really starting to creep, if not if it's already there, into a real business center type of conversation.

SPEAKER_01 2:58

Yeah, I think so. I think so. A couple points I'll I'll say there. One is, you know, to Jeff's point, we are seeing this just now always-on agents requiring always-on infrastructure. And so GPU utilization is critical to that. But also these agents need context, they need data. And I think to your your point, I mean they need you know AI ready data, and that often can be the bottleneck because you these agents need access to secure, reliable, recent and accurate data, and that's the biggest challenge. But I so I I think that's one metric. I think the other metric is just looking at kind of the the token costs and your what we call token economics, and actually looking at what is your cost of delivering those tokens. How do you deliver the lowest possible cost per token? That is becoming the important most important metric. And obviously, you know, data can be a bottleneck there, though.

SPEAKER_02 3:53

Yeah, Mike,

Your Data Matters More Than Your GPUs

SPEAKER_02 3:54

I mean, uh build on that as well. I mean, what are you seeing in terms of data readiness? What's the maturity level of the data estate from the organizations that you're interacting with?

SPEAKER_03 4:02

Yeah, now obviously we've got a bit of a bias because you know, from a NetApp perspective, that's what we focus on is the data. But I think you're really seeing that when you move into the enterprise, out of the university, out of the research facility, where hey, sure, make a copy, make 17 copies, make five times the metadata on it because we're just trying to understand what goes on, translates to real cost when you're in an enterprise. And now you've also got the scattered data everywhere. Like I've got devices that are making data in the cloud, I've got devices that are making data at an endpoint, and I'm creating stuff within my own data center, but I need to bring this all together. And so you're beginning to understand that the criticality of the data, but the weight of that data means that maybe we look at the problem a little differently. How can I bring AI to the data as opposed to necessarily moving all of that data to the AI? And then, furthermore, part of those token economics. What can I do to optimize what I have? Do I really need to copy things out seven times? Can I use the data in place? I'm creating it. Why make another copy of it? But how do I do that responsibly, securely, being able to make sure that I can prove those models later in the future? All of those elements now become into that rich data management that goes alongside of it.

SPEAKER_02 5:27

Jeff, you're interacting with a lot of um organizations on a on a regular basis. Is this a shift that they're embracing and ready for, or is there still a lot of learning to be had here?

SPEAKER_00 5:37

Yeah, I think it's a little bit of both, Brian. I you we had a customer in the other day, and it was really interesting to see some of the workloads that they were bringing to market. They had over 300 agents deployed doing autonomous jobs specifically. And the the thing that CTO said to me in the room was really kind of interesting. He said, I hope everybody works to get their data perfect, because by the time that they do that, I will be so far ahead of them, I will blow all my competition out of the water. And what I mean, or what he meant by that, and what the way I understood it, is you don't have to do everything perfectly. But Mike, you bring up a good point. You bring the data to the AI, focus on small use case, small wins, small agentic capabilities with small language models with the right data set, and you can get a lot further ahead a lot quicker than trying to fix the entire estate right out of the gate with a large ETL process and traditional data management techniques. I think there's a lot of ways to do that. How do you bring the data to the AI? What's the roadmap there?

SPEAKER_03 6:31

It is kind of leveraging that data richness and data management features, you know. Number one, what is it that I have or data? You know, I think that becomes a huge challenge of for years. We've collected stuff without knowing why we collect it, and we've held it for an extended time period without knowing why we hold it. So, what is it that I actually have? And then what is going to bring value to the question I'm trying to answer? You know, ultimately AI is not just magically throw a pile and get a question answered that I didn't even know I was going to ask. I've got to have intentionality. So it's being able to understand that data, classify it. So now I've got manageable sets, not only in terms of how large, but what is it covering, what information am I adding into that? So now I can go get an answer that is very predictable and the desired outcome. And then that also helps with the next step because things constantly change. So I need to be able to keep it up to date. And again, if I don't know what I have and where it's at and why I'm using it, much more challenging to be able to ensure that consistently result moving

The Gold Mine Hiding in Your Data

SPEAKER_03 7:40

forward.

SPEAKER_01 7:40

Well, I think one of the challenges is that so much of this data is unstructured. I mean, I think you know, roughly like 89% of the enterprise data is is unstructured. I mean, there's in in you know, PDF docs and PowerPoint files and in videos and audio files and images. And that data is really challenging to get that AI ready. So I think as you look at, you know, one of the bottlenecks in in data is dealing with that unstructured data. But that can be a treasure trove of information and a gold mine for you know a Gentec AI. Is these agents get contacts and get access to that data if it's AI ready? I mean, you can drive greater business innovation and really deliver value through Agentec AI. But I think again, that they're making that AI, that data AI ready and bringing the AI closer to that data, which largely in many cases is on-prem. I mean, I think that's one of the things we're we're beginning to see more focus on in solving problems with partners like NetApp is getting that data AI ready, particularly that unstructured data and that data that's on-prem. Because I think that's where we're seeing now, you know, a shift towards in and the enterprise, more focus towards driving inference on-prem, driving, you know, building AI factories on-prem, leveraging, you know, the AI data platform reference design from NVIDIA, working with partners like NetApp to take that to market, to solve those problems, help make that AI, that data AI ready, getting access securely to that data on-prem and making it available to agents for context.

SPEAKER_00 9:09

Yeah, I love I love what NVIDIA is doing with the AI data platform framework that you guys have put together. And we've had the luxury of working with Mike and his team on the solutions that they're building in our advanced technology center and our AI proving ground. And, you know, just love to see the partnership where NVIDIA meets NetApp and the work that we're doing to help innovate faster with the software suite and the things that you guys bring to the table with NetApp's, you know, core functional technology backbone with when it comes to unstructured data. Lots of lots of interesting things that we're able to solve together there.

SPEAKER_03 9:42

Yeah, that builds on, you know, ultimately, that's what a customer wants. Yes. Nobody wants to build their own database and figure out how you relate to various data tables. They want an outcome that gives them a report to see trending. Now, AI is fantastic in that the world is your oyster, but sometimes that's just overwhelming and daunting. Where do you even begin? So, again, that's where having key partners like WOD to help come in and tell those customers look, here's the infrastructure, and it's harpice designed to make sure that that data gets to the software stack so the software stack can act upon it and ultimately give you an answer that helps move business forward. Because that's really where they want to be.

SPEAKER_02 10:26

So the data challenge is clear: scattered, unstructured, underclassified, and multiplying faster than most enterprises can manage it. The question now is where you put the AI and whether moving it closer to the data changes the economics. That tension is where this conversation goes next.

Bring AI to the Data

SPEAKER_02 10:43

As AI workloads become distributed as we're as customers are talking about on-prem versus cloud, are we just entering a world where it's just always more of a hybrid conversation, or what types of conversations are you having right now in terms of uh workload plates?

SPEAKER_00 10:55

It's definitely an and conversation. It's not an either or. It's it's we're definitely seeing customers actually just within the last, I would say, three months to six months. It's been a lot about, okay, we've been running this in you know, in the public cloud. Costs are starting to go up because with the adoptions getting a little higher. We want to look at some different models and operations that we want to maybe potentially do this on-prem. So we, of course, can help navigate a cloud conversation, a hybrid conversation, or a fully on-prem. And it's really just kind of meeting the client where they're at. We call that our practical approach. We look, we use the AI proving ground as an underpinning to help kind of learn what it looks like to do that on-prem. Um, and those conversations really are ramping. It's like over the last three or four months, a lot of TCO conversations have come up. It's it's like, all right, we're getting this bill now, it's costing this much. What are some options for us to be able to look at maybe building this out on a an AI factory with Cisco and NVIDIA on-prem to really drive that tokenomic tokenomics or token economics down to a point where we're not getting so such a high bill on a monthly basis?

SPEAKER_03 11:58

And I think AI as a as a workflow really does lend itself to that hybrid approach because I can work in the cloud to kind of understand what's the art of the possible, what degree do I need to continue to clean my data up to get a result and what framework am I going to use to be most successful and then scale that up in my data center where I have that massive collection of data. But the model doesn't do me any good unless I can get it to the edge where it's going to be utilized. So you can imagine a retailer who creates something that allows them to get better utilization or better sales out of their store, but still capture those anomalies to further refine the model while at the same time they're working as what is that next thing we can do in the cloud? And that ability to seamlessly move back and forth through where it is best to achieve each one of those outcomes, but do it in a seamless manner just really does make a powerful story.

SPEAKER_00 12:51

Yeah.

Why Smaller AI Models Win

SPEAKER_00 12:52

Mike, you mentioned something a minute ago that kind of resonated with me on, you know, like the bringing the AI to the data in an edge use case for like retail. What we've seen, you know, and what I like about kind of the way Nvidia goes about this from a software perspective is they build these blueprints. They build things that uh allow customers to act quickly with their software. Um and we've had the uh AI data flywheel blueprint, which is a fine-tuning mechanism running in our AI proving ground. And just to show you how you can take a 70 billion parameter model, fine-tune it down to a billion parameter, and run it on an edge device with the right data at the right time. Um, these are the things that resonate with customers really. And you know, we we kind of like to show the art of the possible in the proving ground. And I think that's something that we really appreciate that NVIDIA is doing, packaging these types of solutions, and we try to make sure we showcase and bring those to life in the lab to show the art of the possible. Yep. Yeah.

SPEAKER_02 13:46

Is the outcome there just efficiency, management, uh just a better grip on things?

SPEAKER_00 13:50

Yeah, I think I think, Bill, that exactly. I mean, we did the Nemo Claw hackathon a few weeks ago. We worked with the NVIDIA on that. We learned how that works with OpenClaw and Open Shell and the security uh around that. And yeah, I mean, packaging the things up that NVIDIA is doing, it really is about an efficiency play. It's about performance and scalability and doing it the right way when you're actually in the production, when you're in production with the AI use cases, it's very important to have a supportable, referenceable type of an archite architecture to really design behind. And NVIDIA brings a lot of that with partners like NetApp and the AIDP type of a platform there.

SPEAKER_03 14:24

And the other side of that comes in the operation paradigm. You know, the gap of staffing ability within IT is just constantly growing, and AI and security materials where it's just a massive gap. So when you can build that type of consistency in terms of operation through the entire stack, now I don't need as many individuals to keep it running, whether it is in the data center, in the cloud, or at the edge. So Cisco unveiled their unified edge device a few months back. We're unveiling some designs around that that fit into the flex pod architecture, which means that the operating paradigms at the physical hardware layer, whether it is 50 racks in a data center or a little half height or half-width box that sits in a retailer, remains consistent. And then when you layer on top of that an AI infrastructure from a software perspective, now again, I've got that same data science team that I don't need to break up and say, you're gonna focus on our inferencing at the edge, and you're gonna focus on our art of the possible. No, you get to utilize that team for where the business needs to go while maintaining that consistency of operation.

SPEAKER_02 15:36

So the hybrid model is real. Some in the cloud, some on-prem, some at the edge. The architecture is taking shape, but there's a layer that cuts across all of it that the industry keeps under waiting. When

Don't Build AI Without This

SPEAKER_02 15:47

agentix systems act autonomously on enterprise data, security, and governance aren't afterthoughts. They're the foundation you build from day one, or the problem you go back and fix after it's too late. If we're talking about bringing data to the AI, what what do what do organizations need to know about their data before they can ingest it in, make sure it's secure, make sure it's has the appropriate context, things like that.

SPEAKER_03 16:12

Yeah, I mean there's a lot of pieces that go into that. You mentioned a lot of the unstructured side of things. And then just think about your own use case. Like your home folder where you prepared your notes even for this, how many versions of that do you have? Do I need all of those versions? So, and you mentioned, you know, the the CTO's saying, hey, I need to train my people into creating perfect data. So it's that combination of where do I have what? Getting folks into that mindset of, yeah, maybe when you do have your final presentation, we tweak the procedures a little bit that it gets saved off somewhere else. So that now I know that most of the stuff that lands in this bucket should already be clean, final, prepped. Because you know, that's your worst case scenario. All my blueprints that didn't make it to market are now fed into the engine and become part of the story. And that's not what we want. So there is a little bit of a mindset, I think, as well, in terms of being very prescriptive in your procedures. And the end goal being that yes, that little bit of work up front all of a sudden now creates a much better model on the back end, much better results on the back end. I mean, you think about when you had a login 10 years ago. Well, pick the traffic like fixtures. Well, how many of those were actually to check, and how much of those were using free labor to help classify and tag images? Right? So that same mentality of hey, I get my workforce in this mindset of how do you make sure that your good data that you're happy with gets put somewhere special or somewhere different, so that when I come through to collect and pick it up to go feed it into this AI framework, I've got most of the classification already done out of the gate.

SPEAKER_02 17:55

Jeff, is what he's describing, is that kind of the fundamentals that start to churn into that data flywheel? And if so, how do you build on top of that to start to pick up more momentum?

SPEAKER_00 18:05

It's really about what data you're using and how you actually go about bringing that into a way that you can use it to differentiate yourself. And I think it kind of goes in, yeah, it goes very much in line with the data flywheel for you know fine-tuning a model. But in you once you fine-tune that model, what you do with it at the edge, if it's a small language model that you're using it in that use case, definitely is based on how you train the model and what data is feeding it. So, you know, we have a lot of projects we're doing internally where we're we're feeding a lot of different sources, whether it's in our AI proving ground with various open source LLMs or we're using frontier models, we're doing all of this stuff and you know, bringing all of our internal data. And I'm sure NetEv and NVIDIA are doing the exact same thing for your differentiation, whether you're doing next generation AI native engineering coding assistant stuff, or you're doing you know, traditional line of business advancement stuff where you're becoming more efficient. We're no different as an organization as we're getting our data prepared for our differentiation, which could be comparing contrasting technology, it could be it's it's so many different things that we do, but it's you know, the LLM is only as good as what you feed it, and and it only can advance your organization if it's been prepared the right way to Mike's point. So yeah.

SPEAKER_01 19:18

Yeah, I one thing I'd add to that is just the importance of security as well. So it's data preparation and then securing the data as well. So I think, and that's one of the things, particularly with agentic AI, we're you know, we're seeing and and we focus on this a lot as NVIDIA and releasing the our secure runtime with Open Shell and driving innovation in that area and and the ecosystem adoption for OpenShell has now been phenomenal. But but that is really solving that problem of making sure you have the right security and governance for your autonomous agents in the enterprise and then getting that secure access to the data. So I think data security is you know as big you know one of the big challenges here as well.

SPEAKER_03 19:56

And the security piece you bring up a really good point is we're actually adding a new aspect to security. Because when you say security from an agent perspective, you're thinking guardrails, you're thinking making sure it's only accessing what you want so you get good information. But there still is the other side. I just was a hacker's best friend. I took all my critical data, put it in one location for them. So that physical security and that traditional security needs to be married in with that as well, right? What am I doing to ensure that that data is encrypted or ensure that that data is got just the basic role-based access control around it, or even just segmentation and making sure that that gets brought up to that application level where now, yeah, I'm hooding the guardrails around it, I'm defining who's allowed to state what data the agent can look at, what data it can't look at, and then how do you archive all that? Because, you know, worst case scenario, somebody gets able to manipulate an agent and buy a car for $2. I need to be able to go prove in a court of law that there was no malice, no intent there, and here's the model, here's the data that went into that. So it does bring some additional layers, and they all need to be taken into consideration across that whole conversation.

SPEAKER_00 21:08

Yeah, Jeff, I mean, what does that mean from an identity and access? When I think about identity access, I think about secure multi-tenancy, I think about the protecting the data. There's so many aspects of that, right? I think when we look at how we approach it, when we are talking about an AI framework when it comes to security, we talk about um our AI readiness model for operational resilience, which covers the entire stack that from infrastructure to application.

SPEAKER_02 21:32

The stack is complex, the pace of change is accelerating, and most organizations are somewhere in the middle trying to move fast enough to stay relevant without breaking things that they can't afford

Where Leaders Should Invest First

SPEAKER_02 21:43

to break. So the question isn't whether to invest, it's what to prioritize. And these three have seen enough deployments to have a clear answer. As we approach the end of this conversation here, what are some of the priorities that leaders need to be putting into place now from a data perspective, from NetApp's perspective, to make sure that they're ready? To handle that disruption because it's going to come.

SPEAKER_03 22:02

Intentionality is a big one in terms of at the end of the day, if you this AI model, agent inferencing, et cetera, has some impact on PL, then it needs to get the same treatment as whatever database you're using or whatever customer system you're using that also has impact on your PL. So rather than you know, this race to let me get a result, just that little bit of pause to say we've got governance processes in place already. We have security processes in place already. Let's not bypass them in the rush to come have an outcome. Because if we are successful in the AI deployment that we're putting forth, we want it to stay in production. We don't want to have to go, all right, now that it works, let's rip it all down and make sure we put the security in from day one. Let's make sure we put in the you know governance of access and the backup and the resiliency and all the other pieces that we put alongside our current enterprise applications. That same design intent needs to go into the AI. And you know, this really where we help try and help customers understand absolutely, we'll get you there. But you know, let's make sure that we've are building properly. Just because it's the shiny new thing doesn't mean at the end of the day, it should shortcut the processes you've already put in place.

SPEAKER_01 23:24

I think the you know data should not be an afterthought, storage should not be an afterthought. Storage is critical and central to you know your agentic AI strategy. And so I think you know, organizations need to be thinking about how do you make sure you make that a that data AI ready for your agents because today they're not, you know, it's it you know, agents don't have secure, reliable access to that information. But again, that that is the source of differentiation for the enterprise. I mean, that is a goldmine for how enterprises are gonna take advantage of this agentic AI era and differentiate and drive innovation is giving them access to that data. If they don't, if that data is not AI ready, if if the AI can't access that data, they're not gonna drive that innovation.

SPEAKER_00 24:08

You know, it's lots of excitement with the agentic frameworks, the agentic harnesses, the things that are coming out. The security of of this is critical and important. I think seeing the innovation that Cisco's bringing to market with some of the uh the uh you know new tools that they have, the uh the things they announced with Cloud Center today. It was our cloud control center, I should say. Really cool. That's an entire agentic framework that they're building to manage your infrastructure, right? So that's gonna have MC connect MCP connections into all I think there was 52 he talked about, G2 talked about today, starting. I I think those are the things that we need to be mindful of as we

The One Thing to Remember

SPEAKER_00 24:44

build these things.

SPEAKER_02 24:44

Jeff, Mike, Jason, thank you so much for taking time uh here at a busy Cisco Live, a busy industry event as always, uh, to sit in on the AI Proving Ground podcast. Look forward to having you again on soon and extending the conversation. Thanks, Brian. Great. Thanks for having me. Good to be with you guys. Great to be here. Okay, thanks to Jason, Jeff, and Mike for joining. The lesson here is not that every organization needs perfect data before it can begin, it's that AI must be approached with intention. Bring the AI closer to the right data, build security, governance, and resiliency into the architecture from day one, and treat every successful agent like what it may eventually become: a production system capable of affecting the business. This episode of the AI Proving Ground Podcast was co-produced by Nas Baker, Kara Kuhn, and Sarah Chiadini. Our audio and video engineers were John Novlock and Brian Gagliano. My name is Brian Pelt. Thanks for listening, and see you next time.