Your GPUs Are Waiting for Something

Buying GPUs is easy. Building an AI system that keeps them busy is something else entirely.

Recorded live at Cisco Live, this conversation with Cisco's Will Eatherton and WWT's Eric Fairfield explores what happens after the hardware arrives. Networking, storage, orchestration and security rarely get the same attention as GPUs, yet they're often the difference between an AI investment that scales and one that sits idle. The organizations that get the most from AI won't be the ones with the most compute. They'll be the ones that build the best systems around it.

More about this week's guests: 

Eric Fairfield helps organizations design the networking foundations that power modern AI infrastructure. With deep expertise in Cisco ACI, VMware NSX and high-performance data center networking, he works with enterprises to solve the complex architecture challenges that determine whether AI environments perform at scale.

Will Eatherton leads Cisco's Data Center, Internet and Cloud Infrastructure Engineering team, helping shape the networking architectures behind enterprise AI. From high-performance Ethernet fabrics to AI infrastructure at scale, his work focuses on the systems that make modern AI possible.

The AI Proving Ground Podcast leverages the deep AI technical and business expertise from within World Wide Technology's one-of-a-kind AI Proving Ground, which provides unrivaled access to the world's leading AI technologies. This unique lab environment accelerates your ability to learn about, test, train and implement AI solutions. 

Learn more about WWT's AI Proving Ground.

The AI Proving Ground is a composable lab environment that features the latest high-performance infrastructure and reference architectures from the world's leading AI companies, such as NVIDIA, Cisco, Dell, F5, AMD, Intel and others.

Developed within our Advanced Technology Center (ATC), this one-of-a-kind lab environment empowers IT teams to evaluate and test AI infrastructure, software and solutions for efficacy, scalability and flexibility — all under one roof. The AI Proving Ground provides visibility into data flows across the entire development pipeline, enabling more informed decision-making while safeguarding production environments. 

Transcript

What The Bear Gets Right About AI

SPEAKER_02 0:00

I've been re-watching the popular TV show The Bear lately. For those unfamiliar, it's a show about a world-class chef trying to run a chaotic, understaffed restaurant where everything is constantly on the verge of falling apart. There's elite talent, elite equipment, and an impossible ambition, but the tickets are piling up, the stations are out of sync, and the path between the kitchen and the dining room can never work fast enough. The kitchen isn't failing because the chef can't cook, it's failing because the operation can't keep pace. And that's a useful way for me at least to think about where enterprise AI infrastructure is right now, because the headline investment is easy to see. The GPUs, the racks and power, the big capital decisions. Those are all visible signs that an organization is serious about AI. But GPU clusters are a living-moving system in which data has to get to the right place at the right time. Storage has to keep up, jobs have to move across the environment. Security has to follow the workload. The network has to coordinate all of it without becoming the thing that slows everything down. And that's where the economics can get uncomfortable. $100 million in GPUs can sit at 30% utilization, all because the infrastructure around them cannot keep pace. So in this episode, which we recorded live while at Cisco Live, we're talking about what it really takes to build the fabric for AI at scale. My guests, Cisco's Will Eatherton and WWT's Eric Fairfield, help us move password the obvious GPU conversation and into the harder architecture questions. Like how do you design a network that can keep up with AI workloads? How does storage, security, orchestration, and policy enforcement fit into the same operating model? And why does the network increasingly look less like a box and more like the management plane for the entire AI factory? The lesson here is simple but expensive to ignore. The kitchen can have the best chef in the world, but if everything is out of sync, the food doesn't move. In AI, performance does not come from the most powerful component in the system. It comes from the system itself. This is the AI Proving Ground podcast from Worldwide Technology. Let's jump in.

Your GPUs Are Waiting

SPEAKER_02 2:17

What are we missing when we only think about GPUs from an AI infrastructure perspective?

SPEAKER_01 2:22

Yeah, I think there's been a lot of talk lately about refocusing metrics from things like, you know, if you do a you know NVIDIA SMI report, you might see what is my temperature, I see my memory utilization, I see a GPU utilization. That GPU utilization is a very rough metric that just is is my GPU on effectively, or is it off, or what percent? The new metrics now get into like what your model flops or something. What is the actual utilization of the whole GPU from a cave uh versus its total capability? And I think this is where, you know, I think it's uh recent studies, 50-70% of the time a GPU might be waiting for I/O. And IO, obviously, there's aspects relative to memory movement within a box. Uh, but if we we we generalize that a little bit, the network is key to keeping these GPUs fed. Without them, the GPU the utilization will be lower. And given given the cost of these GPUs, you don't want you don't want to you over over optimize on the network and then see that your actual utilization on flops is uh 10, 30 percent kind of thing.

SPEAKER_00 3:26

Yeah, I see one one of the challenges is I I hate to say, but storage is often forgotten about. Yeah, it's it's the last minute, oh, we we gotta worry about our storage performance. What are we doing around that? And the networking that really needs to go into that because you can starve your GPUs if the we're not getting the data there, right? So the storage network that it's been a key element in both the Cisco and NVIDIA are uh reference architectures that you put together together is being able to design the storage properly to get the data to the GPUs. So we're not starved and waiting for that data to arrive.

SPEAKER_01 4:03

And as part of that, one of the interesting things on the high end, like Quantz and some of these, when I've talked with them, is that the front end the the rule of thumb we've all had is 10 to one. You know, I might have 10 times more scale up a scale out bandwidth than front end, as as exactly due to things like storage, not only for the data, but things like KV cash offload types of use cases, as well as as we get into a gen tech more server to server. I'm I'm hearing more and more about a move towards a ratio of four to one target of front end bandwidth. And I mean, we've seen this on the you know 100 gig on front end, uh you know, going going to 100 gig on front end where we might have been at, and then the number of NICs on back end. I think we're seeing the speed, the port speeds going up and the number of NICs dedicated increasing on front end. So the the front end is incredibly important to your overall application performance.

SPEAKER_00 4:55

Oh, absolutely. I mean, you know, I I look at the NCP reference architectures that you know that I run into with the Neil clouds all the time, right? There, those are designed to be basically a non-blocking up to a certain point for the storage. And then we we deal with the oversubscription then on the compute side. But again, those numbers are significantly higher than anything we've done in the past. You mentioned 100 gig, right? Today, it's you know 800 gig coming into the box, right? You know, whether it's using four 200 gig connections or two, 400 gig, it's still the bandwidth numbers for storage, is it's incredible.

SPEAKER_01 5:32

And it's not just bandwidth. I found it interesting to see that some of the capabilities that have been really developed on the back end RDMA and large elephant flows. We're seeing more of that on the front end, which by the way, coming back, I love silicon. I was an ASIC designer in my career, but we're seeing that uh customers are getting more interested in capabilities like to be able to mix, you know, pack packet spray with elephant flow management. So some of the conventional front-end load balancing with some of the things we do in back end, they might need both of those on the front end network.

SPEAKER_00 6:02

Which which, you know, that that's a great example of how your relationship with NVIDIA, right, and building in the capabilities to support adaptive routing, yeah, right. Which you utilizes the packets brain capabilities on your Nexus platform. Yeah. That it's it helps

The Part Everyone Forgets

SPEAKER_00 6:20

that problem significantly. Yeah.

SPEAKER_02 6:23

So I mean, Will, it based on what you guys were just talking about, fair to say the network is becoming a a true enabler and operate or uh a utilization layer. I mean, really the difference between realizing your GPU investments and ROI and not. That's kind of the delta here.

SPEAKER_01 6:37

Yeah, I I think absolutely. Well, you could put it positive, it's a key enabler, or you can put it negative. Right. Yeah. You don't want to have the network be the reason that you have a hundred million dollars worth of GPUs sitting 30% utilized. So we're we're absolutely seeing the the network as a key part of these architecture conversations from the very beginning.

SPEAKER_02 6:55

And Eric, I mean, you're interfacing with with customers frequently. Are they aware of are they in the right mindset for that shift, or is that something that there's a little bit of an education curve of, hey, this isn't, you know, we still want to talk about and know about the the innovation that's happening on the front end, but don't forget about the the fundamentals here that are absolutely paramount to running this.

SPEAKER_00 7:15

Yeah, that there's a lot of times where they they still think of it from a traditional enterprise networking perspective, and it is totally different because they're not used to the concept of building non-blocking architectures if we need multi-node inferencing or full training environments. You know, it it's not your traditional just spine leaf network. It's it it is about how much bandwidth and IO we can deliver to keep the GPUs fed, right? So th that's just a different world that that we have to design around. I'll use an example in in some of these large deployments. You have to think about it as a system delivering a certain amount of performance. So our our traditional concept of redundancy, yeah, right, is totally different because we don't look at it from a host perspective and design the network of are we worried about losing this uplink or that uplink? It's it's dealt on a much larger scale where because we're designing these to take the hit of a host loss or right in a neocloud, a rack loss, right? We're not we're not designing about the nuances of a little failure here and there, it's about looking at from in a system-wide performance. Yeah.

SPEAKER_01 8:28

No, I I I think your point is a key one that it's it can be quite a bit different. Failure handling both in the upfront architecture as well as how you plan for failure handling as far as what the impacts are going to be to you, because it will happen to you even. So, so and as an example, when we do, and and one of the things wanna to mention is the importance of doing heavy uh benchmarking and validation on these architectures ahead of time, which is why the reference architectures are so important. When we've done experiments, for example, we see that at a couple hundred GPUs, just using some of the fabric-based load balancing techniques, you know, the ones, and we have a range of them uh that are better than ECMP, but using those, you can get very good utilization even with failures. As you're starting to get into like the thousand GPU and then you start throwing in failures, that that like the packet spraying and adaptive routing stuff really starts becoming important. And so I think this is where, you know, it both the the f following the reference architectures is is important, but also looking at that benchmark data and not just the sunny side performance,

The Race to Feed the GPUs

SPEAKER_01 9:28

but but what happens when there is a failure, what does that mean? Is is really important.

SPEAKER_02 9:32

Well, let's get a little bit into the the two of you alluded already to Ethernet and FiniBand. I mean, this has been a conversation unfolding over the the course of several years now. Where do where do we stand with with that? I'm just curious.

SPEAKER_00 9:42

Well, I I think you know, when we look at where people are going in in technology and where people are familiar, there's more network people that are familiar with Ethernet than InfiniBand, right? Now, when we look at the the differences operationally, if you're more familiar with Ethernet, I can go in and I can you know do packet decodes, things like that, and I can troubleshoot much easier than a technology like InfiniBand. The other thing is when you look at the performance over the years, everyone said InfiniBand here and Ethernet here. Well, I'm sorry, but it has equalized, right? If you look at NVIDIA over the course of the last two years, almost every reference architecture that's come out has come out Ethernet first, right? If you look at at where things have changed, you know, we're going in Ethernet dual plane, quad plane with Vera Rubin and Octalplane, right? It's brought the ability to for us to look at this from a multifabric perspective, right? In InfiniBand, if you have a significant problem, you have one fabric that you are dealing with. Now I can load balance across multiple fabrics, and I could literally accept a bigger failure than I have in the past. So again, I I see so many uh developments from an Ethernet perspective. You know, you you mentioned you know some of the load balancing capabilities that you've done to improve over ECMP with DLB, package rate, right? We have options. I wrote a great article about that. It was a lot of fun, you know, learn about your your new capabilities there. And again, ECN, POC, that kind of technology. We've taken that that advantage that InfiniBand had and put it to rest.

SPEAKER_01 11:33

I mean, I I would very much agree with Eric's points. I think it is mostly a done story at this point. We still see customers who have had many years of experience with InfiniBand and they need to get comfortable when they've been using it in the high performance computing area. And so, you know, what when I think about even the the high-end scale, you know, any of the the customers who've gotten into the 100,000 GPS, that's almost all been Ethernet based. So I think it's certainly there. I will say, like, what's what it what can we learn? What has InfiniBand done well? I mean, I think we were talking about multi-tenancy. There was some features in there for more dynamic multi-tenancy, which given some of the new capabilities we're adding into uh Nexus for that, I think we're able to again start to bring that capability forward. I also think operationally, while I would still argue that Ethernet has always been the king, king for troubleshooting and diagnostic, there were characteristics of InfiniBand where it's had opinionation and in how you would you know develop given deployment. And I think that, you know, Ethernet is uh so flexible that that is both a pro and a con. So I think we have tried to do more of being more prescriptive in cases with not only the reference architectures, but also uh providing options that if a customer, um, Nexus Hyperfabric is an example, is a very prescriptive solution that is trying to provide, I would almost go in some cases an InfiniBand like experience on

Don't Design for Perfect Days

SPEAKER_01 12:55

top of an Ethernet as far as how a controller managed can do, as far as like, for instance, completely managing an underlay of a network, which without getting too geeky, is something folks can spend a lot of hours with spreadsheets of IP addresses, and we're trying to simplify it. So operationally, how do we make sure that Ethernet has is has nothing nothing to below infinite band? I think that's been a big focus of the whole industry and of Cisco.

SPEAKER_02 13:19

I mean, when it comes to the operating model underneath AI infrastructure, you know, how should we think about openness, simplicity, control? What are some of the the guiding principles that that maybe yourself and your teams are you know, how how would you advise our clients?

SPEAKER_01 13:33

So I I mean, so open architectures and and again, this is I you know, I multi-vendor is one way I'd put it. So certainly you can say the broader, we have things like UEC, which are our great standards. I think you know, we've approached it in the context of you know, Ethernet is a base, obviously. There's things like the NVIDIA ecosystem and Spectrum X, which obviously is an NVIDIA set of uh protocols and standards. But now with Cisco participating in that, there is a multi multi-vendor angle to that and and you know peace of mind as far as how you might procure and how put together those pieces. So I think there's the the multi-vendor aspects, there's the the base open. Um, there's also areas, for instance, in the hyperscale and in some neo clouds, we see a lot of interest in in technologies like Sonic. And so that's something that from a Cisco standpoint, we put a lot of work into. We're depending on the the month we're between the number three and number four contributor to Sonic. Uh, we've worked to offer that not only with our hyperscale customers, but also offer that if there are neo cloud customers that, um, which there are some that are interested in in deploying that. So I think it's very important to have that you know open set of not only protocols and standards, but open software and it really customer choice. If anything, one of the challenges of the AI infrastructure is so many options and choices, and we want to provide them all, but you can you can get yourself into trouble. There's there's so many choices in there. So, what do you think, Eric?

SPEAKER_00 14:55

Yeah, I I think you know, one of the big things that we've seen with with openness, you know, again, is I'll use your relationship with NVIDIA, right? That's changed significantly in the last 18, 24 months, where it they were fairly closed around their reference architectures and really have opened that up. I mean, with your involvement in the Spectrum X architecture, it gives us options now. It previously in the NCP, we we had one choice and it was NVIDIA networking. And now with the 9164E based on the Spectrum 4 ASIC and your upcoming uh 9364 F series, right? That the that's gonna or the 9164 100, yes. Sorry, 9164, you know, that's gonna be based on the Spectrum 6. Yeah, and and again, it gives us the ability to expand and play in areas you know that were more challenging if you had to follow a reference architecture to a T. You mentioned Sonic, you know, they're you know Hyper Fabric uses Sonic, right? It you know, the ability to use that as an operating system across different platforms. You know, NVIDIA they use Sonic as a a potential platform, right? It it it's definitely something that that we're seeing more interest in because it is an open platform, you know, the the ability for Nexus Dashboard to look deeper right into the NVIDIA architecture to give you more information operationally, not just about the network, but how's my fabric performing? How are my jobs performing, right? That openness is is key to the day two operations, Eleanor.

SPEAKER_01 16:37

By the way, I I love Eric, you mentioned on the silicon side that that's something we've been uh flexible about. I've I've been asked a lot, like, hey, Cisco has so much pride, puts so much into Cisco Silicon One. Why would you build a switch, including the Spectrum Silicon from Nvidia? And again, this is really around being able to support our customers uh based on the technology stack that they're looking for. And in that back-end network

Is Ethernet Finally Winning?

SPEAKER_01 17:00

for NCP, that does require the Spectrum ASIC. If a customer is looking for Spectrum technology, but not NCP, we can support that. We can support Cloud Reference with Silicon One. But by having the full gamut, I mean, we're Cisco's no stranger to having a range of silicon options, but by having that, we can now meet our customer where they're looking. And with that said, with silicon one, we have our G Series, which is this really the high-end, high-performance Ethernet in, you know, going 100 T and then it'll keep going. We have the P series, which is scale, you know, deeper packet buffer routing tables. We can use that for scale across for you know essentially routing functionality. And then we have others for modular chassis for our campus. And so that that flexibility both within the silicon one, but then leveraging external, I think is very key. So it's really openness at every you have to think about the silicon level, the protocol level, and then you know the software involved. Yes.

SPEAKER_02 17:51

I mean, like you mentioned just a moment ago, I mean, there's a lot of decisions that go into this. Is the idea that we can all that we can, I know you know Cisco, WWT, and NVIDIA as well, all touting Cisco's secure AI factory, is the idea that it ties together in there to create kind of a more cohesive system?

SPEAKER_01 18:08

So Secure AI Factory, you know, first of all, starts with a lot of what we were taught, Eric and I have talking about, which is the key components of compute and networking put together, validated in a reference architecture that NVIDIA is approving, and that we can support that, whether it's the enterprise reference architecture, it's a Cisco cloud reference architecture that can be generally used, or very specifically the NVIDIA cloud partner NCP. So that is a base to it. Then you take with that, you take a storage partner. And for instance, we've worked a lot jointly with VASP, but there are several other NCP approved that we we've worked with and that we've had great relationships with for many years. So an NCP approved, or if you're looking for the NCP certification, so that is your base infra. Then next you need orchestration, and that comes into it. And then for us, a key part has been you need to have a security aspect of this. And while I think generally folks think it's important, I will say that sometimes with customers, especially new clouds, they're they got funding, they're moving fast. Sometimes security is a second phase of consideration. We want to make sure that that is really upfront. And so this comes into how do you put security in uh cohesively? And that I was mentioning earlier, there's multiple parts to that. There is the the service orchestration. How do you make sure that as you're setting up each tenant that this the security is is properly connected for that? But also, how do you have that policy be consistent across your Kubernetes layer, your network layer, and your actual compute infrastructure? And I think all of those pieces coming together are really important to a security factory, which we think should be the default. Everything should be a security factory, not that should not be a special case. Sure.

More Choice, Less Lock-In

SPEAKER_02 19:42

Here at Cisco Live, I mean certainly Cisco leaning into agents, agentic ops. Curious just what that means for the network for those that may not be familiar.

SPEAKER_01 19:50

Sure. So I think one of the demos that we've been showing at Cisco Live is examples where we use our AI Canvas, which is the a key part of Cisco Cloud Control. And we have examples where, for instance, you can ask fairly general questions. We have, as an example, InnerSight, Hyper Fabric, Nexus Dashboard that plug into this. And so you can ask questions around something that isn't specific to this controller or this box. You can ask something about your network level that might have state that is needed between our firewall FTD, between hyperfabric, inner site. And then the AI canvas is really has the A the API gateway functionality across these controllers and then could start put together. Here is information I've gathered from InnerSite, here's what I've gotten from hyperfabric, and then across that suggesting what can be the issue. Um it's still early technology, but we've been working to really, and we really think this is like bringing multiple domains and controllers together by itself is useful, just from a but it's really functionality around AI Canvas and this agenda model of working across these APIs where you should not have to know which API or I need to go to Nexus dashboard number five out of the 15 I have in order to get this state. And that is the one of the key parts is to make it simpler, faster, uh, and make it a team sport. So, how do I have my network operations team, my security operations team able to hand off uh views of this in a single canvas? And so this is something which you know, I'll say for us in engineering, I has been almost uncomfortable how fast we've been urged to move on this. Uh, so we have all the like the day jobs, like get another switch out, add security, and then we have the night job, which is working across the engineering teams to pull this together. But I will say, as uh engineering at Cisco, I've never seen something where we have had um so much cross engineering and product engagement in order to come out with one delivery. And G2, who's our all right, he was very clear. You know, this is the the top thing. Everything we do around the product and the technology is important, but the top thing we can do is has to have this commonality and then stitch in the agentic behaviors for that. So I'm I I think we've made great progress, but I think the next six months is going to be amazing some of the the follow-ons that keep coming. Yeah. Well, agents never sleep, neither should your don't so we can't sleep either. Yeah.

SPEAKER_00 22:09

Yeah, Eric, build on. I I'm I'm really excited. I I I look at, you know, being old-time ACI guy, right? I I I remember the challenges of trying to integrate ACI to the campus environment, right? And have visibility across multiple domains and how we tried a couple of different ways to do that and the challenges it brought. And now stepping back and looking at instead of trying to have that one tool to do it, we're we're stepping away from it and saying, okay, we are going to have multiple tools. There's no way we can try and have that one all-encompassing tool anymore, which really it caused us to get in our own way, right? And now we're doing a Gentec AI to go out to the different uh areas and get that information. It's gonna be so much easier for us to get that a holistic view, right? And being able to get better information in a much easier fashion than really trying to create this big old monolithic thing that we tried for years and just it it didn't go as well as we had hoped.

SPEAKER_01 23:17

One one addition I like to make, I think the tools coming together is key, but another, and again, just to click down one more level, uh one other key technology that in the data center and now data center campus, we've been seeing more and more important are

Security Isn't a Feature

SPEAKER_01 23:30

these border gateways. And so the border gateway concept, which it's not brain surgery, but it really comes down to is as an object that gets configured on the interface. So for instance, between, and this is part of our Nexus one uh interface that between ACI and an EVP and VXLN fabric, or a Nexus provision EVPN fabric, EVP and VXLN fabric and a campus provision one. Customers aren't necessarily saying, I want one fabric that is going to be stitched across everything I'm ever gonna have, but they do want to have clear policy on the communication. This is around this border gateway concept. And so we've been very happy that's something we've been bringing out and we've been making our default on how does ACI talk to an XS Dash or EVP and VXLAN? How do we bring hyperfabric when it's managing an AI cluster and having that communicate? And then how do we have Catalyst and an EVP and so these border gateways, along with Cisco Cloud Control, along with AI Canvas, like we're finally, after decades, bringing all these pieces together where it is a more cohesive architecture? And that that like that's a lot more than you know, a browser-based commonality.

SPEAKER_02 24:34

So well, I mean, Eric, maybe apply that to the enterprise

AI Starts Running the Network

SPEAKER_02 24:38

here. How how should enterprise leaders be thinking about policy enforcement or identity management as it relates to you know what he's saying here?

SPEAKER_00 24:45

Oh, that's a good question because I typically haven't lived in the enterprise now that I'm in the high performance side, but you know, when when I look at you know how how we're trying to implement policy, right? You know, we've been trying to do it, you know, in the in the wrong methods in in the past, right? Like I said, we were trying to enforce it in ACI into the campus and and really moving it out further to the edge and having a you know, looking at it instead of trying to have the one tool do everything. Let's let's step back and deploy policy closer to the edge and let the edge deal with it instead of trying to at least from my perspective, right? We we started looking, you know, I'll I'll use you know cybersecurity within the AI fabric, right? The network really can't keep up, right? You know, the speeds that we're talking about, you know, if we're transmitting, you know, terabits of data through a switch, the be the best place to enforce it really is the endpoint, right? And and being able to do that with your product sets in the blue fields, for instance, is a great example. I love yeah, right. Where we're gonna be able to enforce that policy, but instead of relying on the network to do it, let's rely on the endpoints and push that performance that they have and utilize its capabilities.

SPEAKER_01 26:18

No, I by the way, I I love the blue field enforcement point. And one of the things I'm very proud to say is within a Cisco context, I have not been in a conversation where anyone said, Hey, if we do that, does that mean that there's not a that there's gonna be a box that we can't sell? It's like this is the good architecture, let's move as fast as we can to make this happen. And then that enforcement point, whether it's you know in the K8s layer, whether it's in the blue field, whether it's in a network switch as part of a smart switch, or whether it's a standalone firewall box. But I mean, I think one of the things that I I hear a lot is the conventional firewall architecture where you had all the ALG. I mean, as many of us have involved years, you don't need all that complexity. Uh, it is a lot of software. And by the way, in the mythos age, having a hundred million lines of firewall code is not an advantage. Every line of code is a potential risk. So slim it down, make it high performance, just hit the stateful aspects of segmentation that you want to have, and then put that in the right locations. That's the architecture the customer is really interested in.

SPEAKER_00 27:16

Yeah, I I see that's where, you know, like your smart switch, right? That's fantastic in the sense that not every I don't have a blue field in my laptop, right? Every server isn't going to have a blue field adapter. But again, we can push that policy a little bit closer to that edge, right? And do that in the switch. You know, people say, Well, we've had ACLs for years. I'm sorry, but that doesn't scale, right? It it and and yeah, we we it may have the performance in no, no, you know, we can do wire eight with ACLs, but that's a bear to manage. Yep. With with the smart switch capabilities, again, we can provide you know stateful inspection and firewall-like capabilities at that level and get it yet closer. So it's really exciting to see where you guys are taking that, even outside of the AI Watt. Yeah.

Move Security Closer to the Workload

SPEAKER_02 28:05

Well, as I mean, as AI workloads become more distributed and autonomous, I mean, and you're talking about, you know, keep it simple. You don't want that 400 lines. Is that the future of what a good network looks like? Is you know simple and sturdy, or how do you see the network evolving even beyond kind of the next six, 12, 18 months?

SPEAKER_01 28:24

I I'm gonna say I think, and I'll try to keep it simple. I think there's forces pushing it to be more complex. Like when I think about what's changing with inference and where that adds even more network interfaces between chips, between systems, you know, the the conventional GPUs and these accelerators. When I think about the NVIDIA world as well as the hyperscale world, which is a whole different set of interfaces and chip, and the non-NVIDIA world, I think there's a lot of forces that actually push you towards complexity. At the same time, I think from a Cisco standpoint, our goal is gonna be that uh as we get in place, you know, common uh models of of management, we want to be bringing in network and compute more cohesively is one key key tenet. We want to be stitching that in at the the the orchestration layer more clearly and really making the software, no matter how complex the network might get, how we present that to either a neo cloud or enterprise as simple as possible. Because I I I um I fear it's gonna get more complex before it gets more simple at a underlying technology standpoint. So I don't know, what do you think?

SPEAKER_00 29:28

So I I see the the location of complexity changing. Okay, right. Now, if we look at the upcoming reference architectures, right, we're gonna see things moving in some of these AI factories towards post-based networking. Yep. Right. In which it it changes the location of some of the complexity. Now we're gonna have the Bluefield adapters potentially acting as a VTEP now. Yep. We're still going to have the fantastic Cisco networks behind that, but we're just going to change how we're pairing with those networks and where we're going to do some of the the different nuances of the networking, right? The the load balancing, right, that we're talking about. We're still going to have some of those tools, but we're going to change what we're using on the the Nick itself, right? We're going to have load balancing moved out there even more.

SPEAKER_01 30:20

So and actually, I'm going to dovetoe over there. I think it's a very good point that networking is not a switch. It's not a switch topic. And we've been very excited to see that from an underlying technology primitive standpoint, the direction Bluefield's gone from Nvidia has been a very good one. And it's not just, hey, there's a lot more ARM core or a lot more cores. That's been great, but it's also underlying things like within the server, the Bluefield NICs, which are really the brainiest of the NICs, now have you know connectivity to those back end NICs. And so you can start to think about it being a management plane within the box. So you have your host software that's running all of the hard work low. You then have this management plane of network software and service software that's running in this blue field across all of the NICs. And then that has to be cohesive with the network. And I think that's why it's important that as a vendor that we're playing across those and we're thinking about lifecycle, for instance, of the software that runs. If I upgrade the software in my network switches and it has a feature that needs software on that blue field, Nick, that all now has to be managed at least with some coordination. They can't be ships in the night. And so the network is not a buy. So I think is that that's a great, great put, great

The Complexity Didn't Disappear

SPEAKER_01 31:30

point, Eric.

SPEAKER_02 31:30

Well, we're coming uh short on time here, but I like uh we'll close out on this. I like what you said about it becoming a management plane, but then just a moment ago, you also said you fear it's going to become more complex before it becomes more simple. Yep. Eric, we'll start with you, and then Willie can close this out after. What are some of the core priorities or principles based on this conversation that IT leaders should be keeping in mind so that they can navigate through that complexity until we get to the simplicity?

SPEAKER_00 31:54

Well, I I think you know, the the the big things right in in this is developing proper architecture, right? Understanding what is our use case with these AI factory networks, because we see a lot of complexity that's created that doesn't necessarily need to be there. You know, Will and I were talking about this earlier is you know, if you're designing uh an AI factory that's driven primarily by inference, right? And not multinodal inference, that changes the complexity of your network significantly, makes it a lot easier because I don't need all the complexity of the back-end network, right? But then again, on the flip side, you have to know that if you're getting into training L LLMs, your network complexity will go up a little bit more, right? Then the switching architecture changes right to a non-blocking architecture, things that they might not be accustomed to. So to me, you know, it's it's the six Ps, right? You you you have to do the the proper planning up front and be ready. And that that's not just networking, that's facilities, right? We we see it all the time, right? People, facilities, you know, they buy all these systems and how the hell do I turn this on? Right. You know, things like that.

SPEAKER_01 33:17

So if I were to leave with one thing, I think I often talk with enterprise customers who are saying they're they're waiting for things to settle down a little bit before they start to put their first on-prem infrastructure of any beyond like a test cluster. And I think it's kind of like investing in the stock market. Like, when's the right time to get in? Yeah. I uh my advice would be first of all, there's a lot of talk about these moving so quickly, you know, Blackwell and then Vera Rubin. The reality is the time is much like there's discussion about 12 months, 15. The reality is these are more like one and a half, two year cycles. So, you know, the the waiting is not necessarily gonna, you know, the the the length of time you'll be at the the front end will be longer than you think. The second part is the time value, and I think we see this in neo clouds that are still running a series, like the time value of how long you'll be able to leverage these is a good number of of years. So, you know, we H H1 or 200 is still very useful, very valuable. We're gonna see that keep going. So, you know, it's even if there is something new, you're you're you're back in. And the other part would be the expertise, not only from an infrastructure standpoint, but from uh building through the application and how you leverage that infrastructure and start bringing down your total costs versus a pure cloud deployment. You know, uh don't wait. I think that would be the key thing. And don't overtly worry about, you know, at 12 months, all new technology is going to replace all this. So that'd be my thing.

SPEAKER_02 34:35

Jump in and get busy, yes. Well, Will, Eric, thank you so much for for taking the time. I know your schedules are are are chaotic right now during Cisco Live. So um I very much appreciate you taking the time. Thank you for the partnership, and uh, we'll have you on again soon. Great,

Don't Wait to Build

SPEAKER_02 34:49

thank you. All right, thanks to Eric and Will for joining the show. The big takeaway here is this AI infrastructure is not a GPU decision, it's a systems decision. The chips matter, but so does the fabric that feeds them, the storage that supports them, the security that protects them, and the operating model that keeps it all moving. As Will and Eric made clear, the network is no longer just something underneath AI. It's what determines whether AI infrastructure performs, scales, and pays off. This episode of the AI Proving Ground Podcast was co-produced by Nas Baker, Kara Kuhn, and Sterra Chiadini. Our audio and video engineers were John Knoblock and Brian Gagliano. My name is Brian Felt. Thanks for listening. See you next time.