Bots, Bursts, And Bare Metal: Because The Internet Wanted Drama

We break down Cloudflare’s outage, why a small config change caused big waves, and what better guardrails could look like. We then unpack AWS and Google’s cross‑cloud link, Megaport’s move into bare metal and GPUs, Webex adding deepfake defenses, and a new startup aiming to tune AI networks at microsecond speed.

• Cloudflare outage root cause and fallout
• Automation guardrails, validation and rollbacks
• AWS–Google cross‑cloud connectivity preview
• Pricing, routing and policy gaps to watch
• Megaport acquires Latitude SH for compute
• Bare metal and GPU as a service near clouds
• Webex integrates deepfake and fraud detection
• Accuracy risks, UX and escalation paths
• Apstra founders launch Aria for AI networks
• Microburst telemetry, closed‑loop control and SLAs

If you enjoyed this please give us some feedback or share this with a friend we would love to hear from you as well and we will see you in two weeks with another episode

Purchase Chris and Tim's book on AWS Cloud Networking: https://www.amazon.com/Certified-Advanced-Networking-Certification-certification/dp/1835080839/

Check out the Monthly Cloud Networking News
https://docs.google.com/document/d/1fkBWCGwXDUX9OfZ9_MvSVup8tJJzJeqrauaE6VPT2b0/

Visit our website and subscribe: https://www.cables2clouds.com/
Follow us on BlueSky: https://bsky.app/profile/cables2clouds.com
Follow us on YouTube: https://www.youtube.com/@cables2clouds/
Follow us on TikTok: https://www.tiktok.com/@cables2clouds
Merch Store: https://store.cables2clouds.com/
Join the Discord Study group: https://artofneteng.com/iaatj

Chapters

• 0:13 Welcome And Format
• 1:01 Cloudflare Outage Overview
• 2:50 Root Cause And Bot Management
• 4:28 Impact, Transparency, And Lessons
• 7:37 AWS–Google Multi‑Cloud Link Announced
• 10:38 How The Cross‑Cloud Wiring Works
• 12:19 Pricing, Routing, And Adoption Questions
• 13:26 Megaport Buys Latitude For Compute
• 15:14 Bare Metal And GPU As A Service
• 17:18 Integrating Compute With Network Automation
• 18:17 Webex Adds Deepfake Detection
• 20:47 Accuracy, False Flags, And UX Risks
• 22:35 Apstra Founders Launch Aria For AI Nets
• 25:04 Microsecond Telemetry And Real‑Time Control
• 27:22 SLAs, NOC Reality, And Operator Burden
• 32:12 Wrap Up And Thank You

Transcript

Chris: 0:13

Hello and welcome back to another episode of the Cables to Clouds monthly news update. Um I uh my name is Chris Miles, I'll be your co-host today, and uh I'm very happy to be back. Um my uh my co-host here, Tim McConaughey, as always. Uh I was out last time they did this and we brought in a uh a ringer, uh a powerhouse uh Steve McNutt to fill in for me, so I have some big shoes to fill this week. But um so if you're not familiar with this format of the show, basically we we do this once a month where we kind of go over the uh recent news that has come out in the last month and and give our um synopsis as well as hot takes or um you know very uh lukewarm opinions about them. But um with that out of the way, let's go ahead and hop in. So uh two weeks ago, we had um kind of just on the heels of a major AWS outage that happened last month. Uh Cloudflare was uh, I don't know if that was we would call it a hold my beer moment or what, but uh a couple weeks ago they uh had a similar issue where um they were, you know, Cloudflare, if you're not familiar with Cloudflare, they basically provide um uh front-end security for probably majority of the websites out there on the internet. Um so I th a lot of the articles that came out about this outage was were kind of explaining uh what Cloudflare was. You know, if you look at the one from like The Guardian or something like that, it was more so explaining who Cloudflare was a company and why they took out all of your um internet connectivity because there was a lot of uh consumer things affected by this, like X and uh OpenAI, uh Canva, etc. So this was obviously a very widespread thing. Now we've we've included some articles in the news uh breakdown that you can see in the show notes. Um, but it it looks like they've they've done a kind of post-mortem report about the outage and showed what caused it. Um and it was actually quite interesting. So uh I wanted to uh get into kind of some of the details of that. So, you know, Cloudflare posted this um uh, like I said, this breakdown of what happened, and you know, they kind of talk about how you know the way traffic is ingested into their network um through, you know, kind of predefined points, um, very uh predictable um from their from their perspective. And then they do all this processing on, you know, kind of the the security features that they offer on top of that, and then eventually the the services are delivered to the end uh kind of customer hosted um resource, whether that be in a cloud provider, on the internet, what have you. Um and uh apparently what had happened was uh there was a change to a a ClickHouse database um on their end. So it was just a very kind of a minor change that they made to this database. Um but the thing was this database was responsible for generating a file that was used by their bot management system. So the bot management system is basically a um a set of criteria that um is inspected on every request that goes through Cloudflare so that they can use that to determine whether or not the request was generated by a human or an automated thing, you know, a bot, potentially a botnet, yeah, you know, kind of DDoS style attack, things like that. Um so that's that configuration file is basically used by the bot management system. Apparently, the a very simple uh error happened where there was some duplicate rows in that database which generated the file, causing the file to be basically twice the size that it normally is, and the software that the bot management system used was predefined to look for a certain size of file, and that cell that file being double the size caused uh tons of um internal service error or internal server errors um to be generated back from uh the HTTP request. So um kind of an interesting uh an interesting one. Um, but it was uh it was cool to read through that, and I think they did a good uh synopsis of it. But um yeah, so if you want to see the breakdown, that's in the show notes. Um and Tim, how did were you affected by this? Did you did you lose millions in this?

Tim: 4:24

Millions, yes, that's true. Every minute I was down, I was losing another million dollars. I mean, I only have the one website and it's served out of uh cloud front, so I in AWS. So when what it means is that when AWS has an outage, I'll I'll I'm sure my website will go down. But uh this time not so much. I mean, I mean Cloudflare's been around forever, right? There the there's CDN, which gives them point of presence and and also the giant pipes required to provide this service as well, right? Like uh there's not there's not a lot of uh providers out there that can do this. But yeah, I mean it's it's essentially an anti-DDOS or a security front end service. And that means that when a request comes into the website, it goes to that, it gets scrubbed more or less through Cloudflare and then forwarded on if it's legitimate. So what what's interesting about this, I think, is um first of all, just I mean, I I'm not surprised that it was very, very wide, like like it affected a lot of websites. There's not that many like I said, there's not that many providers out here that offer this service, right? So if you're of a certain sp uh uh size and you have a certain you know amount of expectation of DDOS or whatever you'd call it, or uh you know, hackers coming in or whatever, you you you don't have a lot of options, right? So so not no surprise there. I do think it's interesting that as usual, it ends up being some form of automation and some kind of configuration file that breaks everything. So this is very normal, right? Obviously, you're not gonna have a person editing this file anyway. So um I am curious. They gave us the root cause, which is good. I would love to know, since this is an automated system, like what is the new process they're putting in place to verify or change or like are they gonna are they gonna make it so the software from now on doesn't care about the size of the config file? Are they gonna do a validation step as part of CI C D to you know what I mean? Like it's that's always what I and they almost never tell you what it is, of course, for for obvious reasons, but I'm curious to see uh what what they do to fix it, like what what uh what they did to fix it, and then what is the the process moving forward. So yeah, good stuff.

Chris: 6:24

Yeah. I would imagine they're probably gonna um if it's something as simple as the configuration file being too big from a size perspective, I imagine they're probably gonna put in something to just either expand that or uh maybe not meet leave that as uh hard hard coded in the software. Um I included uh a piece in here from about this from Network World and and they do make a good call out in there about um how many of these outages that we've seen lately that have been kind of self-inflicted gunshots, right? Um by things like this, like automation internal stuff, as opposed to general, like uh or genuine, I should say, cyber attacks, right? It's funny because like we see all these things happen, and the first thing is like, oh, is this a cyber attack? Is is is Cloudflare getting attacked? Is AWS is is so-and-so vendor getting attacked? Seems like over the last year, 99% of the time, that has not been the case. It's been some type of automation, uh, some type of um user error or what have you. Um so maybe cyber attacks are over. Maybe we fixed it. They're they're not happening, right? Right. Just kidding.

Tim: 7:27

Well, if they'd been using Cloudflare to stop their cyber attacks. Uh no, absolutely true. Um, okay, so moving on. This next one, uh I did I would not have had this on my 2025 2025 bingo card, but probably eventually where this is just gonna be something that people are gonna that that providers are gonna do, which is funny because I remember when I worked for, you know, a multi-cloud networking company, uh, one of the biggest things we would always tell our customers is, you know, hey, don't, you know, uh the CSPs are never gonna solve this problem for you of of connecting clouds together. And then you had Oracle uh doing back-end connectivity to uh Azure, right, recently. Well, not recently, that's been like a year now or more. And now uh AWS and Google Cloud just announced that they're collaborating on a multi-cloud networking solution. So the the solution looks pretty much like anybody who understands hybrid cloud networking would expect it to look, which is that uh both basically both providers are providing a a workflow, a service, if you will, that is gonna use predefined meet me points uh for using uh Google Cloud Interconnect or like AWS uh direct connect. And these are just gonna be you know pre-provisioned uh racks in in meet me points, like uh, you know, Equinix Data Center or Digital Realty or whatever, you know, wherever, wherever those services are currently offered, you're they're just going to pre-build cross-connects. Like imagine you put in the LOA, the uh other letter of authorization authorization to a data center to connect racks together, like you would do in a managed provider. Well, they just did this essentially on their own. And so now they're offering this as a preview in certain uh AWS regions or Google Cloud regions, as just essentially probably just to test the water and see what's the what's the um you know, what's the uptake? What are people interested in doing this? So there's a little bit of a chicken and egg here because it it seems pretty basic, but also like a service, right? So I think they're uh they're putting out the minimum service required on both sides and they're just kind of testing the waters and see if people are want to go want to use a managed service from the CSP directly to connect clouds together, right? Because I mean, right now you could do this with any multi-cloud networking, or you can do this with uh, you know, megaport and equinex both have a a healthy business, you know, right in this space, connecting clouds together with uh with the private circuits. So I think the I think it's a cool offer. I think we're gonna see more of it over time, especially if it gets adopted. I th I really think this is like a little bit more than a toe in the water, but not much more than a basic, let's throw throw it out there and see what people go for if people go for it. Um but yeah, I mean it's it's it's a really good suggestion. Uh it's gonna help uh it's gonna help true multi-cloud applications, I think. Um, or at least I don't know if it'll help them more than you know, multi-cloud networking were would before, but more like it'll help enterprises consider it probably a little bit more because it's kind of a vetted, stamped, you know, seal of approval solution. So I don't know, Chris. What do you think?

Chris: 10:31

Yeah, uh I'm kind of with you in that I think this is probably a bit of a toe in the water. But the thing is, I think it was probably relatively low effort to develop something like this. Um, you know, I mean it's I think there are there are some uh I guess writings on the wall about why AWS is doing this with Google and probably not Azure, because um, you know, the number one cloud provider probably not gonna pair up with number two, but hey, number three, we'll we'll go number three um for that. But nonetheless, like like you like you said, this is like I've seen customers in the wild that actually built this exact thing themselves, right? Um a friend of mine was kind of telling me about how like he ordered a direct connect into this facility and then ordered a cloud interconnect from Google into the same facility and then basically paid the the colo provider to run the cable between the Meet Me room between the two, right? So this is this back-to-back thing already exists um in in certain scenarios where customers are building this. So it makes sense that they just say, hey, let's let's reach an agreement in these few data centers. We'll build some interconnects between our our devices, we'll build a couple of simple APIs to allow for um some automation to bring up a peering. It seems relatively low effort for potentially um um uh easing their uh easing their customers' processes at the end of the day. So I think that's good. One one thing I can't seem to find too much of about this is pricing. Um it's obvious that it does ride on top of, you know, AWS typically uses the concept of the building blocks, right? So everything is kind of built upon something else. Um, you know, a lot of services rely on something like Private Link. This is this is a service that they're offering that relies on Direct Connect, right? So you always terminate this on a Direct Connect gateway. Um, but yeah, I can't I can't tell if this is going to alleviate some of the pricing or potentially increase some of the pricing because it's you know technically a service that they're offering. So I don't know if this is gonna make Direct Connect prices higher than they normally would be in this scenario or not. Um and I don't I don't see a lot of details in the routing uh at this point as well. Like, you know, obviously customers of this scale that are doing things like this sometimes have very complex routing policies. Um, so it's kind of hard to tell what capabilities they'll have here, whether or not this needs to be like a static connection, or if it'll be, you know, uh using something like BGP under the hood to exchange between the two clouds. But like you said, this is this is still early days, so maybe that stuff is yet to come. But yeah, interesting offering. Um I I too uh for the longest time was saying that the CSPs will never do this because they're not they're not operating in your best interest. Um, but you know, our our bingo cards are busted this year, I guess. So that's fine. Still a month to go, too. All right, next up, uh we have um an announcement that came in on the 10th of November from Megaport. So Megaport, um if you're familiar, network as a service provider, um we've had Alexis on the show who's kind of a tech evangelist for them. Um but Megaport, who is is focuses on very um you know robust and uh reliable network connectivity geographic locations and making that very simple um is now acquiring a a new company um called Latitude SH, which is focused on compute as a service. Um so this this is very interesting. I'll say in in my day-to-day, in my day job, I have seen a lot of advancement from Megaport in them changing what they're offering. Um from things like uh I've seen them potentially getting into the managed services game, things like that. But this is kind of takes it to a whole new level, right? Because now they're if we look at this from a broader scope, um, Latitude SH, from what I can tell about them based on their um, you know, what's in this announcement article versus what's on their website, it's you know, they do offer compute as a service, but it seems like there's a very strong emphasis on one bare metal and two GPU as a service, right? Um so typically, you know, offering compute as a service, you can kind of say the cloud providers do this as well with something like EC2, virtual machines, etc. Um, but typically you have to pay extra for the option for bare metal because obviously that removes the kind of complexities of a of the hypervisor um sitting in between you and then the actual compute. So if this is the offering that Megaport can provide, then you know they could potentially eat eat some of the CSP's lunch with with this offering, right? Because they're they're I've seen a lot of customers use them for things like connectivity out to the internet versus doing it in the cloud provider, because obviously the cloud providers charge you um uh per gigabyte as you go out, whereas typically megaport does not do that. So we've seen that be a very strategic thing for customers. Um so now you could start potentially putting the compute there as well, more than just like uh virtualized network appliances like they do with their uh megaport virtual edge platform. So I think this is this is a quite interesting um move from Megaport. Didn't see this coming. I knew they were wanting to uh like I said, change their change their game up a bit, but this is uh this could be this could be something. Um so I don't know how do you feel, Tim?

Tim: 15:46

Yeah. So honestly, when I read it, uh the first thing I was thinking of was uh Equinix Metal, because Equinix Metal has been Equinix has been offering bare metal uh for several years now. Three, three plus or at least. I don't remember how many years now. Um so I was thinking, okay, great, makes perfect sense for Megaport to do the same, essentially. You know, and also, you know, Megaport has their Megaport marketplace where they connect you to third-party services and whatnot. Um I don't know if they had I don't remember. Uh honestly, I didn't look it up. I don't know if they had a partnership like that, a marketplace offer with Latitude before they acquired them or not. I don't know that that would have even played a part in whether or not they acquired them. I do think it's a good move on Megaport's part, right? Megaport basically has up until now offered connectivity as a service to whatever, those third-party marketplaces, the clouds, other megaport facilities that are other facilities that have megaport in them, I should say. Um, so this is literally just tacking more services onto that connectivity. So it's a my point is it's a good fit, right? Like server compute plus network has always been very tightly coupled. And so it makes perfect sense for them to do this. Um, acquisition-wise, I don't know. Like, I don't know, I don't know enough about latitude service to say whether they should have acquired them or part or of partnering. I'm I'm gonna trust Michael Reed on that one. The GPU as a service thing is a very good point, right? It's not it's not neo cloud, like the GOPU thing they're doing, it's not like a neo cloud thing that they're offering, but but they're offering compute as a service plus GPU as a service, um, you know, combined. So it makes sense. Yeah, I think I think you nailed it when you you pointed out that the CSPs often, you know, provide a high level of abstraction, but also automation in front of it. So the question now is gonna be how does Megaport fold in this offer into its uh automated uh offering, you know, like solution essentially. How do you how do I go as a megaport customer? Am I gonna go to the megaport website and can I orchestrate my compute as a service? And I'm sure the answer today is probably not yet, but I what I'd I'd I'd like to know the vision, how they're how they're planning on integrating that. And I'm sure it's something like that, right?

Chris: 17:52

So no, this is good. This is really good. I totally agree. I agree. It's it's gotta fit into that automation ecosystem, right? Because that's what the the CSPs have kind of excelled at for a while. So if this is gonna um participate in that space, it's gotta, it's gotta fit in, right? So that is a good point.

Tim: 18:07

All right. So uh I was I was looking around uh and I found this one, and I'll be honest, this one completely this this uh this uh story completely slips under my radar. I only found it when I went to to go look to see the stuff I might have missed over the last month. So this was yeah, so this is this is bare this barely scrapes in, honestly. Uh so a new feature has been added to to collaboration, uh, to WebEx Collaboration, or will soon actually be added to the WebEx collaboration platform uh from Cisco. And the the collaboration is from a new cybersecurity company or a cybersecurity company called Pindrop, which is a collaboration cybersed company. I didn't know those existed, by the way. Um there's there's a new collaboration plan to bring there. Pindrop has an authentication fraud detection and deep fake detection for collaboration platforms. That's what they do. So this is already kind of like I was like, man, this barely skates in, but at the same time, what a cool idea. Because I several times now on uh you know cables clouds, we've we've reported several, honestly kind of funny uh stories where you know uh deep fake hackers have have gained access to systems by you know interviewing uh for jobs and then getting the job and you know via uh some interview platform where it was obviously not, you know, it was a North Korean hacker or somebody, it was somebody else, right, that was that was doing the interview um or that ended up you know doing the interview and getting through it. Uh so this is kind of interesting. Uh the now it's being, like I said, it's being integrated with Cisco WebEx, and I just really love the idea of doing deep fake detection and fraud detection as part of your collaboration platform. Because you know, by now everybody's seen probably those videos where somebody's in a video uh a video interview and the person, you know, and they're like, uh I think you're like masking something, you know, move your hand or something, you know, like in front of your face and all that, right? So that's this is this is literally what that's setting out to to deal with. So uh it mentions the article mentions that the capabilities will be available through the customer contact center so that companies can better safeguard customers' employee and proc and reputations. And this is specifically to combat AI attacks, if you will, or deep fakes, right? Deepfake type of attacks. So this is the first product I've seen that is targeting that specific, I guess, attack vector, like the AI attack vector. So we're gonna need to see a lot more AI security products, I think, or we're gonna see, I think, more AI security products moving forward, you know, targeting specific stuff like this. So pretty, pretty cool, uh, pretty cool article, actually.

Chris: 20:47

Yeah, not a ton for me to add here. I I I'm just I'll say that I'm totally in agreement with you that the the these first of all, this type of security um needs to exist. So I'm glad to see it kind of coming into the forefront. Um, and then it needs to be directly integrated with the platforms that people use, right? We need the these companies need to be able to meet people where they are, right? Because we're it as much as we uh we may do it in anger, we we all use Teams, we all use WebEx, we all use Google Meet, whatever. Um, and we're not gonna switch to a new collaboration platform just because it has this feature, right? This needs to be built in uh from day one. So um glad to see this coming into the fold. I am interesting, I'm not interesting, I'm not interesting. I am interested the most interesting man in the world right here. I am interested to see what happens when these things get it wrong, because as we know, AI gets a lot of things wrong. So if if we have something that like like what if you know something happens and either with the the processing on the software or maybe I just look relatively shitty that day and I get flagged as being a deep fake. And then then like I assume like alerts are going off on the other end. Like, what does that mean? Are they just not gonna trust anything that I say? And you know, what's what's the fallout from that? Do you get do you get tagged by HR after that or what? I don't know what the uh I don't think we've we've determined what the triage uh process is afterward, but uh I'd be interested to see what that looks like.

Tim: 22:16

Or like uh you'll get you'll get uh tagged because your background doesn't look real and you're like on a beach in the Caribbean or something.

Chris: 22:23

Yeah, exactly.

Tim: 22:24

That'd be cool. Exactly.

Chris: 22:25

All right. Uh last article that we have here. Um so it has been announced as oh, this is just uh a little over a week ago. Um the founders of Apstra. So if you're familiar with Apstra, Apstra is now um, I believe owned by Juniper, which uh I guess now is owned by HPE, um is a data center automation platform, um open source. Um if you've if you've talked to someone that uses Apstra, they're probably pretty um fond of it. I've said I've heard a lot of people say really good things about Apstra um in their day-to-day. But the previous founders of Apstra have now um joined together to launch a company called Aria or ARIA, I'm not A-R-I-A, I'm not sure how they pronounce it just yet. And their focus is on tackling AI networking performance. Um, so if you kind of look at the way they the way they talk about this, um they mentioned that networks that are built for AI operate at a very different um level than your standard networks do, right? There's a lot of very strict requirements. Um, and they specifically call out that you know a lot of these Broadcom chips that would be in use in these um kind of high performance networks provide a ton of like microsecond uh telemetry about the networks, and and the networks need to be able to react based on that telemetry. Um so it looks like that is the overall goal of uh of this company is to be able to kind of act as that platform that reacts to the telemetry that's um pouring out from those um from those chipsets and make decisions that impact um the AI operations, right? So um just looking at their website, um it looks like they're using they're they're focusing on tap tackling suboptimal GPU performance, vendor lock-in, overtuning, um, and time to revenue. So like initially, I mean we've we've had we've had people on the show to talk about these types of networks, right? We've had Peter Jones on from uh Cisco to talk about this stuff, and this is I mean, this is definitely a real thing. People are designing these things. Um I do think this is pretty niche though, however. Um I think we've entered into the realm where we've discovered how many companies will actually be doing inference and uh training on on LLMs and things like that. Um so I don't know how widely it I mean they're probably gonna be after the big end of town, right? That it's gotta be large customers that are doing this, maybe even uh hyperscalers, maybe even uh very, very large vendors. So I think this is a pretty niche product, but um I I gotta admit, I don't I don't quite um understand it just yet, um just based on the complexity. And um, you know, we're we're we're very much dealing with usually milliseconds uh in networking, not microseconds. So this is there's a lot to lose here, um especially from you know the points of GPUs being bought and idling, etc. So it's it's about extracting the maximum capacity from the network at any given time. So um interesting. Um uh be keen to see where it goes. Um what about you, Tim? What's your expert opinion?

Tim: 25:37

Well, so on AI networking, I'm still I still would not consider myself uh in any way an expert. Um I do they mentioned something as part of the there's a I mean I have multiple thoughts, but they point out specifically that while the company is targeting telemetry of the back-end Ethernet network that connects the GPUs together. So does they're say basically your network has to be optimized or your GPUs won't work at full efficiency. And and and I'm all for AI efficiency because God knows those things suck dry, like so much power and empty rivers and all this other stuff. Um, I don't know if targeting the network directly is the is the is the answer, but it's certainly part of the answer. It's gotta be part of the answer. I mean, assuming that bottlenecks and and whatnot are the are something that's keeping GPUs from being able to essentially do inference process, you know, deliver the results to the, you know, wherever it needs to go. Um it does mention here they're building it with merchant silicon from Broadcom and using the open source Sonic network operating system. So I don't know enough about that, about the Sonic network operating system to say, is there a reason, you know, to say, hey, you pick this for a reason, or if this was, you know, like I don't we're talking about something that that where we're measuring things in microseconds. Like it points out that the hyperscalers are doing it, which they are with the enhanced networking. Um, I but I don't know enough to about Sonic to say, like, oh, that's great. I'm glad they picked that because that's a great choice, or you know, hey, it's not gonna be fast enough. Because if you think about it, if you're really gonna make this useful, if you're gonna be trying to harvest telemetry, get it together, and then do something. I don't know. I don't know. That's the other thing. I'm still it's there's a giant fluffy cloud on the do something part because if you know, if I'm in my if I I'm having, I don't know, microseconds of latency in rack five of my, you know, 10 GPUs or whatever, however this is is is organized. Um, how do I detect that and then take an action that will do something? And what is the action I will take in the amount of time that it will do something meaningful, right? Because a lot of the stuff is like my again, I'm not an expert on this, but my understanding is a lot of this is micro burst type type networking, right? Which has been a bane of our existence since you know, you could find data that can move that fast, essentially, or you can find applications that could send data that fast. Um, and micro bursting has always been hard to catch, and it's always been extremely hard to correct for because by the time you've detected it, it's over, right? You've moved on and you don't have whatever corrective action you were going to take is worthless at that point because it's gone. So that's yeah, that's to me, that's the main one, right? My understanding is a lot of this inference and and and data transfer that's happening with these GPUs is is micro bursting or or just spending a ton of data all at once and then essentially not falling silent, but like back down to normal version, you know, whatever. So I'm that's the other question I have, right? And uh this doesn't this just says, hey, there's a problem, we're trying to solve it, and here's what we think we need to do to solve it. It mentions, it does mention probabilistic AI-driven methods for telemetry action, meaning essentially meaning we're training the AI, we either have trained or training uh some kind of machine learning or AI, whatever you want to call it, to recognize this problem quickly enough to do something about it and just kind of like glaze over and do something, you know, like it just do something quickly enough based on pattern recognition, if you will, to try to cut cut through. Um will it work? I don't know. I don't I we're we're in like I don't know. Uh so I am interested to see.

Chris: 29:19

Yeah, I mean it you you kind of speaking on that bris kind of brought something to my mind, and and I'll be honest, I don't even know if this is kind of a um a focus of this company at all, but it it's just something that kind of popped into my head. Um, I really feel for potentially the operators of networks like this that need to be providing this kind of stuff. Because the reason I say that is with the rise of the GPUs, I think even more than seeing kind of conversations around GPUs, we're seeing a lot of um options for GPU as a service, right? So obviously, if if companies go that route, if companies go that route, then they're gonna have customers paying them for a service. service and they're um depending on the size of the customer they could be held to a very strict um uptime uh or basically just need to be able to afford for every little thing exactly so I have you know a a good friend of mine used to work for a um uh an IS a major ISP in the US and they did a lot of sell backhaul for and some major um mobile carriers and I'll tell you like they would they would call in and and need a uh um a root cause analysis if they dropped like you know 10 10 packets on a on a spanning tree ring or something like that. So if now we're getting to the point where like hey like oh I came in and you know over the last hour since all this telemetry has been built in microseconds there's been 12,000 events that have happened where things have moved and potentially loss of performance, etc. If people have to answer for that then I think we definitely need some type of platforms that exist that can give that type of feedback in user readable form. Like I said I don't know if this is the focus of this company whatsoever but uh it just made me think about like we're really going to need that in terms of uh providing accountability and just um even being able to put it into kind of uh layman's terms about you know what kind of events are happening.

Tim: 31:20

That's a really good point yeah like I mean yeah because if you have an SLA you have a knock or I don't even know what a knock would look like for that. I mean it would just because I mean by the time a human can actually react to anything other than a catastrophic break which is essentially like a rack lost power or cables got pulled or you know something like that. You have all the screens and they're just always red they're permanently red and what and what do you do? What is the corrective action that you can take within the time span that again unless it's a catastrophic event that actually requires hands to solve like what is the what are you doing?

Chris: 31:54

Like so yeah troubled walkers foot I think yeah but we'll uh we will see all right uh well with that we'll go ahead and wrap up so thank you again for joining us for our monthly update on the news for the cables to clouds podcast my name is Chris Miles at BGP Main on socials joining me was Tim McConaughey at at Wangz.

Tim: 32:16

Wangez yeah no it's one golbez on Xer so I don't know which one to I don't know which one to collect I need to collapse it all into one. Maybe I should just make a new one and collapse them all. I don't know.

Chris: 32:26

That's the make it a fourth one. Collapse it all into a new one. Yeah that'll be even better. That's it um so then with that uh thank you so much for listening. If you enjoyed this please uh give us some feedback or share this with a friend we would uh we would love to hear from you as well and we will see you in two weeks with another episode