How To Store And Serve AI Data At Scale

Everyone is obsessed with bigger models and faster GPUs, but the part that decides whether AI actually works in the real world is the data behind it. We talk with Paul Speciale CMO at Scality, about what happens when enterprises try to store, protect, and serve AI data at petabyte and exabyte scale, and why the rise of huge context windows turns “just a chat” into a massive storage and latency problem.

We walk through the full enterprise AI pipeline, from data collection and cleansing to inference and the long archive tail most teams never plan for. Paul breaks down which AI phases truly need the fastest storage, why inference can demand microsecond access, and how tiered designs (hot, warm, cool) align to modern GPU stacks. We also dig into what’s changing in the market right now: flash price spikes, constrained supply, and the reality that power per rack often matters more than raw capacity.

From there, we get practical about operations and risk. We explore autonomous infrastructure as “tell the system what you want, not how to do it,” including human-in-the-loop recommendations that can move data to cheaper tiers and cut power bills. We also cover cyber resilient storage, immutability for ransomware defense, and the growing pressure around data sovereignty and even code transparency. If you’re building an AI infrastructure plan, this is the blueprint mindset that keeps GPUs busy and data trustworthy. Subscribe, share this with your infrastructure team, and leave a review with the biggest AI data challenge you’re facing.

Transcript

Why AI Suddenly Needs More Storage

SPEAKER_00 0:01

Hey everybody, really intriguing conversation today as we look at the data behind AI that's needed to be stored, protected, managed, and ready to use with Skelety.

SPEAKER_01 0:12

Paul, how are you? I'm doing great, Evan. Thanks for having us here. It's a pleasure.

SPEAKER_00 0:17

Thanks for being here. Uh, such a timely and relevant conversation today, as we're all obsessed with models and GPUs, but there's much more attention that is being paid to the data behind AI. That's your wheelhouse. Before that, maybe introduce yourself and how do you describe Scality these days?

SPEAKER_01 0:36

Yeah, absolutely. So I'm Paul Spochali, I'm the chief marketing officer at Scality. I've been with the company now a dozen years. Uh, my career has been in data management and data storage for all these years. Scality is really a focused expert on storing massive capacities of unstructured data. Uh, we're talking petabytes to exascale. Uh, and we do that for big corporations, for big governments, for service providers that need to provide cloud storage, cyber resilient storage, and now storage for AI.

SPEAKER_00 1:07

Well, that's uh it sounds so simple and elegant. Of course, there's a lot behind that. Why now? Why are companies paying so much more attention to data and not just uh the big flashy models?

SPEAKER_01 1:21

Yeah, I would say that this has been kind of an evolution, right? As everything is in tech. Uh for me, having been around a while, I've seen the evolution from web, you know, the web phase 20 years ago to the cloud native phase, and now we have the AI phase, right? So three years ago when these models came out, you're absolutely right. All the attention was on that. Uh people were, myself included, bowled over by what they could do. Now we're a few years into it, and it's pretty clear that where this is going is data is going to be key, right? So the data is just going to grow as a function of the model size as the parameters increase. But what you also don't think about kind of intuitively is that as the context windows within these LLMs get longer, that also has huge demands on data storage, right? Simple way to think about it is if I have a conversation and I say hello, that alone can be a thousand tokens to the LLM, right? And now it can retain a couple hundred thousand of those for me, but it has to compete for thousands, tens of thousands, or even more other users. So quickly you get into memory spaces of petabytes and and beyond. So this is why this is so critical right now. It went from kind of shiny

The Many Phases Of Enterprise AI

SPEAKER_01 2:32

new models to now we realize that the infrastructure is going to be important.

SPEAKER_00 2:36

Yeah, important to say the least. And what are some of the many problems that you think companies typically running into as they scale with their AI data?

SPEAKER_01 2:46

Well, I guess the number one thing at the top level is this realization that AI is not a single workload. That's that's really a kind of a crux aha moment. You start thinking, is it the LLM interacting, which is really inferencing? But it's not, right? If you're going to do enterprise AI as a big corporation, you've got a problem starting with data collection. You need to pull all these data from different sources, you need to make sure that it's standard and cleansed and filtered and prepped the right way. Um, you need to feed the model, although historically that's been done by the frontier model companies. Uh, then you've got inference, right? Which is the problem we just talked about. There's a huge storage demand on that. And I have not seen people throw away data after the fact. So there's this long-term archive tale, right? I think that's the realization is that there's many, many phases of AI. You have to ask what the requirement is of each phase, and everything's different. I think you find super high performance phases, you find things that are okay with warm storage. And then if you're doing archive, obviously you don't want to pay the premium for flash and have everything sit on that forever. So that's the spectrum that we really think about is addressing something that needs to flex, something that needs to vary as the workload change as the workload changes.

SPEAKER_00 4:02

That's a great point. And uh we're we're entering the storage wars, uh, not the TV show, the AI edition. Um what parts of AI need the fastest and most expensive type of storage? What parts don't? How do you think about that across the continuum?

Training Versus Inference Storage Demands

SPEAKER_01 4:20

Well, the the easiest way to think about it is that uh obviously model training, where the you know the frontier companies needed lots and lots of throughput, that is a specialized domain of high performance. You're talking terabytes per second. Um inferencing now is a real demand. I mean, you're talking super low latency access to storage. You've got smaller data elements, but they need to be accessed at microsecond type latencies. Uh if you look at the NVIDIA stack for storage, they've got kind of five layers defined in a pyramid. At the very top is the HBM, the host bus memory. That's the fastest it's going to get. But the problem is it's super expensive and at some point it's going to be limited in capacity, right? Behind that, you can start extending out through what's called KV cache. So KV cache is really an ability to take that HBM memory and kind of page it out to fast NVMe flash storage. Um, that's where a lot of us vendors are working right now. And that's integration with NVIDIA GPUs. Uh, it's with their libraries like Dynamo and uh Nixil. Uh beyond that, there's three more tiers, right? You have uh kind of a hot tier, a warm tier, and a cool tier. Um, so they understand that that's what has to be organized in that manner. But I would say today for inferencing and the fact that we see these trillion parameter models and huge context windows, that's where the attention's going, and we need to just provide ultra fast storage for that. Very low latency.

SPEAKER_00 5:47

Indeed. Uh, and you know, AI is making IT systems harder to manage. It used to be a pretty straightforward, boring part of IT, I guess. But you've got developers and their own storage requirements, individual users. Now I'm running out of storage almost every week on my Mac here. Why is that?

Data Gravity And Storage Sprawl

SPEAKER_00 6:08

What's happening that's making this all so complicated?

SPEAKER_01 6:11

Well, I think again, I'll use my historical perspective. 20 years ago, you know, let me say 30 years ago, I thought a gigabyte was a lot, right? And then the web came along and we all wanted to store photographs and you know, cat pictures and videos and things like that. The petabyte became pretty normal. Uh, multi-petabyte, you know, 10, 15 years ago within corporations was just standard. I think now we're in the exabyte arena, and it's because all of this data is just being generated and generated and generated, but you have to feed it with something. You better feed it with quality data. That's something that you can trust, high-integrity data, so that gets into cyber resiliency. But there's just no shortage of data creation on this planet. You've probably seen the stories about zettabytes of storage being created every year. I believe it, right? A lot of it is gonna be on small pocket edge devices, of course. But within the corporation, they have a fair chunk of that. I think the challenge is this, right? If you're gonna have hundreds of storage silos, you mentioned operational headaches. That's a headache, right? Imagine a staff, a small staff having to run around and care and feed all these different systems. That's one burden. It's just the number of devices, the number of silos, right? But then imagine the juggling, right? Because this data comes in hot, it becomes warm, it becomes cold. Who's gonna take the responsibility for moving that all around, right? And you know this moving data around takes time. It's costly. People have this state statement around data has gravity. It does. The thought of moving a petabyte from point A to point B, that's an endeavor, right? So it'd be better off, we'd be better off if we have systems that can manage that. And that was kind of the thinking that we had with our latest technology innovation, which is to start managing things more autonomously.

What Autonomous Storage Really Means

SPEAKER_00 7:56

That's interesting. So, autonomous infrastructure, I guess can mean different things to different people. Yeah. Uh, what does it mean to you in in sort of simple terms?

SPEAKER_01 8:04

Well, what it really means is that you have a system that can be told what you want, but you don't have to tell it how to do it. So the analogy I've heard is I'm getting in my self-driving car and I do tell it what restaurant I want to go to. It doesn't make that decision for me, right? But once I tell it what restaurant I want, it'll take care of getting me there. That's kind of the metaphor that I'd like to use for the storage. It's human in the loop, it's autonomous uh operations once you understood, once it's understood clearly what you want it to do, and it's always checking back with the user, right? So this could mean simple operations like I'm adding capacity to the system. Can you take care of onboarding the new servers and provisioning the drives, right? That's sort of a low-level task. But what if I want it to be looking smartly at the data and telling me, hey, this workload doesn't no longer justifies being on the most expensive tier of storage. And make that recommendation to me and say, I think the profile of this data access pattern can move it down to the next tier and then consult the user and the user okay that, right? That's sort of this human-in-the-loop consult, but it's really a loop, right? We need to make sure that the AI advises, the human comes in and just checkpoints, and it continues in that manner. That's what we mean by autonomous. Um, last point I'll make none of that works without, again, the data, right? So if we we need the data from the user's operational system to make these recommendations, so it starts with a big repo of kind of this operational metadata. What's the user's workload profile? What's the hotness of the data or the different data sets? And once it has that, you know, it can make these recommendations about data movement and also make you uh recommendations about power consumption, for example, because you really can start saving a lot of power, which is a big restriction in the data center. Oh wow. Right. And then, of course, system health. We've done that for many, many years. You know, predicting that there may be failures based on patterns. That's been done, but now to apply AI kind of techniques to that, that's really the next level.

SPEAKER_00 10:06

Brilliant. And in your experience, uh, what has it meant to your customers' IT teams on the ground,

Flash Prices Rise And Power Savings

SPEAKER_00 10:12

how they're spending their time, what they're doing? Uh, do you have any anecdotal or feedback in general on that today?

SPEAKER_01 10:19

I I do. So, I mean, we've had customers for years that have looked at this and thought, I'll over-provision storage and buy flash for everything, right? And that for the last few years was an okay answer. I mean, you were going to pay a lot for it, but it doesn't work now, right? Why? Because we have this super cycle of NAND flash memory prices happening and the supply chain is all impacted. All of that storage is getting consumed by the hyperscalers and the neo clouds, and the rest of us are sort of left at the bottom of the barrel, which means flash is expensive. It's gone up four or five times in price in the last year. It's also hard to procure. You could be waiting six to nine months to get it. So, our sense is that the best thing we can do is start using that more efficiently. Okay. And we've done that with customers who have said if I put 10% of my data on flash, 40% on you know, warm class of storage, and maybe the rest on cool, they found out that they can actually save like 80% of their power bill.

unknown 11:16

Wow.

SPEAKER_01 11:17

That's huge in the data center today. And obviously, you can get the commensurate cost savings as well if you look at the different media tiers. So that's real world stuff. And you know, today the data center is not judged by how much you could stuff in, it's how much you can power for the rack. So this is real screen.

SPEAKER_00 11:33

Brilliant. In the meantime, we see uh you know, tsunami of cyber attacks, fishing uh attacks, uh outages and power challenges.

Cyber Resilience And Data Sovereignty

SPEAKER_00 11:44

Um, so I think resilience has been kind of a theme for this year. Yeah. What should companies be doing more of to protect their data in general, also with solutions like yours?

SPEAKER_01 11:56

Well, I think over the last few years, people have become pretty aware of cyber resiliency needs in storage like immutability. And that's really a great technique to protect against ransomware, right? To make sure that some nefarious actor can't come in and change the data or lock it down or encrypt it. But now we need to be more holistic to the whole thing, right? So top-to-bottom architectural resilience. That's one philosophy. I would say the other thing that we're seeing is, especially in Europe, sovereignty concerns ruling everything, right? They want to make sure that they're aware of the lineage of their data. They also make sure where it's coming from, where it's, you know, where it's located. Uh, that's been a huge concern. There's one other thing that I'll mention, and it caused a decision we made. Um, the code itself is something that uh customers now want to inspect, right? So we've had open source in the past. Uh we now went to an open code model where it's not community managed, but we let the customer inspect the code. So make sure that the sovereignty of the code itself doesn't have anything that you're not trusting or not uh you know certified with. You can also make uh rec, you can make change requests or submissions to the code. We will accept those, we'll inspect, we'll inspect them. And if we deem them to be you know something that we can manage, we carry them along in the code base from then on. So it's kind of you know three-quarters of the way toward open source without the whole community thing, but the vendor standing behind it.

SPEAKER_00 13:21

Wow, what a unique approach. I haven't heard that before. Pretty interesting.

Infrastructure Planning And Edge AI Next

SPEAKER_00 13:26

So if I'm a CEIO, maybe starting uh an AI infrastructure plan for my enterprise, besides working with Skality, obviously, uh uh one point, but what should they focus on first, just to uh to get ready? What's the blueprint to be successful?

SPEAKER_01 13:45

I I think the key thing is to consider the entire infrastructure as important to your success, right? We did a a survey recently where we pulled over 500 enterprises in the US and in Europe, and we looked at those that were kind of advanced in their AI usage and those were less advanced. The more advanced ones really considered the storage infrastructure up front. And the decisions they made to support that were really to look at things that were optimized for the workloads. So they weren't in this model of let me get us, you know, all flash everywhere, right? They really were more intelligent about this workload needs this kind of profile, and others need different things. And they made those decisions consciously, and that ended up leading to a much better success equation. So plan ahead, not only at the GPU level and not only at the model level, but obviously we all know that it's process. And then for IT teams, it has to be the infrastructure. You've got to be able to feed these things. I'll I'll throw one other metric out, uh, Evan. You people buy GPUs and they end up being idle, right? And the industry stats show something like 10 to 30 percent utilization rates in GPUs. So now you imagine you buy these super expensive boxes, right? They can be a half a million to a million dollars per rack unit, and then they sit around, right? You need to be more intelligent about that. Buy the storage that fits because in the end, it needs to feed those machines as quickly as it can and keep your expensive investments running. So I would say plan the storage infrastructure a priori.

SPEAKER_00 15:19

Brilliant advice. So it used to be in tech, you know, July, August would be a little quiet, quieter. Uh not the case anymore. Certainly in AI, there's so much going on. What's on your roadmap or your radar for the next few months? Yeah. Quarters, what are you excited about?

SPEAKER_01 15:35

Well, I I agree with you, first of all, that uh, you know, you used to go to sleep kind of in the summer and not do much, but it's been busy. I just came out of Europe where we had two big uh summits. Uh I don't know if you've heard about these, but Raise AI is a big summit that pulls together a huge number of people in Paris, France. There's also Viva Tech. I see absolutely every player there. So that's becoming a hub. And you know, in the past, uh Europe would uh be one of the more quiet periods in the in the summer. So that's happening. Um in the fall, I think we see more advancements coming out from our side in terms of edge AI. We see inferencing moving to the edge, right? There's so many use cases that are not kind of in the core data center. But think about you know, factory surveillance, factory process automation. Uh obviously there's surveillance on the edge, but what about man, you know, manufacturing and agricultural yield checking? There's so much happening on the edge, so much data being created that we think solutions that are easy to deploy and easy to manage on the edge with kind of zero touch, uh, but they're they're kind of combining AI technology and storage together in a package. That's where we see excitement happening next.

SPEAKER_00 16:46

Well, I would agree with you there and with Kimmy dropping yesterday. Uh this is only going to heat up. It's only going to heat up. The the AI war has continued. Thanks so much for sharing your quite unique and innovative vision. Really learned a lot.

SPEAKER_01 17:01

Happy to do that. Great, great having a chat with you, Evan.

SPEAKER_00 17:04

Likewise, thanks everyone for listening and watching. Also check out our TV show, techimpact.tv on Bloomberg and Fox Business. Thanks, everyone. Thanks, Paul.

SPEAKER_01 17:13

Thank you. Bye bye.