Ep. 57: AI Datacenters Explained

AI data centers are at the center of today’s technology boom — but what are they, really? 

In this episode of Talk To Me Petey D, I break down the fundamentals of AI data centers in plain language:

what they are, how they work, why companies are spending hundreds of billions of dollars building them, and what it all means for jobs, the economy, and local communities. 

We go beyond the headlines and look at:

 The difference between traditional data centers and AI data centers

Why GPUs changed everything

How cloud economics drive massive infrastructure investment

The real-world tradeoffs: energy use, water consumption, and environmental impact

Why communities are pushing back on new data center projects 

If you’ve ever wondered what’s actually powering AI systems like ChatGPT—or why these facilities are showing up in neighborhoods across the country—this episode gives you the full picture.

 This is part of the AI Literacy Series, where I break down the concepts shaping our world in a way that’s practical and accessible.

🔗 Follow & Connect

• 📘 Book: https://www.amazon.com/People-Management-Ground-Up-Aspiring/dp/B0DBGQ57XT
• 💼 LinkedIn: https://www.linkedin.com/in/pete-dempsey/
• 🌐 Website: https://peterdempseywrites.com/
• 📰 Newsletter: https://peterdempseywrites.com/newsletter/
• 🦋 Bluesky: https://bsky.app/profile/petedempsey.bsky.social
• ▶️ YouTube: http://www.youtube.com/@TalkToMePeteyD
• 🍎 Apple Podcasts: https://podcasts.apple.com/us/podcast/talk-to-me-petey-d/id1745885025
• 🎧 Spotify: https://open.spotify.com/show/4NrlsWzansuCfuApMCZzj0

Transcript

SPEAKER_00 0:01

There's a huge amount of money being spent building AI data centers. By the end of 2026, companies will have spent around$1 trillion in the past two years on capital expenses related to data center construction, primarily to run AI workloads. But this amount of spending and money is really hard to comprehend. In this episode, I want to break down and explain what data centers are, what makes a data center an AI data center, what jobs these data centers create, how they make money, what resources they consume, why they're difficult to build, and the harmful side effects that they might have, or may you may you why you may not want a data center in your own backyard. This is the Talk to Me PDD podcast. Talk about all things tech and society, knowledge work, leadership, and AI. I'm of course your host, PDD. This is episode 57, AI Data Centers Explained. And this is part of the AI Literacy series where I cover what I think everyone needs to know about AI's impact on our lives. You can check out that playlist on YouTube for past episodes. Please like and subscribe, support the channel. You can follow me on LinkedIn and Blue Sky. Check out my newsletter. It's available at peterdempseywrights.com. I'll have the details in the show notes. I appreciate all your support. Okay, so let's get into some basics. Everything with computers is built on layers and abstractions. There's so much complexity if we try to understand it all at once, it would be too much. So we'll see the same sort of thing when we're looking at data centers, how they're constructed, and how they work to support AI workloads. We're going to start by breaking it down to the machine level. So you're likely familiar with your laptop or tablet, probably also a desktop computer. So thinking about computers as the as the building blocks here. The computers and data centers are referred to as servers. And servers are pretty similar to desktop computers into the types of hardware that's in there. One of the things you might find challenging though is if you had a lot of desktop computers, trying to organize them all would be difficult. Stacking these towers on top of each other might fall over at a certain point and would get unstable pretty quickly. So in data centers, you want to put a lot of these computers, these servers together. A server often looks like a long pizza box of various varying heights depending on the type of server it is. And they're designed to be held in racks, which you can think of like bookshelves for storing your books. So vertically stacking up these servers, so you can have a bunch of computers held together in an organized way. They often refer to these servers as quote unquote headless, which just means that they don't have a monitor attached to them. So for most of your desktop machines, you're gonna have a monitor, you're gonna have your keyboard, your mouse, all those types of things. Um, usually not in a data center. You're just having these pizza boxes stacked on top of each other on racks. Now, these racks are not just as simple as a bookshelf. Um they often have a power supply that these servers plug into. So instead of plugging all your servers with their own cords into a wall, you'll plug them into your rack, and then the rack is connected to a power source. Um These computers also need to be connected to a network so they can get to the internet potentially or communicate with each other. When you're at home, you have a router that connects your home network to the internet and potentially to other machines and devices within your home network. Um, in a data center, you have the same type of thing, um, often referred to as switches that perform a similar function, just at a bigger scale and with more machines and more efficiency. Switches are often rack mounted as well, so they might look like a server or fit into the same rack. Um, and then instead of having a bunch of Ethernet cables running all over the place from your servers, you have them running into the switch on the rack, and then the switches connected together in other components. So that's sort of the basic building blocks of of the data center, um, how you're gonna put these components together. So now as we get inside a data center, um, you're then gonna organize your racks into rows of racks. So you can sort of think about this like bookshelves at the library where you have multiple rows and organizing the different books in different places. Um I haven't been in a ton of data centers, but I've been in a few, so I can share some of my personal experience. Um, you know, they're interesting places, they can actually be you know kind of beautiful in a lot of ways. You have you know really neat rows, neat and organized rows of flashing lights, um, kind of just pretty cool to see that amount of computers and servers um organized together. Um there's a ton of cabling with all of the power cords and Ethernet, uh kind of running things overhead, running from machines. And that's actually an art form that people that are able to take that many cables, cut them, crimp them, put them together in a way that is neat and organized and clear where they're going. So there's I think there's actually some beauty to be found within the data center itself. Um, but it could it can also be an environment that doesn't really feel very hospitable uh to people. Um it's really loud, kind of in a unnatural way. Um, you know, you think about as you start doing a lot of stuff on your laptop or your computer at home and you hear the fan speed up and start to get louder, well, you know, multiply that by a couple thousand times, and that'll give you an idea of what it sounds like in there. Uh, there can also be kind of strange temperature shifts. Um, you want to keep the environment cold because it's more efficient for the computers to run that way. Um, but at least the data centers that I was in, and this was a while ago, so maybe before some of the liquid cooling that they have today, is you get kind of these really cold sections, and then you get to the back of the rack where all the heat is exhausting out from with the fans and it would feel hot. So just kind of this weird feeling of air, cold and hot. Um I don't know, maybe it was because I was always a visitor to data centers as opposed to working there, you know, in the data center often. I always sort of felt like uh like I shouldn't be there, kind of uh intruding on the machines, so I don't know, maybe that was just in my head, but just trying to give you a little insight of what it what it feels like being in inside the data center. So that's kind of a a data center in general. Um, so what what makes an AI data center? So instead of these sort of general purpose computers, which are kind of like a desktop, just in a different physical form factor, um, you have servers in an AI data center holding AI GPUs. So a GPU is just short for a graphics processing unit. So this is specialized hardware. Originally they were designed for video game rendering or video rendering. And renderer just means how you show something on a screen. So they're designed to work really efficiently with large lists of numbers. So if you think of every dot on your screen or on a TV just as a different number, um, kind of representing what color that dot should be, you can kind of visualize what this would look like. There could be a very long list of different numbers at different coordinates. And GPUs can process changes to that list very quickly. So if you're trying to show you know a video where things are moving, or a video game where you need to represent things fast to keep up so it looks natural to a user, GPUs are really good at that. So it turns out that kind of math that matrix operations or linear algebra, um, it's good for more than just video games. Uh so gaming GPUs started getting used in scientific research and and that sort of thing to crunch numbers and do different types of operations at a level and scale that general purpose computers and CPUs just couldn't do. So as that took on, then you started to get more specialized GPUs, so not really kind of the ones that were built for gaming, um, but AI or machine learning specific. So these are GPUs that don't do video game rendering at all, really. They're just for processing AI workloads. Um, so that's kind of the main difference in the hardware that you're gonna see within an AI data center. Um, one of the differences or a big difference here is that GPUs that use a lot of electricity, um much more than a general-purpose computer, um, generate a lot of heat, and they are also extremely loud. If you've ever had a gaming PC or maybe you've built a crypto miner or seen those, they can get very loud with just a couple GPUs, so you can imagine what an entire data center would sound like with all these GPUs running. So, yeah, AI data center, just a data center with lots of GPU-specific servers. Um, it's much louder, uses much more electricity, generates a lot more heat. Um, as I mentioned, you know, GPUs were were also using crypto mining, which is why you see some of these crypto companies that may have hosted their own data centers or um rented out GPUs for crypto mining switching into the AI space. So now as we look into what jobs these data centers create, um, because that's something you know that everybody's interested in right now, sort of the impact on jobs in the economy from AI and AI-related things. Uh the number one is is construction. Um, there's a lot of people, machines, and specialization that goes into building these data centers on the construction side. And this is the bulk of the increase in the the GDP, the gross domestic product that we've seen. This is spending on the construction side and the buying of components for AI data centers. Um, and that's the vast majority of the sort of the positive impact on the economy that we've seen from AI so far. Um but it's important to keep in mind this construction is temporary. Once these data centers are built and they're done, the jobs that were created and the spending from construction stops. There isn't any need to continue doing it. Once data centers are built, there's really not that many jobs within the data center itself. Um there'll be some technical people that are there kind of working within the data center, but even in a large data center, it's not usually that many people. You have physical security and that type of thing, um, which again will be some people, but not all that many. Um, and then potentially some what you would call remote hands, so people that can go walk to a physical server, push buttons, plug things in, pull them out, that sort of thing. Um so they really don't provide that many long-term jobs to the local community that they're built in, um, which is one of the reasons, and kind of we'll get to that on why people might might push back, but it's just something to consider that once the data centers are built, that um they're sort of just there and operate with a pretty limited staff. So, now how do data centers make money? Um for this, we have to go back to this concept of consumption economics and talk about the cloud. Um, so we're gonna remind ourselves of this idea of having layers and abstractions, and that's really what the cloud is. So, you know, before cloud computing, if you wanted your own servers to do various things for your business, maybe host a website, um run your email server, that sort of thing, um you might host those yourself kind of in the office where you ran your business, but a lot of the times people would rent out part of a data center for the business. So you'd have this kind of little piece within a building that maybe you had only you had access to, but you could use the power and network and other other services within the data center. But you bought your own racks, you bought your own servers, you had to maintain them, you had to install the hardware, um, whatever software you wanted to run for your business, you had to install that, you had to maintain that software and upgrade it, um, which can be kind of a pain and some additional overhead. It's also a security risk if you're not keeping your software up to date. So there were some downsides about this self-managed approach. Uh so then we you know companies introduced this idea of the cloud. So instead of managing all of these data center components yourself, you would then pay a cloud provider to do it. So this introduced various kinds of abstractions where instead of just having this physical server, um, you could get access to components at all sorts of different levels. So you could have access to application software without thinking about, okay, what are the servers that I need to run this? Um this is known as the as a service model or X as a Service. So you have software as a service, infrastructure as a service, platform as a service. Um now you just pay for your services as a subscription versus buying and maintaining sort of the capital, um, whether in hardware or software yourself. Um so you trade ownership for ease of use. So the cloud was originally born from Amazon, building out its own infrastructure. It needed to do that to scale its business and to be able to operate dynamically as needs for hardware or software came up with keeping things updated, all of that sort of thing. Um so one of the things that that is interesting about the cloud and software as a service or X as a Service is it gives you this illusion that you have unlimited resources. You can just add more often in a you know, kind of a graphical user interface. Whereas if you have your own data center space, you're managing your own racks, your own servers, it's pretty clear that you have limits there. You may need to buy more servers or more racks or space to put them in, that sort of thing. So shifting over to this cloud computing model, um, I think one of the things that we may not think about that much is that this really introduced this illusion that we can have as much computing capacity as we want, and things can just scale and be elastic without any constraints on the real world. Um so it turns out this as a service model has very good profit margins. Um, it's really become the de facto offering for a lot of the forms of software or even hardware. Um many companies actually stop making this standalone software that you could host yourself, and that's effectively forcing everyone into the cloud. It continues to make those margins good because you you don't really have any other offers. Um AI data centers are following along in this same same model of the cloud. Um so they have, as we know, these servers with GPUs, and they can sell access to the GPUs as a service that could be used for for model training, which is to build the the AI models, or it could be for inference, which is just running the model. Um so it could be at this kind of direct GPU level, or it could be various services on top of those those models. Um could be APIs to access and run services, but with the the abstraction that you're talking to the API, not directly to the model, or even you know, the user interface on top of all of that, like chat GPT. Um, so again, we're seeing this this user interface layer, but we don't say all the layers and abstractions underneath. We just assume those resources are there and it'll work. Um AI GPU systems are much, much more expensive than traditional servers. Um that's one of the reasons why these capital expenditures are so high, um, because it just they just cost a lot. Uh they also have a relatively limited lifespan. Um, I think three years is about average, maybe three to five or three to six, depending on on how they're used. Um and whereas with a traditional server, let's say you're running a website, that can run on a single server or part of a single server. Uh, whereas applications or the models that we're running, uh AI AI on these GPUs, they often require multiple GPUs for large language models, quite a lot. So you have these really expensive limited resources, and then you need a lot of them to make some of these models work. Uh, so there really are some real resource constraints here in a much different way than with traditional computing resources, which we thought of as unlimited and kind of were effectively unlimited into what we could access. Um I won't get into it now, but I would say profitability and the margins for um AI data centers are not nearly as clear-cut. We'll leave it at that for now, maybe a future episode, um, but I think there's still more for us to learn on that. All right, so now we've got kind of an idea of data centers, AI data centers, what's going on there. So, what resources do they consume? Uh, so the big one is electricity, um, especially in AI data centers, and you'll often hear these data centers described um by how much energy they can consume, so megawatts or gigawatts, that sort of thing. So it's just a measure of how much power they can consume based on the hardware that's there. Um and then something that's important to bear in mind with that, the numbers that are given out are not necessarily are they're not how much AI GPU power you have, because there's often additional overhead on making that compute run. So powering everything, um getting power to the racks and networking equipment, all that sort of thing. Um, so I don't know exactly what it is, maybe 20 or 30% of the numbers you see quoted, you could take that off for the overhead to run the infrastructure to support the the AI GPUs that are there. Um within a traditional data center, it's pretty common that you have variable consumption of resources. There are certain times, let's say you're hosting websites, certain websites and certain times are going to be busier than others, so generally not all of your machines are running all out all the time. Uh in an AI data center, especially if you're if you're on a training run, you're basically running these machines and GPUs flat out for a couple months at a time, potentially. So that's a lot of power, a lot of heat consistently. Um and then even you know, when they're running during inference, they're they're using a lot of power and um not sitting idle all that much. Um even when they are sitting idle, these GPUs with all electronics have there's something called uh parasitic consumption. So even when they're not really running and doing anything, they're they're pulling a decent amount of power. Um, especially the newer AI GPUs um consume a decent amount of power just sitting there, not doing things. And I don't know all the details of it, but I I think it's challenging to kind of turn them totally off uh when they're idle and then bring them back on cleanly because of kind of the network coordination that needs to happen. Maybe I'm sure somebody else knows more about it than me, but for the most part, just using lots of power all the time, running flat out for things like training, even when they're not running, they're still consuming more power than traditional servers are. Um, how do you keep all of this cool? A lot of the AI data centers and more modern data centers are using liquid cooling and water, so um, that's consuming fresh water using that, um, and then having to find a way to generate and expel the heat that's generated by all these machines. Um building all of these um components, the the GPUs and all of the other networking and pieces, um, they do also take a lot of rare earth metals, which can um have complex uh supply chains, we'll put it that way. Uh the way that they're extracted is not always the best for the environment or the local community that they're in. So a variety of different um elements and resources that that AI data centers and data centers in general consume. Um so data centers are hard to build. Let's talk about why. Um, you know, you need permits and all that sort of thing, just like if you were building any sort of big industrial building somewhere, um you need to go through that. Um it's there's definitely seems to be growing local pushback on a lot of builds as people are sort of getting more aware of what data centers will do, that they may not bring all that many jobs, general sort of resistance to changes brought on by by AI. Um so lots of concern about what's in it for the community or what type of deals to be had. Sometimes data centers, you know, they've tried to sort of go um around sort of public public permits and then kind of bringing it out at the last minute, which you know is probably not a recipe for the community to be too happy about it. Uh so that continues to be an issue, and probably even more so now than than it was a few years ago. Um they're also just complex. Like they're really there's a lot of specific requirements, um, all the equipment that you have to get in there. And something about AI data centers is that it's often a very specific way you have to build it. Based on the type of GPUs that you're putting in. So new generations of AI GPUs come out every year or so, and they can't all work within the same racks and power and network. So within a traditional data center, if you're upgrading and kind of moving components around, it's not so bad. But when you're within an AI data center, it may be a pretty big effort to go from one generation of AI GPUs into a new one. So that can make it more complex as well. There are sort of generation-specific designs based on the AI GPUs that are going to run within that specific data centers. Alright, lastly, let's talk about some of the harmful side effects of AI data centers or why you or someone may not want a data center in your backyard. So you can kind of think about AI data data centers sort of like factories or anything that was industrial zoning, this kind of general pollution, and in general, you know, residential communities don't want to live that that close to factories. There is local resource consumption, definitely the impact of using a lot of water for cooling. There's been a lot of different arguments about kind of the water consumption side of things, how much of an effect of this is isolated to the local community, how different is this from general industrial use. So to me, I feel like it's a little bit up in the air, but definitely a local resource consumption issue with water. Beyond that, I don't know, do do some more research, make your own decision. All the heat has actually become so so hot that these AI data data centers are creating a heat island, as it's called, where in the local area around the data center, uh the temperature is actually rises and um alters sort of the microclimate of that environment, um, which is probably not a good thing. Um, a really big one that we talked about is the power consumption. How do you get all of the power to these data centers and especially AI data centers where they're running close to full blast all the time, um in an effort to sort of get around some of the issues connecting to the grid or to maintain constant power? Um many of these data centers that have been built are using um fossil fed turbines um to run to create power to the data center. Um and I think you know some of these turbines can be relatively efficient, uh, but up until recently there wasn't really a huge market demand for them. Um so I think there's quite a bit of lead time to get modern turbines that are more efficient and have um you know less pollution output. Um so I think a good example or a bad example, I guess, or a harmful example is looking at XAI's Colossus Data Center, where they built this data center with much older turbines that are not efficient, create a lot of pollution. Um, and that's really damaged the community near those data centers, seeing rates of asthma go way up just as the air quality's gone down. Um so in order to get that consistent power, if these companies or these data centers are making choices on power creation that aren't good for the environment, local and beyond, that can have a significant impact. Um you know, beyond the the local environment, um just the the sheer demand on the electricity grid, um, there's a risk that that's going to push rates up for everyone just because that when there you know there's there's more demand on on the current supply, uh the rates go up. There have been some conversations or commitments to try and offset that. We'll we'll sort of see where where that ends up uh ends up, but that's definitely a risk. Um you know, at least in the US, probably less clean energy if we need more energy capacity than kind of what we already have, and there's a rush to get it there, and we're using these these older turbine technologies, um that's going to uh create more pollution and have more of an environmental impact. Um, and then lastly, for kind of general consumers of computer hardware, um we've seen prices go up, thing availability on components like memory or hard drives, just because there's so much demand for these components, even you know, they're going, they're not necessarily going into the type of computers you would have at home, but the components are similar or built by the same companies. Uh, so we're seeing this boom in data center construction drive up component prices across the board for uh computer hardware. So there you go, covered a lot. Um, there's there's a lot to it with data centers, but hopefully breaking it down to these pieces, you can have a little bit better understanding of how they're built, um what goes into it, and what concerns people might have in the local comput in their local community and beyond. Uh so yeah, hopefully you like the content. Uh please like and subscribe, comment. Um, I'll continue this AI literacy series touching on more topics. Um and yeah, uh see you next time on Talk2Me PDD. Thanks for listening. Bye.