Energy and the AI Race: Why Power Is the Real Bottleneck for Artificial Intelligence

AI isn’t limited by models, talent, or capital — it’s limited by electricity. 

In this episode of the Macro AI Podcast, Gary Sloper and Scott Bryan break down the energy reality behind artificial intelligence, from individual AI usage to hyperscalers and national infrastructure strategy. They explain where AI actually consumes power, why your laptop is just the remote control, and how every prompt to a large language model triggers real energy use inside GPU-powered data centers. 

The conversation scales from home offices to enterprises, introducing the concept of the “shadow data center” — the hidden energy footprint organizations incur when using AI through SaaS platforms and APIs. Even without owning infrastructure, businesses are consuming significant AI-driven electricity at scale. 

Gary and Scott then examine how many gigawatts of new data center capacity are being planned in the U.S. and globally, why grid timelines are becoming the true bottleneck for AI growth, and how energy availability is reshaping competition between the United States and China. 

Bottom line: AI strategy without energy awareness is incomplete. The future of AI will be written in code — but powered by electrons. 

About your AI Guides

Gary Sloper

https://www.linkedin.com/in/gsloper/

Scott Bryan

https://www.linkedin.com/in/scottjbryan/

Macro AI Website: 

https://www.macroaipodcast.com/

Macro AI LinkedIn Page:  

https://www.linkedin.com/company/macro-ai-podcast/

Gary's Free AI Readiness Assessment:

https://macronetservices.com/events/the-comprehensive-guide-to-ai-readiness

Scott's Content & Blog

https://www.macronomics.ai/blog

Transcript

00:00
Welcome to the Macro AI Podcast,  where your expert guides Gary Sloper and Scott Bryan navigate the ever-evolving world of artificial intelligence.  Step into the future with us  as we uncover how AI is revolutionizing the global business landscape  from nimble startups to Fortune 500 giants.  Whether you're a seasoned executive,  an ambitious entrepreneur,

00:27
or simply eager to harness AI's potential,  we've got you covered.  Expect actionable insights,  conversations with industry trailblazers  and service providers,  and proven strategies to keep you ahead in a world being shaped rapidly by innovation.  Gary and Scott are here to decode the complexities of AI  and to bring forward ideas that can transform cutting-edge technology  into real-world business success.

00:57
So join us,  let's explore, learn  and lead together.  Welcome back to the Macro AI Podcast. I'm Gary Sloper joined as always by my cohost, Scott Bryan. And today we're going to talk about something that has been an ongoing story in the world of AI, electricity. Yeah. And this is one of those episodes where if you're a  business leader, you might want to listen in and really try to understand the full macro picture because

01:26
You everyone's talking about models, GPUs, talent, data, ah but none of that works without power. And the pending constraints are definitely very real. Yeah, exactly. mean, AI isn't magic. It's physics.  It's electrons moving through Silicon and as engineers,  whether they're network engineers, infrastructure engineers, cloud engineers, everyone really knows electricity is foundational. You do not get packets without power.

01:54
you don't get compute without power and you don't get artificial intelligence without power. Right.  And the core idea we want to land today is pretty straightforward. AI progress is no longer limited by intelligence or capital, or even to some extent, GPUs based on the amount of manufacturing that's coming online. It's really ultimately limited by energy. So in this episode, we're going to walk this all the way from utilization in your home office to midsize businesses.

02:23
to global enterprises and to even to the hyperscalers. And then zoom all the way  out to the US versus China and global competition that exists today. But by the end, our goal is that business leaders listening will have a very clear mental model of how much energy AI actually uses and why this is now actually a board level issue or at least a level that senior executives need to  be able to understand and articulate.

02:52
I think before we go any further, we need to  clear something up. When we talk about an individual AI power user, the energy story is not your CPU or laptop that you're using at the moment. Right.  Your laptop is just the remote control.  The real work happens  in the data center  on a GPU somewhere else in the world. Yeah, that's a point, Scott. So I think let's talk about what actually happens when you ask a question.

03:21
to a large language model. Yeah, when you hit enter, uh your prompt gets sent to a GPU-backed inference cluster. And that GPU is typically pulling 300 to 700 watts while it's actively processing that one request. So  lighting up a neural network  in the cloud. Right, you're spot on there. And depending upon the size of the model,  the length of the prompt, and the level of response, that GPU that you were just talking about,

03:51
may be engaged anywhere from a fraction of a second to several seconds. And you've probably noticed that um in some of your prompts in the past.  Now here's the important part. A single query might only consume a few watt seconds to a few watt minutes of energy. And that sounds pretty tiny. Yeah, it does. But  it's not the size of the event that matters. It's the multiplication.

04:17
Yeah. So if an average LLM query burns on the order of one to five watt hours at the data center level, which is  reasonable engineering estimate once you include GPU time, memory access, networking, cooling, overhead. Yeah. And then there's, you know, a  single knowledge worker asking, I don't know,  for example, a hundred AI assisted questions uh per day that could easily be responsible for a hundred to

04:44
It's at 500 watt hours per day of remote energy consumption. That's a lot.  That's the equivalent of running a space heater for several hours per employee, but  nobody sees it because you're just working away at your desk. Yeah. And that's a  great reference on the space heater. And that's before you factor in things like retries, embeddings, tool calls, uh multi-setup agent workflows, for example. Yeah. Those start to burn a lot of energy.

05:14
Question in a modern AI workflow might actually be dozens of model invocations behind the scenes because obviously it's reaching out to ah answer that full question completely. Right. Yeah, that's a point. So when artificial intelligence feels like it's free, it probably does feel that way for many people, what's really happening  is that the energy cost has been completely abstracted away from the user. So you're not seeing that in real time.

05:41
Yeah. And when you scale that from one power user to a thousand, you know, a business or to a million, uh, that's where the grid really, really starts to feel the impact. Yeah. Yeah. That's a point. And this is why individual usage matters in businesses, not because of your desktop setup,  uh, but because every prompt is a tiny industrial event happening somewhere else, but not in front of you. think about it that way. Yep. Yeah. So let's, uh, let's

06:10
Zoom out a little bit and take that individual power user and multiply it across a small or medium sized business. Okay. Yeah. This is where the math  starts to get uncomfortable.  And that's because AI doesn't scale linearly with headcount. It scales with usage intensity. And that was kind of how we were setting up part of that conversation at the beginning of the episode. Yeah. So in  a small medium sized business, AI isn't just

06:38
you know, one curious employee asking questions, it's embedded into the workflows. We've talked about workflows and in other episodes and agent, agentic workflows. Uh, so that's, know, customer support, chat bots, sales, enablement tools, uh, internal co-pilots, marketing content generation, analytics, you name it. And each of those systems is making continuous inference calls, not hundreds per day, but sometimes thousands. Right. Yeah. That's a good point. And, and remember from the individual example we just talked about that.

07:08
one quote unquote simple AI interaction might consume one to five watt hours as we were talking about. And that's done at the data center level. When you include GPU time, memory, networking and cooling, all of that's factored into that data center level  consumption. So if a customer support bot, like you were just talking about Scott, is handling, I don't know, 10,000 interactions per day, which is not unreasonable, you're a decent size.

07:37
organization with that customer success minded approach, you're already talking about 10 to 50 kilowatt hours per day for one workload. So that's 300 to 1500 kilowatts per month, quietly burned somewhere in a data center. Yeah.  It adds up.  the business never sees the electric meter spinning per se, because like we mentioned, it's abstracted. They just see the cloud bill. Right. And this is where a lot of business leaders

08:06
can get fooled, think, yeah, well, know, artificial intelligence, it's cheap, it's just software, but software doesn't pull electrons, the GPUs do that. So, you know, and I think this kind of leads into  what I would kind of term and it's term out there around, you know, the shadow data center reality. ah If we start talking about  larger businesses, for example, uh this is where

08:32
most of our listeners actually live. So if you're running a mid to large enterprises,  chances are you're not building artificial intelligence data centers or racking GPUs yourself, right? So that's a given. Yeah. Yeah. I think most are using  SaaS platforms.  Although a lot of our listeners still do have their own data centers. uh You're consuming in the SaaS model, you're consuming the AI through APIs.  You're embedding large language models into business workflows without

09:02
really  touching one of your own servers.  Yeah. And because of that, lot of  hear earlier conversations about racks and megawatts and think, well, that has nothing to do with us. Yeah. So if you reframe that a little bit, if you're using AI through SaaS or  LLMA APIs, you still have a data center footprint. You just don't own it, obviously. Right. We like to think of this as a shadow data center. So it exists. It consumes power.

09:32
It just lives somewhere else. It doesn't live on your campus or in your purview that you're responsible for from a CAPEX standpoint. But every AI powered feature inside a SaaS application, whether that's search,  summarization, you mentioned co-pilots before Scott, forecasting automations, is triggering inference somewhere in a hyperscale data center. So that inference is running on GPUs ah that are pulling hundreds of Watts  each.

10:01
plus memory, and let's not forget about networking, and  ultimately the cooling overhead to make sure that this doesn't overheat. So the physics don't change just because the GPU isn't yours. So that's a really important point for business leaders and also board members that feel like, we're out of the data center business. You're technically not. Yeah. So let's get back to the math, ah So take a medium-sized business, about say  5,000 employees.

10:29
So if  each employee averages just 50 AI assisted interactions per day, that's probably a conservative guess once co-pilots and agents are embedded into workflows. So with 5,000 employees, that's 250,000 AI interactions per day.  hear Doc Brown in the back of my head from Back to the Future. ah From earlier, we talked about a reasonable engineering estimate of, I think we said one to five watt.

10:57
hours per AI interaction. Once you include the GPU time, memory access, networking and cooling, just talked about. So multiply that out  and you're looking at somewhere between 250 and probably 1,250 kilowatt hours per day. So that's roughly 90 to 450 megawatt hours per year. Yeah. And like we mentioned that  in this example, that business  doesn't own a rack and a colo facility. uh

11:27
But still,  it has an energy footprint. It's just embedded  into  your pricing model for SaaS and the API usage fees.  Yeah. It makes me think of another point. another nuance that most people miss, SaaS AI can actually be less energy efficient than internal deployments. ah Because SaaS platforms are generalized by design, they have to serve every customer, right?

11:53
They also have to serve every use case and every workload shape. So that's just something else to  consider about that they're not necessarily always less energy efficient. Right.  Which means bigger models than any one enterprise might actually need higher redundancy. So don't forget about  the redundancy in the larger enterprises  and more conservative capacity planning. And all of that translates into more energy burn per interaction.

12:20
and that cost gets passed back to customers. It's just not labeled as electricity. So  you do need to consider what's going to be happening with your SaaS pricing down the road. And  I think this is where  that abstraction makes you take your eye off the ball. So when you don't see energy consumption directly, you don't plan for it because you don't have to in the cost model. So they don't model it, they don't forecast it, they don't stress test what happens when AI usage scales, because it's not the utility bill. It's just baked into the cost of AI.

12:50
As of today. Yeah. Well, meanwhile, AI usage inside the business is growing rapidly, right? We're seeing it across a lot of organizations we talked to.  You know, keep going back to having more co-pilots, more agents, more automated decision-making, which means your effective energy consumption is increasing even if your head count stays flat. Yeah. And when you consume AI via SaaS, you gain speed and convenience, but you give up the visibility, like we talked about, into energy consumption.

13:20
You give up control over performance under power constraints, predictability of long-term costs. You're relying on your SaaS partners to be concerned about power and the cost of power. True. And if energy becomes constrained, SaaS vendors are going to feel it first. And when they feel it, enterprises experience it as higher  prices, you know, so per seat uh usage caps  might see throttling or even just degraded performance.

13:48
That is the potential future problem that exists based on the energy problem that we're talking about today.  Exactly. Somebody's going to have to pay the bill. ah And this is why  AI is quietly shifting from an IT initiative to an enterprise infrastructure dependency. And when the infrastructure  is outsourced, you're still dependent upon it.

14:11
You might not run GPUs inside your walls, but your business is absolutely consuming megawatt hours of AI power every year. And that's going up. And as AI becomes mission critical, that dependency only grows. And of course, it's just going to multiply across the business world. Yeah.  And  you raised a good point. think  ignoring that doesn't make it go away. Right? So we have to be mindful of this as business leaders. Um, you know, like we mentioned, it just means  someone else controls it.

14:40
doesn't necessarily mean you shouldn't necessarily be thinking about it. And when we move into large global enterprises, that loss of control, I'd say it gets amplified even further because redundancy scale, resiliency, multiply everything we've just talked about. Yeah. Yeah. So talking about,  uh, large global enterprises and Gary, you and I do a lot of work in designing big global networks. And so as engineers designing global networks, we know this, the redundancy isn't free and

15:10
large global enterprises,  and on redundancy. you have active, active deployments and in the AI world,  uh, multi-region inference,  um,  and, and disaster recovery environments that are, that are, you know, hot, hot, not, cold. Right. I mean, every AI workload you deploy once is usually deployed two or three times for resiliency and that kind of N plus one engineering model.  Um, so if you're in one data center, you're probably another data center. Each data center has, you know, multiple

15:40
facets to it, multiple cooling units, multiple generators. So all that adds up. So if your base AI workload is consuming, we'll say one megawatt, your real footprint might be two or three megawatts. And so that's something to just keep in the back of your mind. And unlike traditional enterprise systems, AI inference doesn't idle nicely. Yeah, exactly. It's really, it's always consuming power. It's waiting, responding, it's generating. Yeah. This is where enterprises start asking questions.

16:09
They never used to ask when looking at, know, colo providers and now  even where their SaaS partners are hosted. And some of the questions might sound like this and you may be asking them today. And if not, here's, here's three that I thought about, um, can the local utility provider deliver this load?  Um, you know, I think a lot of folks have asked that over the years, but  it's a common kind of basic question. How long does.

16:35
a substation upgrade take. So if they have to upgrade that substation, are we talking months?  We talk in all kinds of regulations and approvals with the town or the state. And what happens if power is constrained? What is the backup plan? uh So many, many of you know how to build N plus one across your network and, uh you know, different tools, but  you can't just roll in a portable generator for your own consumption. Yeah. Yeah. I don't know. You and I for years, we've done, uh

17:04
location, data center designs,  and everybody always looks at the power, but now is really the time to really understand these things.  So, I mean,  at this scale, uh AI strategy becomes infrastructure, it becomes real estate strategy,  even a topic of local politics that you need to understand when you're looking at these COLO facilities. uh So as a data center tries to connect to grids or propose what can be called power islands  to power those data centers,  like uh

17:33
turbine islands or fuel cell islands to bypass the grid,  maybe even avoid impacting, you know, regional homes. So those, those things are all in play. Yeah. Those, those are all in play and good things to think about. So, so I think if we think about AI becoming this industrial load, we should probably take our focus now at the hyperscaler level. Cause we talked quite a bit on the SAS, but let's talk about a little bit more on the hyperscaler level. Yeah.

18:01
I think at this level, this is where AI stops behaving like software and starts behaving like heavy industry. Yeah. I mean, you know, if you, if you're paying attention into the news and everything else, when hyperscalers start talking about artificial intelligence, they don't talk in servers or racks. They talk in megawatts and gigawatts. So for example, a single high density AI data center today can easily be designed for a hundred to 300 megawatts of

18:30
a power draw and that's the type of data center that's being requested from utilities right now. Yeah. And to put that into context, 100 megawatts is enough electricity to power roughly 75,000 homes. A lot of, a lot of homes. That's a lot of homes. Yeah. So now scale that up. Hyperscalers aren't building one of these facilities. They're building dozens. So it's not just that one 75,000 home footprint.

18:59
Yeah. And here's an important detail or nuance. It's the training. So just the training runs of large frontier models consumes massive bursts of energy to train these models, often tens of gigawatts of hours per training run. Right. And in contrast to the occasional training run inference is continuous. Yeah. We talked about that earlier. So once a model is deployed, it runs 24 by seven.

19:27
serving millions or billions of requests if required. Yeah. And that right there turns AI into a uh base load power problem. So not just a spike problem and base load problems are exactly  what power grids struggle with. Right. So  just looking forward, let's talk about what's actually being planned because this is where the scale becomes kind of unavoidable. Sure. Yeah. You know, if you look in the United States  by itself, uh

19:57
current planning and utility interconnection requests suggest, and I had to do some research here, tens of gigawatts of new data center capacity over the next decade. So conservatively, the estimates cluster around 60 to 80 gigawatts of additional data center demand in the US by the 2030s. So much of it's driven by artificial intelligence, but to put it in perspective, that's the equivalent of adding

20:23
the electrical load to 50 to 60 million homes. So if you think about where we've come over just the last 10, 15 years, not many folks would have predicted that we would now see a complete reverse course in the power requirements needed for data centers because everybody was moving to cloud, but AI has just really just shattered that forecast. Yeah. And what are we at? 350 million residents in the U S

20:52
So we're talking about, well, you just mentioned by 2030, the electrical equivalent load of 50 to 60 million homes. That's a lot of homes. That's a lot of homes. Yeah. And globally, the numbers are even more ridiculous. So worldwide projected data center growth tied to AI is estimated in a range of 150 to 200 gigawatts over the same timeframe. So that's, over over a hundred million households. Yeah. And, and what's interesting, these aren't

21:22
abstract forecasts. These are facts based on land purchases, power interconnection requests, and utility planning documents that are out there. So it's not just numbers that Scott and I grabbed out of thin air. This is what is coming. This is where we're at. And this is what we could expect. Yeah. And really you can build a data center shell in 18 to 24 months. So that's not the problem.

21:49
You can't build generation transmission substations and permitting infrastructure that fast. No, mean grid uh upgrades,  I think they routinely take five to 10 years, maybe something longer. And so this mismatch in timelines is really the central constraint of AI. Exactly. Yeah. So that bottleneck there, that kind of brings us to the core thesis of this episode. AI isn't...

22:16
bottlenecked by models, or really even chips at this point based on what we're hearing some of the manufacturers talk about as far as their production runs. No, it's bottlenecked by power. Yeah, I was gonna say it's not bottlenecked by capital at the moment either. It's bottlenecked by energy to your point. So no power, no GPUs. Yeah, no GPUs, no inference. And no inference  obviously means there's a bottleneck to AI at scale. ah So  this is why you're seeing uh

22:45
Hyperscalers talk about some of the things we've mentioned, you know, long-term power purchase agreements,  on-site generation, like we mentioned, uh small  cell nuclear, dedicated gas plants,  you know, things like that. Yeah. And this is a challenging problem and a lot of needs  to happen to secure the ability to run AI in the next decade. ah So, you know, if we were to talk more about what we've mentioned at the beginning of the show, you know, US versus China.

23:14
So then if we were to, you know, widen the lens oh a little bit here, because what we're talking about now isn't just enterprise strategy or hyperscaler strategy. It's really national strategy. When AI becomes the energy intensive, the question of who wins starts to look, you know, really like a question of infrastructure and power. So if we start with China, for example, because the scale there is often misunderstood.  China today already operates one of the largest data center footprints in the world.

23:44
depending on how you measure it, current estimates put China at roughly 30 to 35 gigawatts of continuous data center load. So in annual terms, that's on the order of 100 to 150 terawatt hours of electricity per year, which already represents a meaningful percentage of the country's total power consumption. So that's the baseline they're operating from today, not even talking about future growth.

24:13
And where this becomes especially important is when you look at where China is headed. So based on uh announced projects, like we talked about, uh national compute initiatives  and utility level planning, China is clearly preparing for a very large expansion and compute capacity over the next decade. So even if they have lower performing chips, they just scale  up the infrastructure and the power.  And more credible projections suggest that by around 2030,

24:40
China's data center electricity consumption could roughly double pushing to 250 to 300 terawatt hours per year. So in terms of continuous load, that implies something closer to 50 or even 60 gigawatts dedicated to data centers and AI workloads alone. Yeah. So it's a lot of power. So to put that in perspective, that's like adding the power demand of an entire industrialized country.

25:10
just for compute. And the reason, you know, I'd say China can even contemplate that at scale is because of how tightly coordinated their system is. uh Energy generation, transmission, land use, and industrial policy are all kind of planned together. So when AI and computer declared strategic priorities, power infrastructure can be planned alongside them. That doesn't mean China doesn't have constraints, they absolutely do, but the execution model is fundamentally different.

25:39
It is different. So let's, let's contrast that with the United States. So today, uh, us, we actually have more installed data center capacity than China as of today. Uh, so roughly 50 to 55 gigawatts of continuous load are already operating.  Um, and like China, I were projecting very aggressive growth. Like we talked about,  um, 60 to 80 gigawatts of new data center demand by the  2030s.  Uh, like we talked about  on paper, that looks like a lot and it would

26:09
It would more than double today's footprint,  but this is where the structural differences really matter.  The US grid is fragmented, uh power generation, transmission, permitting, land use, all those things that we talked about are handled across  federal, state, and local jurisdictions. So even when the capital is actually available,  even when the chips are ready to go, projects can stall sometimes for years waiting for interconnection approvals or transmission upgrades.

26:36
And so we're already seeing data center projects delayed, not because of GPUs or financing, but because the utilities just simply can't deliver power fast enough to these geographies. Right. So the contrast isn't about ambition or innovation. The U.S.  is still unmatched in private capital, cloud platforms and AI research. ah But China's advantage at the moment isn't better models, it's speed of coordinated execution. So it's key difference.

27:06
America's advantage is dynamism and innovation.  But its risk is timeline. So it's just something to consider when we're talking about all of this. It's not that we don't want to do it. ah It's just our coordination is a little bit different. Yeah. And when AI workloads are scaling this fast,  like all the examples that we used earlier in the episode,  the timelines matter. And a delay of three to five years in power infrastructure

27:33
isn't just an inconvenience, it's a competitive constraint  on  our nation and our economy as a whole. ah So this is why energy policy and AI policy are now completely inseparable. uh The models can't run on,  you know, roadmaps or some of those big press releases we've been hearing. They run on electrons delivered through real infrastructure on schedules. Yes. ah I think  we both agree that, you know, it's over the next decade, the AI race between

28:03
US and China is going to be shaped less by who announces the biggest model and really more by who can reliably bring tens of gigawatts of stable power online,  connect it to compute and do it repeatedly.  The innovations in AI models will quickly be matched between the two countries, but it really comes down to the power. Yeah, totally agree. And the race between the US and China is increasingly a race between two very different energy and infrastructure systems.

28:33
Um, so let's, let's start to sum it up. So if you're a, if you're a CEO or CFO board member, or even in politics, AI strategy is, uh, you know, without at least energy awareness is, is incomplete. So if you consume AI entirely through SaaS and APIs, you're still dependent on the power availability, the grid stability and the energy economics of the hyperscalers. Right. And, and I can hear.

29:02
some of my clients and future clients saying, know, what should I be asking? What should I, what should be top of mind? And I think the questions business leaders should be asking now are, you know, where does our AI actually run and where are SaaS instances? uh How energy intensive is our AI usage becoming? What happens if power becomes constrained or more expensive? Just a couple, you know, questions to do your diligence on as you're preparing to

29:31
to enable those partnerships. Yep. Totally agree. And because AI costs, they don't just scale with usage, they'll also scale with  physics. Yeah. And the companies and countries that win the  AI race, uh they'll be the ones that understand power as deeply as they understood models. And that's it for the episode of the Macro AI Podcast. Please subscribe for more insights. Share us on your network. Check us out on LinkedIn. Contact us on LinkedIn if you have any questions.

30:01
or want to connect with us. We really appreciate you  attending today's episode. Thank you.