Silicon Futures for January 2026 - CES, Panther Lake, Vera Rubin, Maia 200 and more!

Silicon Futures is a neXt Curve reThink Podcast series focused on AI and semiconductor tech and the industry topics that matter.

In this episode, Leonard, Karl and Jim talk about some of the top headlines from January of 2026. 

• Silicon highlight from CES 2026
• Silicon supply chain issues of 2026
• The accelerating agentic AI disruption of 2026
• The evolution of the accelerated computing thesis
• The next big thing in AI infrastructure
• The new memory architectures for AI inference
• AI infrastructure starts off on a different foot in 2026
• The edge AI movement in 2026
• The CPU picture for 2026

Hit Leonard, Karl, and Jim up on LinkedIn and take part in their industry and tech insights.

Check out Jim and his research at Tirias Research at www.tiriasresearch.com.
Check out Karl and his research at Cambrian AI Research LLC at www.cambrian-ai.com. Check out Karl's Substack at: https://substack.com/@karlfreund429026

Please subscribe to our podcast which will be featured on the neXt Curve YouTube Channel. Check out the audio version on BuzzSprout or find us on your favorite Podcast platform.

Also, subscribe to the neXt Curve research portal at www.next-curve.com and our Substack (https://substack.com/@nextcurve) for the tech and industry insights that matter.

NOTE: The transcript is AI-generated and will contain errors.

DISCLAIMER: This podcast is for informational purposes only.

Transcript

Copy Transcript Copied

0:06

Next curve.

Leonard Lee 0:09

Hey everyone. Welcome to, uh, this episode of Next Curve's Rethink podcast, EPIs, um, episode, uh, we break down the latest tech and industry events and happenings in the world of semiconductors and Carl's favorite topic, ai.

Karl Freund 0:26

Ai.

Leonard Lee 0:27

Yeah. And s favorite

Karl Freund 0:29

two logos. We got two logos. Jim beat that.

Leonard Lee 0:31

Yeah, he's beating you, man. You gotta catch up. I have thing, I, I've

Jim McGregor 0:36

got one that's got a whole background just full of logos

Karl Freund 0:39

Let's see it.

Leonard Lee 0:40

Yeah. And also, maybe at some point quantum, We're distilling all this stuff into the inside stat. Hey,

Jim McGregor 0:47

why stop there? Let's go to neuromorphic. Come on.

Leonard Lee 0:49

Geez.

Karl Freund 0:51

I The around when neuromorphic finally happens, that's way future.

Leonard Lee 0:57

Yeah. Well. Some people argue it's already happening, but, I'm Leonard Lee, executive Analyst at Ncur, and I'm joined by some very, very good friends and fellow, tech industry colleagues. first off, the inference optimized. Oh, you like that one? Jim? Jim McGregor of Fame. Serious research.

Jim McGregor 1:20

I'm not real. MAI. Yeah,

Leonard Lee 1:24

I believe it. And then of course, Carl Fe of Cambrian AI Quantum, and eventually neuromorphic research. How are you gentlemen? Doing? Oh my. We have so much to cover. it's insane. And this year is already starting off as being one that will be very busy for us, no doubt, right?

Jim McGregor 1:52

Oh yeah.

Leonard Lee 1:53

Okay. before we get started, remember to like, share and comment on this episode and subscribe to the, rethink podcast here on YouTube and on buzzsprout. And, this podcast is for informational purposes only. Statements made by my guests are their own and don't reflect those of next curve. we just wanna provide a forum for debate and discussion on tech and the semiconductor and AI industry. I think we have to start with CES and yes, CES, it was that long ago.

Jim McGregor 2:25

I don't remember. CES That was too long ago. That, that was months ago.

Leonard Lee 2:31

Yeah. We had a CES and there was a lot of announcements, right? I think like the biggest

Silicon highlights from CES 2026

Leonard Lee 2:36

one was probably Intel. with the launch of, Panther Lake, right.

Jim McGregor 2:43

For the PCs, I would say that was the, the PCs. Yeah. there, I mean, the show, I, I would actually say some of the biggest announcements of the show were data center, you know, where we got a view of VE Rubin, from, Nvidia as well as the MI 400 series from, a MD. and quite honestly, the underlying theme of the entire show was robotics, robotics, robotics, robotics.

Leonard Lee 3:06

Yeah, robotics, largely humanoid, right?

Jim McGregor 3:10

Largely humanoid, but seeing everyone get on the bandwagon. And especially, and I think this is a key part that's coming out, is those companies that really led in autonomous vehicle technology are probably going to be the leaders in, robotic technology, especially humanoid robot. We already saw that with, Qualcomm looking at the show, they're IQ 10, so a brand new part just for robots. And they came out with, leveraging all that automotive technology. They came out with a demo in just four months, a working demo at the show in just four months. But Nvidia. Nvidia is definitely the, just ai, they are definitely the leader in that segment with just so many technologies and so many, from anything from the silicon all the way up to the models and new models being introduced for different types of ai, the show.

Leonard Lee 4:01

Yeah, they intro, what was the name of the, um, automotive, model that they introduced? It's, Anna or something like that. do you remember?

Jim McGregor 4:10

It's, ALPA Mayo.

Leonard Lee 4:11

There you go. Alpha, alpha Mayo. Yeah, there you go. yeah, they announced that. Right. but that was, that's less for the robotics, more for like the in cabin type of AI experience.

Jim McGregor 4:24

is true. Because, They already have the drive platform.

Leonard Lee 4:27

Yeah.

Jim McGregor 4:27

for the command and control systems.

Leonard Lee 4:29

Yeah. it was interesting to see the West Hall largely, absent of a lot of the big name automotive companies that used to be there. And, less of a focus on a DAS

Jim McGregor 4:45

of industrial stuff. A lot of industrial stuff from the likes of, John Deere and Caterpillar and Case and stuff like that.

Leonard Lee 4:52

Right. And, pivot toward cars as a service, vehicles as a service, right? Yeah. I don't think I've ever seen Waymo with as large of, a presence as they had this year. And then of course, Amazon's Zoosk, that's their own, I would call it more of a, autonomous shuttle. Yeah, they also were running a few of those up and down the strip. It seems like the West Hall is evolving, right?

Jim McGregor 5:19

And I wasn't impressed with a lot of the automotive stuff 'cause a lot we're still waiting for a lot of that stuff to come out. So you're li you're right. A lot of it was shifting towards the services. However, there was one company that still continues to amaze me and that's Verge Motorcycles. Now Verge is actually three companies. 'cause they have the donut labs, which actually develop the motor. They've got this round motor that goes inside a wheel that can be on a, a motorcycle or a car. They've also got. I think it's called Watt. They're battery technology company and they were introducing at the show, they were talking about having, they are in production with solid state batteries for the motorcycle and for future, other future transportation solutions. So I was very impressed with that. I've always been impressed with the design and everything else. Because it's one of the few electric motorcycles I think that will actually feel like an electric motorcycle. But the fact that they're in production with solid state batteries, which, everybody's been, trying to get to for automotive industry, I was very impressed with that.

Leonard Lee 6:18

These are not new things, they've been on an evolutionary path for quite some time. going back to the AI pc, so we had Panther Lake, right? Yes. there seems to be a lot of excitement about 18 a and from the sounds of it. You know, some of the OEM customers that I spoke with, they seem to be pretty happy with the core Ultra series three. portfolio, I think they introduced like, nine X and something else. There was eight x,

Jim McGregor 6:47

Carl and Carl and I actually toured, fab 52, late last year, which is producing Panther Lake and the future Eons on 18 A. So they are in full production now. they haven't announced landing any customers for 18 a and quite honestly, they were kind of. Late with the PDK to get a lot of the leading edge customers, but you have to remember that 18 a is probably gonna be a 10 or 20 year process node. So there's still gonna be a lot of customers that may not be on that leading edge. They may not be the, AI accelerators. They may not be the server chips. They may not be the handsets, but there's. Everything else that may be able to leverage that technology. So, there's a lot of excitement still, and the fact that they're in production with the world's most advanced process node, a lot of people are taking note. Yeah. And they're introducing the PDK for 14 A.

Leonard Lee 7:38

Yeah.

Jim McGregor 7:38

So that's definitely gonna help them maybe get on that leading edge.

Leonard Lee 7:42

Yeah. And based on the earning call Late in, January, it looks like yields are still kind of an issue. but it looks like they've nailed at least the technology, you know, introducing backside, power as well. As, the new, get all around fat. architecture. I think that's the thing that everyone has to appreciate this. There's always gonna be a really tough one for Intel to deliver on.

Jim McGregor 8:07

that product. Well that then the fact that it goes hand in hand with the packaging, the advanced packaging technology. Yeah. And that's really where Intel has its true leadership right now. Matter of fact, I'm gonna be touring their advanced packaging facility out in Rio Rancho, New Mexico, uh, on the 24th of this month.

Leonard Lee 8:24

Yeah. And e MIB tea looks really, really interesting.

Jim McGregor 8:29

Yes.

Leonard Lee 8:30

You know,

Jim McGregor 8:30

well, even the stuff they're doing in Chandler with class substrates is very interesting too.

Leonard Lee 8:35

So a lot of interesting things happening at Intel actually some bright spots. Do you know what I'm saying? This year, I was at the Qualcomm vent. during Media Day and they introduced the Snapdragon, X two plus, right? Yes. So that's the, let's say mid-tier offering. And so, Qualcomm's looking to expand its A IPC base. You know, so that was kind of cool. and it was something that a lot of folks talked about,

Karl Freund 9:04

you know, speaking of Qualcomm, we might wanna mention they had their earnings yesterday.

Jim McGregor 9:09

Yes.

Karl Freund 9:09

And market not happy. Well, I think a lot of their projected under performance expectations is really due to the me global me issue.

Jim McGregor 9:18

it is. And quite honestly, that's one of the topics we really need to cover here is the fact that they're just the first to really highlight that. Yeah. I think that we're gonna see that from semiconductor companies, through system companies, through even possibly dealers

Karl Freund 9:31

and hand.

Jim McGregor 9:32

Yeah, 2026 is definitely the year of capacity constraints, whether it memory's the big one. But also, manufacturing. I mean, there's a lot of companies looking to leverage, tsmc three nanometer and two nanometer. That's gonna be a challenge for them in terms of capacity. And, you know, and intel still ramping its stuff. So it's, there's gonna be a challenge there. And if that's not enough, geopolitics could actually throw capacity issues in the mix with, rare earth materials.

Leonard Lee 10:00

Yeah. And you know, this is the thing that I think we have to watch out for there. Right now there's this, shift in the, mobile memory, right? Lp DR to HBM. Right? So that's why everyone's concerned.

Jim McGregor 10:19

and Lp, DDR. So even the servers are now going to lp, DDR,

Karl Freund 10:23

that, that's next, next generation, Vera Rubbin, with, for a large contact Windows will be L-P-T-T-R.

Leonard Lee 10:30

Everyone's talking about HBM because everyone still thinks AI infrastructure is, that reference design, the Blackwell and the hopper.

Karl Freund 10:43

you know, I agree. But I talk to a lot of startups that are really trying to address the memory wall that that is creating. Because you don't have enough capacity

Leonard Lee 10:51

Exactly

Karl Freund 10:52

for these new large models, large contact windows, using h HP m. So yeah, people are coming up with some wild stuff. I was talking to Majestic this week and they've got a completely different approach.

Jim McGregor 11:04

yeah. No, it's gonna be a challenge. I mean, with, the big three micro and SK Hynes and, Samsung. All kind of abandoning, DDR four to go to DDR five and HBM, especially HBM, they're leaving everybody in a pattern that we've never seen before. Typically, when memories constrained, everyone starts double or triple ordering. We actually saw the end of last year people canceling orders because the price was too high, especially for DDR four when it's going up four x or more.

Leonard Lee 11:34

Yeah. And then we're also seeing, I heard a lot of talk about the increased demand in NAND as well.

Jim McGregor 11:39

Yes.

Leonard Lee 11:40

And so, there's constraints all across the board, but I think, you know, to the point that, we just raised Carl about the memory, you know, the kind of memory that's going to be driven by AI infrastructure. it may be the types that support. supercomputing or whatever it is that you're sticking in your

Karl Freund 11:59

there, I'm a big perplexity fan. and perplexity never ceases to amaze me in how it knows me. It knows how old I am. It knows what business I'm in. It knows where I live or where I'm moving. And 'cause once you tell something, it remembers it. Well that starts all sort n

Jim McGregor 12:16

Yeah.

Karl Freund 12:16

Yeah, they all do that now. So that personalization. Capability that's coming online. all the major chat models that required mant and a lot of it.

Leonard Lee 12:27

Yeah.

Karl Freund 12:28

IM not gonna sit around and wait for answers. Right?

Leonard Lee 12:31

Yeah. And so that is another topic that is, starting to become a big one, which is the long memory right. Of these AI systems. And so, I think the mindset of, Observers of the industry are still maybe about a year off.

Karl Freund 12:50

Yeah.

Leonard Lee 12:50

We're actually going this year where, you know, last year the talk was all about the memory wall and HBM, but coming into this year, the inference stuff looks very different.

Jim McGregor 13:02

Mm-hmm.

Karl Freund 13:03

And it's interesting to see Google's earnings yesterday, right? Because Google came out and said, Hey, you know that 95. Is that right? $95 billion?

Jim McGregor 13:13

Yeah.

Karl Freund 13:13

Yeah, we're not gonna do that. We're gonna do $160 billion in infrastructure investment. So Google stock goes down and everybody else's stock that supplies and goes up, and the industry is just churning right now from one supply and demand crisis to another.

Leonard Lee 13:29

Yeah,

Jim McGregor 13:30

there, there's also been a bit of a wake up call that, has come across recently. and I'm trying to remember who, I think it was open, was it Open AI that was talking about it, but just the fact that these AI. Capabilities, AI bots can replace a lot of the other applications that are out there. Whether it's, the productivity apps like Word and Excel, or whether it's, for purchasing. Matter of fact, humane ai, that's exactly what their CEO is pr pro promoting when he was talking to us at the Snapdragon Summit was, listen, every function in my company should be one person, an ai. And, we define what it has to do and then we go and we program it. And the fact is that, AI may lead not just to, increasing productivity for each individual, but actually customization capabilities for every function.

Leonard Lee 14:21

Yeah. I'm not bought as bought into that whole, talk tracker thesis.

Jim McGregor 14:26

cause you're not a programmer.

Leonard Lee 14:28

Yeah, no. I used to be a programmer. the problem there is it's this thing called the Brownfield, and that a lot of these agents have to interface with, older systems. And also we're completely discounting the value of the older systems. I mean, look at IBM's result. There's these series. Did really well, and I really don't buy the notion that it was all ai. If you look at the banks, they do a lot of transactional, their transactional business is actually going up. Yeah. These older systems do their job extremely well, you to do these things. And so I think this is the prevailing misconception out there. I think it was really, spurred by this idea that accelerated computing is gonna take over everything. And you're gonna throw all the old stuff away that never, ever, ever happens.

Jim McGregor 15:23

you do realize that most of IBM's investment now is in ai, right?

Karl Freund 15:28

Yeah.

Leonard Lee 15:29

Well,

Karl Freund 15:29

and in fact, the extension of mainframe systems with AI built in and augmented with additional AI parts is what is enabling banks to do real time, scoring on credit card transactions using ai. and so that is a very, productive synthesis of these two technologies. And we're gonna see the same thing happen with quantum.

Leonard Lee 15:52

sure.

Karl Freund 15:53

Where we're building quantum systems adjacent to GPU based systems. Mm-hmm. Nvidia and others have been talking about this for a while, and it's gonna be, it's gonna be a major shift.

Jim McGregor 16:06

that was the biggest discussion point at sc back in November. Supercomputing back in November was, how are we gonna have these hybrid data centers that have, the CPUs, the NPUs, and the qps all operating simultaneously and with each other.

Leonard Lee 16:20

Yeah. Just going back to CES really quick, one of the things I thought was a really big deal, and I think maybe a lot of folks missed this was. NVIDIA's announcement that they're partnering with Lenovo for the Vera Rubin, MVL 1 44. Uh. Liquid. Cool.

Karl Freund 16:39

I liquid. Cool.

Leonard Lee 16:39

warm, warm water cooling, so it's their Neptune cooling, system and, technology. And this is Lenovo specifically and that was super interesting. Like what we had talked about, I think about a year ago regarding cooling systems. 'cause that that's when, you know, about a year ago is when all the water cooling stuff really became. A big topic.

Jim McGregor 17:02

Well, that's when we found out that Blackwell just wasn't going to work without liquid cooling.

Leonard Lee 17:06

Yeah, yeah,

Karl Freund 17:08

exactly.

Leonard Lee 17:09

But yeah, a year later now we see, Nvidia now pivoting toward this other technology and it is different because it uses, you know, near room temperature water instead of. Cooled or chilled water. And that has a huge impact in terms of, you know, the requirements for cooling, right? Mm-hmm. the cooling, the chillers and all that stuff. and so, you know, what we saw at, hot chips. Last year, some of the topics around the innovations around cooling systems as well as, you know, the fancy little cold plates and stuff like that. that's becoming an area of innovation that's driving, some differentiation system and the architecture, right? Yeah. So it, I think, this year also is. Going to be an interesting one as we look at the data center facilities and how those are architected. But it's moving now back toward, you know, these, first generation AI infrastructure cooling, architecture sort of data center now back toward traditional. Because that's one of the benefits of the, Neptune Cooling Systems, is that you can deploy these in a more, you know, a more traditional, data center, configuration. and with the cooling architectures,

Karl Freund 18:30

I think it's a big, it's amazing watching the, you're exactly right. And it was amazing at Super computing this year. Last year now, it looked more like a plumbing show.

Jim McGregor 18:40

Oh yeah.

Karl Freund 18:40

I mean it was, there was these big honking pipes

Jim McGregor 18:45

Yeah.

Karl Freund 18:46

And massive cooling systems.

Jim McGregor 18:48

from Verve, they're from Vertiv, flex Schneider, all kinds of companies. Yeah. Schneider, it was, everyone is focused on that.

Karl Freund 18:57

It was just amazing.

Jim McGregor 18:58

I think liquid cooling is a key part of it, but I think the biggest change we're gonna see in 2026 isn't necessarily from the cooling. It's gonna be from how we architect data centers. we can't afford, two years at a minimum to four years data center. It's gotta be. it's, it has to be industrialized. So we're seeing the entire industry take note of what happened to the automotive industry a hundred years ago and saying, we need to be able to manufacture these things on a production line and then ship 'em to wherever they need to be and be able to mod, have a modular solution. Whether it goes in a building or whether it's a building itself, to be able to. put these things together very quickly.

Leonard Lee 19:37

Yeah.

Jim McGregor 19:37

So I think we're in a brand new industrial revolution of data centers.

Leonard Lee 19:42

Oh, geez.

Karl Freund 19:43

Agree.

Leonard Lee 19:44

Wow. Yeah, and we got a taste of that in, com text last year. Right, Jim? When we were there. Bit Vertiv, I think. Yeah. Showcasing a lot of the modular, Data center units.

Jim McGregor 19:54

actually IV probably had the best display there. They had a full center display

Karl Freund 20:00

through it.

Jim McGregor 20:01

Yeah, it was incredible.

Leonard Lee 20:02

Yeah. And one of the things I heard a I've been hearing just a lot of lately, and this is probably, this is, something that you and I talked about, Carl, but I would love to get, Jim's reaction to this is SRAM just keeps popping up everywhere.

Karl Freund 20:22

Yeah.

Leonard Lee 20:23

it sounds like a really boring thing, but it's actually a pretty big deal, right?

Karl Freund 20:27

You're seeing people develop SRAM entire SRAM chips, and it's kind of turning it on its head instead of A-C-P-U-A-G-P-U with a whole bunch of. HBM. How about a farm of SRAM surrounded by processing units all sharing the same pool of sram? That's a radical concept, and it really does turn the entire compute architecture on its head.

Leonard Lee 20:53

Yeah.

Jim McGregor 20:53

As we get to this densification of compute, memory is a critical part of that. So whether it's in memory compute, near memory compute or whatever, solution you're trying to create, it becomes part of that. We have to remember that. it's the memory, it's the computing, it's the acceleration, it's the io. All of that has to be very, very closely tied together. it's gonna be an interesting time. It's an innovative time. we're going through a period of innovation that I don't think we've seen for 30 years.

Karl Freund 21:23

Never seen since transistors took over. I mean,

Jim McGregor 21:26

okay. 40 years,

Karl Freund 21:27

four

Jim McGregor 21:27

50 years, maybe. I'm dating myself now.

Karl Freund 21:31

yeah. Still a youngster.

Jim McGregor 21:33

I worked at Intel back then.

Leonard Lee 21:37

You worked everywhere, man. think about how, different of a place we are today than the same time last year.

Jim McGregor 21:46

Yeah,

Leonard Lee 21:47

right. I mean, the same time last year it was all Nvidia. It was all about Cuda. Uh, it was all about, GB 200, 300 MVL 72.

Karl Freund 22:01

And it was all about

Leonard Lee 22:02

network. A unified, and literally singular roadmap. Now we're seeing works. All over the place.

Jim McGregor 22:08

Well, you know, we're seeing these fork through a lot of interesting acquisitions. Nvidia acquiring the assets of gr. Yeah. So now that they've got a new type of accelerator technology We see Qualcomm acquiring, Alpha Wave. I was thinking of the other one that actually does risk five. That does risk five based, they're targeting that towards the data center, which they talked about on the earnings call yesterday. So yeah, we're seeing a lot of innovation, a lot of change. and you're right, it's not just all about Blackwell now,

Leonard Lee 22:36

even in Fabrica call, right? Remember. Talked about Infa.

Karl Freund 22:43

Nvidia bought em

Jim McGregor 22:46

and this is going to be the year of co-pack optics. You know, we're gonna see a lot of that coming to production this year to where we're going to be accelerating everything from the chip through, through light, through optics.

Leonard Lee 22:59

Yeah. I know Hot accelerate a lot.

Karl Freund 23:03

One of the hottest companies in the world right now is Momentum.

Jim McGregor 23:06

Momentum

Karl Freund 23:07

Ment is the hottest company of the year so far on the s and p 500.

Leonard Lee 23:11

That's great.

Karl Freund 23:11

And they're just going through the roof because everything's going to optical. Yeah. Maybe people get a lot over excited about it, but it's gonna happen.

Leonard Lee 23:18

Yeah.

Karl Freund 23:18

it's gonna be a big deal.

Leonard Lee 23:20

Yeah. What I heard from, the Marvell guys was. they're trying to, extend. Copper a bit more. Co packaged copper. So CPC that they were touting I guess we'll have to see. There's a lot of optimism. There's a huge push for co packaged. Optics and optical in general. but then, maybe Jensen was right. Copper has to have a couple more cycles before you ditch it. one of the things that happened last year that I think kind of spurred all of this was, huawe introducing there. Cloud matrix, right? they're largely using, optical for that system. But, coming into this year, it's interesting to see the diversity, that is being spurred by, inference. I don't know what you guys think. I think it's turning out to be a completely different beast than, people

Karl Freund 24:13

thought. So I think you're right. And speaking of that, I'd love to get your perspectives, Jim and Leonard on cereus and wafer scale computing at large. cereus announced, I think it was yesterday, a $23 billion, valuation. based on their latest a hundred million dollar funding. round they just did with Tiger Capital and others. And I keep asking myself, and in all honesty, Cerebra is a client. I keep asking myself, what's everybody else doing? Either Cereus is right and everybody else is wrong, or Cereus is wrong and they're not worth 23 billion. I'd be interested in your perspectives. Are we gonna see other way for scale implementations?

Jim McGregor 25:04

I don't know. first off, what Cerebra has done is phenomenal. The engineering feat of creating a wafer scale solution, but there's other ways to skin the cat. Through dye stacking, through multichip modules, through everything else. And, it's gonna be interesting to see, all of these technologies, I think are on their, the beginning of their curve. So we don't know how far they're gonna go at this point in time. and it depends on what you're trying to do. Are you trying to run a massive model? Are you trying to run a bunch of, millions of smaller models? And which is gonna be more efficient depending on a. The type of application you're running. I think Cereus has found a great niche, and the build out, they're one of the few AI startups that successfully transitioned to becoming a service provider and that threw a huge investment from, the Middle East to be able to build out data centers to do that. It's really gonna come down to eventually of that ROI, which is what the financial community has really been questioning as of late, service providers, we are not quite sure what that ROA is gonna look like. You can build up a lot of clients, you can sell a lot of data, you can, or a lot of data services if you tank the price or you basically give the services away, but when are you going to make money and how are you going to make money? And I think that's still gonna be the challenge.

Karl Freund 26:21

Well, interesting to see how open ai. Fares with their Exactly. Deployment of Cereus, that's gonna be the test case. 'cause you're right Mo most of their very, really large customers are in the Middle East. And this was the first time I've seen a global AI service provider, adopt cereus at scale, at significant scale. So we'll see if that starts the, the trend you're talking about. Jim, what, what do you think LA as

Leonard Lee 26:43

No, I think

Karl Freund 26:44

good.

Leonard Lee 26:44

Yeah, I mean, I think. We will have to wait out this year to see what the scale out application is. So if it's, large language model inference, and that architecture, continues to demonstrate. differentiation, right? Just performance that can't be denied. Then I think then there's gonna be entertainment of that architecture. and I think you're making a really good point about this because I mean, I, I wanna talk about Microsoft's Maya 200 announcement really quickly, and I think it represents This movement amongst the hyperscalers with their custom silicon moving toward a transitionary architecture. and maybe moving more toward like what you're talking about with wafer scale and alternative architectures. 'cause if you look at what, Microsoft's Maya 200, it looks a lot like your conventional training. micro architecture as well as system, right? The way it's networked. although we can talk about the networking a a little bit later. It is different. it kind of goes off of that, GB 200, H 100 kind of, Format, right? Or design, but they start to inject or integrate SRAM in there for inference optimization, right? they're slowly changing the memory architecture. But I think this is transitionary. I think, with the introduction of CPX, when that comes out, that's gonna change the game. And all these guys who are the hyperscalers, who really are going to need very power, efficient, highly scalable inference, compute, they're probably going to, make that transition. Towards something, either closer to a cereus or some of these alternative, architectures that use novel networking and interconnect, right? And so if you look at what they did with Maya 200, what do they have like, they have an integrated, Nick, right? And they have this two tier, interconnect. this is really interesting because there's a recognition that inference is different and it is curious that they can scale up to something like 6,000, in their scale up domain. Uh, it's like 6,000 or something like that, right? 6,000 accelerators.

Jim McGregor 29:07

I like how you say inference is different. Every type of inference is different. Every model is different. every data set is starting out

Leonard Lee 29:15

be

Jim McGregor 29:15

Yeah.

Karl Freund 29:16

Gen AI as well.

Jim McGregor 29:18

that's the challenge. and if that wasn't, complex enough, we're moving to a world of mixture of experts,

Karl Freund 29:25

right?

Jim McGregor 29:25

Where all these things are working together, to be intelligent. And we haven't really gotten to what I call. collective intelligence where you've got competing solutions that have to figure out who has the right of way or who has control.

Leonard Lee 29:40

Talking about Malt book.

Jim McGregor 29:41

So it's gonna be interesting. I would agree with you. There's a lot of room. For innovation. There's a lot of change going on and there's gonna be room for all these architectures because, well, just like in the data center, we've never had a single CPU architecture. 'cause no two workloads are the same. It's the same for ai. Matter of fact, it's, it's even bigger with ai,

Karl Freund 30:02

you're right. I think we'll end up with a mass specialized CGG accelerators, not GPUs, massive specialized accelerators for different. Kinds of models. And if you want something that runs everything, you're gonna use Nvidia. Right. At this one thing, you're gonna go to a company that can provide a really good solution for that one model.

Leonard Lee 30:23

Yeah. I mean, I think. Nvidia has the advantage when it comes to inference. It's very obvious that they recognize that this, this is a different animal, right? And Cuda doesn't save 'em. this is where, to your point Jim, the diversity, what, what Optum optimal looks like in the diversifying inference, landscape. It is actually looks a lot more like Edge ai, like what I saw, edge ai, so with CES, you know, no one talks about GPU. you know, when you're talking with the Edge AI community, depending on how you define that and think of it, but it's M-P-U-D-P-U right. And maybe some FPGA. So I was at MD and m west, which is a medical manufacturing, conference. And I was talking to some folks that were doing, computer vision stuff and they were using FPGAs. I was like, kind of surprised. I thought they were using MP or DPU, but,

Jim McGregor 31:21

you mean DSP.

Leonard Lee 31:23

Or D DSP. Right, right, right. DSPs, right.

Jim McGregor 31:28

DP is a whole nother animal.

Leonard Lee 31:29

Yeah, yeah. Different animal. And then when we look at inference, we are looking at NPUs, DSPs and these asics that are application specific.

Jim McGregor 31:41

And not even that, even CPUs because a lot of the CPU architectures are integrating the ability to do inference processing.

Leonard Lee 31:48

Yeah.

Jim McGregor 31:48

it's a logic element that's designed to run, an AI workload of some type.

Leonard Lee 31:53

Actually, I think CPUs are coming back when you look at the a MD numbers, right? for the past few quarters, they've been putting, epic front and center. It's not necessarily all because of ai, right? I think, there's a silent, data center refresh that's happening with traditional compute. and we hear a lot from the OEMs as well. They're getting their margins and increased volumes in their data center businesses, through, a lot of the traditional, data center compute. So, but um, yeah. Um, wow. That was great. Carl likes to keep it short and sweet, but unfortunately,

Jim McGregor 32:31

there's just too much to talk about.

Leonard Lee 32:33

There's too much to talk about. What do you do? I mean, this is gonna be year,

Jim McGregor 32:37

this is gonna be an interesting year. I mean, between the elections, the geopolitics, the capacity constraints, the, innovation. The innovation. Yeah, just the innovation. It's gonna be phenomenal.

Leonard Lee 32:49

Yeah, it will be an adventure to be sure. So, hey everyone. Thank you for tuning in, gentlemen. Thank you so much. make sure that you reach out to Carl at Cambrian AI research@www.cambrianai.com. He's on Substack, he's on Forbes, he's everywhere. He's Mr. Ai. And then of course, reach out to our good friend here. Jim McGregor of TEUs research@www.teusresearch.com. And also, please subscribe to our podcast, which is featured on the next cur YouTube channel. Check out the audio version on Buzz Sprout or find us on your favorite. Podcast platform. Also subscribe to the next curve research portal@www.next curve.com for the tech and industry insights that matter, and all the semiconductor chip and AI goodness that you could ever ingest in less than an hour. Gentlemen, take care.

Jim McGregor 33:59

Take care.

Karl Freund 34:00

Take care.