Qualcomm's HBC Memory, Alphawave, Modular, and more

Qualcomm flew Austin to New York for its investor day, where the communications company laid out a plan to make data center, automotive, and IoT two-thirds of its business by FY29. Austin was in the room and asked Cristiano Amon a question on the record. Vik watched the whole thing on YouTube. Together they break down what actually changed.

The technical centerpiece is High Bandwidth Compute (HBC): stacking LPDDR on top of logic to expose the whole face of the chip for interconnects, claiming up to 100x more lanes and a path around the HBM bandwidth bottleneck. Austin and Vik dig into what's really under that memory, why "no advanced packaging needed" just moves the hard problem somewhere worse, and how it compares to d-Matrix. And as Austin put it: don't let anyone tell you HBM is dead. MOAR memory.

Then the roadmap: the AI200/250/300 accelerators, the C1000 server CPU (5 GHz, 250+ cores, Meta as a customer), the Alphawave and Modular acquisitions, and Chris Lattner's Mojo.

The most interesting takeaway might not be the data center at all. Qualcomm's edge play — AI-defined vehicles, cars as token generators, and a $1T robotics opportunity by 2040 — could be where High Bandwidth Compute matters most. 

Chapters:
 0:00 Communications? That's just the start
 4:08 Inside Qualcomm's investor day
 9:16 Can Qualcomm build a data center business?
 13:09 Disaggregated inference opens the door
 17:57 High Bandwidth Compute: memory on the XPU
 30:29 "No advanced packaging" just moves the problem
 36:20 The roadmap, Alphawave, and Modular
 46:00 The C1000 CPU and the agentic shortage
 50:40 Cars as token generators, the $1T robotics bet
 57:32 The memory market: MOAR

Follow Semi Doped:
 Get more of Austin and Vik daily, free!
 Sign up: https://www.semidoped.com/

Connect with Vik and Austin:
 Vik's Paid Substack: https://www.viksnewsletter.com
Austin's Paid Substack: https://www.chipstrat.com

Transcript

SPEAKER_02 0:00

Qualcomm's main point is that they're diversifying their businesses and the majority of their business in the long run will be non-communications. And I thought this was interesting because the name of the company and the legacy of the company is communications, quality communications, Qualcomm. But the crazy thing is the data center business could become so big and it's in it could inflect so much that what Qualcomm makes from their new businesses, data center and automotive, could ultimately dwarf what they make and have made in the aggregate of the company's history as a communications player. So that's something big to wrap your head around, which is just Qualcomm being a communications player could just be the start of their story. All right, hello, welcome listeners. Welcome to another semi-dope podcast. I'm Austin Lines with Chipstrat, and with me is Vic from Vic's newsletter. Hey Vic, what's up, man? It's been a while since we chatted.

SPEAKER_00 0:59

Yeah, it's been a while. Uh we I've been traveling, you've been traveling. Finally, we're back at the home studio. So it's time to do something now. Totally, totally. So you were in the UK and it was hot? It was crazy. I thought I ran away from India to get away from the heat and feel some cool air in UK because everybody says, oh, it rains all the time. So I was hoping to get like rained upon a little bit. But anybody listening from the UK will be like, no, no, you'd never wish for that stuff. Don't do it. You know, we like so a little bit of sun once in a while. I know it's like a lot of sun right now in Europe, but it's okay, baby, for a for a week or two.

SPEAKER_02 1:36

Yeah, totally. Yeah, I saw I saw it was like 110 degrees Fahrenheit or something in France. Just like like when you said it was hot, I was like, oh yeah, it's like UK hot. But it's like, no, dude, it was hot, hot.

SPEAKER_00 1:47

It was hot, hot. And you know the other thing is in the UK, they're not just equipped to deal with this stuff. Because in India, like either there's AC some in some places. If not, if you go by buses, for example, there aren't like windows, sometimes there are no windows, okay? So you always feel the breeze. It's not cool breeze, but at least there's movement of air. Sure. In the buses in the UK, it's just like sealed, so there's no air movement, the windows don't like open out or slide out or anything like that. And you're just trapped, and you just have a bus full of people just like sweating. It's a sweat lodge.

SPEAKER_02 2:24

Just suffocating, yes, brutal. Well, um, I was in New York City and Qualcomm flew me out for their investor day event, and the weather was really nice in in New York City. It was very beautiful. I woke up early. Obviously, the sun's out for a long time right now. I ran in Central Park. Um, I ran one day to the west side along like the Hudson River. Um, I saw crazy huge yachts. I have never seen such big boats in my life. I don't know if it's like someone who lives there's yacht or if it had to do with the World Cup because there's a bunch of people in town for the World Cup. But it was like, I was like, dude, you need a whole staff to run this thing. Like, this is crazy. And here it is, and I'm just right by. I was gonna take a selfie in front of it, but I thought, like, I don't know. That seems kind of touristy.

SPEAKER_00 3:11

I mean, all the investment bankers live there, right? Like half the people who listen to our show must be their yachts. Maybe they should let us know.

SPEAKER_02 3:18

That's too much and you saw our yacht. Live semi-doped podcast from your yacht. You send it to me. It's Austin at chipstrat.com, send me an email and we can chat. Okay, okay.

SPEAKER_00 3:32

So wait, wait, you were at uh in New York for uh the Qualcomm Investor Day then.

SPEAKER_02 3:38

Yes, yes, Qualcomm Investor Day. I was there. Obviously, uh, you know, they they hosted in New York City because the audience is the sell side analyst in the financial community, but they brought out industry analysts as well because they had a lot to talk about. So uh really the story that we'll walk through, and I want to talk through the technical bits and hear your take on some of it. Um, the story that Cristiano Almon and Qualcomm were trying to communicate is was all about their diversification, which uh away from handsets, which ultimately um Qualcomm has always been, you know, a communications company, big smartphone business. Everyone is seeing the writing on the wall with smartphones, which is just we're at the top of the S curve. Everyone has a smartphone, right? Um there's lots of competition in the space, and so on and so forth. Um, Qualcomm has been moving into automotive, and that business has been growing. And it's a proof point that, hey, they can use MA to get into a new business and then build the technology, use their channel, use their manufacturing scale to grow that business and take it to market. And the whole point of this was Qualcomm trying to say, we are going to do that with data center as well. And here's our plan. And so ultimately, I there was, we'll include this the image, but like they had this sort of one money slide. I think it was from the CFO's presentation, where there was like three circles. It was like a donut ring slide, and they were showing like, okay, and in fiscal year 25, handsets were two-thirds of our business, automotive and IoT is one-third. Our goal for 2027 is like roughly half, half and half, half handsets, half other, which now starts to include data center. And then fiscal year 29, the goal is actually two-thirds of the business is automotive IoT and data center, and only one third is handset. So they're trying to move sort of beyond their legacy communications only business and move into these other new businesses. Which last point, and then I'll let you chime in. Obviously, communications is part of all of these businesses. So it's there the the IP and the technology is still important to the other businesses. It's just that they're gonna have to build more and do more than just handsets.

SPEAKER_00 5:53

So I I saw the whole investor day talk. Uh I wasn't there, but I saw it on YouTube. It was interesting because Cristiano Amon said that for the 40th anniversary of Qualcomm, uh, you know, they had the founder over Irving Jacobs. And uh during that conversation or the talk that he gave, uh, he mentioned that he made one mistake in naming the company Quality Communications. He's like, I should have just stopped with one M in the Calm, you know, because now it could be quality compute. But now it's a problem. So they couldn't. In my opinion, they they should they should rebrand it and drop one M. Yeah, totally.

SPEAKER_02 6:38

That's oh yeah. If only he had the foresight to think 40 years ahead, right? No, that's that's so funny, it's so true.

SPEAKER_00 6:44

Yeah, so it so that's the big deal. What you say is actually a big deal because this company from the get-go has always been about communications, they have a rich history. It started with like satellite communications, and um, when I joined there in 2018, they uh had a tour of a mu the museum within the campus that showed the entire history of all their devices that have been made from the beginning, you know. So it's really nice. It's part of the orientation is that they take you to the museum of their parts. It's a very inspiring way to start working there. But that's the legacy because they've been through 3G and 4G and 5G, they are also on the 6G train. We'll talk a little bit about that. Uh but now they are saying that only one-third of that business is going to be uh handset-based, which is a way of saying that's communications business. Two-thirds will be IoT and data centers. It's a big shift.

SPEAKER_02 7:42

Indeed. It is a big shift. So let's start with maybe the confidence question, which is do you think Qualcomm can get into new businesses? Maybe we'll start at the top there, kind of strategically high level. Um, do you think they can build a real data center business? Uh I'm gonna let you have your take, and then maybe I'll I'll provide some color of what they said as well.

SPEAKER_00 8:07

They can, because there is no reason a company like Qualcomm cannot get into data center business. Because they have compute, right? They have the CPU business, they have NPUs, which they use graphics acceleration on their SoC's mobile handsets anyway. They have all kinds of IP now through the alpha wave acquisition. Uh, I was looking at uh what what all they got through that, and it's it's quite a lot actually, because they have Certi's IP in copper and optical, uh, they have PCIe Gen 6, CXL for like servers and storage stuff, then they have uh Ethernet IP, uh 800 gig 1.60 for like switches, routers, DPUs, NICs, and they have like uh HBM and DRAM IP for GPUs, which is very, very important. We'll talk about that because that's one of their key technological innovations in the era of AI and the way they are doing inferencing chips. And they also have chiplet IP through the alpha wave acquisition through like UCIE or bunch of wires or other standards like this. So they are very well positioned as a company who has also shipped like billions of devices into handsets world over. So it's not like they are new to doing this. You know, I could argue that if OpenAI says that I'm now a chip company, Anthropic wants to be a chip company, they have never been a chip company. It takes experience to be one, which Qualcomm has. So yeah, they are well positioned to do so, although one could argue that they are a bit late to the scene.

SPEAKER_02 9:48

Yes, yes. Okay, so so let's get into that. You know, now in the grand arc of things, late and timing's interesting because uh, you know, obviously this has been going on since late 2022, but it's only 2026, and this is the type of technology that will be here for the next, you know, 30, 40 years, whatever, forever. And right. And so, you know, being being late now feels like a big deal in the grand arc of time. It won't, it'll just be a little blip. But the question is if you're going, what does it take to succeed? Which I think you hit on nicely, which is you need to have IP, you need to have talent, you need to have manufacturing prowess, and you need to um have tech like technological differentiation. It can't just be a um me too fast follower, like, oh great, uh a company did this, I'll build the same thing and sell it into a different market or something, right? Like that's not gonna work. Like there has to be a reason for a customer to choose you. And then to the point of being late, you especially have to differentiate on some vector. I think of like when I think of um Nvidia's, you know, Hopper era GPUs, and then eventually AMD came in with Instinct. And what was nice um was they made a different decision about the amount of HBM capacity that would be on a single chip. And that just that small decision on one vector opened up some particular workloads that could run on like eight um instinct 300s, I think, or maybe it's 350, I can't remember off the top of my head. Um, but it but it took 16 NVIDIA GPUs because they just didn't have enough um memory capacity on each GPU. And and that illustrated for me that like even if you come in late, if you make some sort of different technological bet, it might unlock a certain set of customers, which can help you get in your foot in the door and grow from there. And so I think to your point, and as we'll get into, um Qualcomm has checks a lot of the boxes as far as like they're a big company, they have experience, they have IP. Oh, and by the way, they were able to use MA to get IP that they didn't have, um, get talent that they didn't have and kind of fill those gaps, which is something we've seen them do before in their automotive business. We could talk about that, but um, it's beside the point a little bit. But um, there still needs to be something different so that at the end of the day, a customer like a uh hyperscaler or an AI lab will know why they want to choose Qualcomm. And it's not just because it's cheaper, or it's not just because everyone's out of supply and Qualcomm happens to have some. None of those are sustainable, those are temporary, but there has to be a technological differentiation that really impacts ultimately like TCO for a particular workload. And this is even why we saw like the GROCs and the Cerebrists um have success is because they made uh design decisions and there was lots of sort of negative consequences, but but for a particular workload, high interactivity, it was much better than GPUs could do. And so obviously Qualcomm had to come out swinging and say, we think we can do something different here.

SPEAKER_00 12:54

Yeah, their key differentiator here, uh, apart from their memory architecture, which we will discuss, is basically the fact that inference now is getting disaggregated. This was their key theme that went through the whole talk because they kept saying this is now disaggregated. Which means that you can actually develop hardware to do a particular task. So you could put in a Qualcomm rack along with some other racks. It doesn't mean that you know you have to fill NVIDIA racks with um, you know, all these uh Blackwell GPUs along with their Vera CPUs. You don't have to do that. Now you can put a rack full of CPUs separately. You can put a rack full of low latency decode, like you were saying, the Cerebrus or the Grok LPUs. And now you can put maybe a rack of uh Qualcomm uh inference chips just to do that one portion of it. If they have a high memory bandwidth, uh you could just do decode on that if you want to and still continue to use maybe NVIDIA GPUs for the pre-fill part. So the disaggregation across the whole inference space, which is becoming more and more a theme because one size does not fit all. You've spoken about this in your Substack as well. Like there is a right-sized hardware for the right-sized workload. So that is increasingly becoming a theme, which is why you can run uh different kinds of hardware along a common software platform, which Qualcomm also wants to have you know open and uh developer-friendly for everybody to work with.

SPEAKER_02 14:32

Yes, uh you're you're totally right, which is disaggregation has really opened the door to everyone. Um, once it we started to break the workload down for inference, you know, today LLM inference is obviously the defining workload. Uh the value is actually all being created by agentic inference workload. So uh, you know, reasoning models. Um, but in in breaking up that workload into inference and decode, then people could say, okay, I'm late to the market. I'm just gonna focus on decode. I'm gonna do something very decode specific, and it's gonna be very different, and that will help me get my foot in the door. Um again, worked for the AI ASIC startups, and and this will have to be the mindset that Qualcomm has. Because ultimately, yes, would they like to sell data centers full of Qualcomm Dragonfly, which by the way, that's what they branded it for anyone who's not watching Dragonfly as like their data center brand. Would they ultimately like to sell uh it in the in the branding was white and gold, by the way, so it looks very nice. So would they like to have a data center full of white racks so that whenever on FinTwit you see a picture of you know someone saying, look, I built a data center and all the racks are white, everyone can go, whoa, Qualcomm to the moon, you know? Yes, they want to do that. But the reality is, which I think that they're very um fully aware of, is they need to sell like a few racks into the existing data center that's full of NVIDIA GPUs. And so trying to focus in on D code, for example, would be one way to get there. But um, okay, take us further. Let's talk. Do you want to talk about the memory, the HPC? That was sort of like the big star of the show.

SPEAKER_00 16:12

Yeah, let's go to HPC uh because that is an underlying technology that is very useful, regardless of whether they want to use it in data center hardware or IoT or automotive or robotics, even, because this is going to be the platform that lays the foundation for all of their businesses going forward, in my opinion. It seems like that, that way, at least from what I heard. So, high bandwidth compute uh is their way of getting a better memory bandwidth hardware packaged right next to an XPU so that you can get a really fast um inference bandwidth like you would get with uh you know like Cerebrus or Grok LPUs. Because those things can do like uh SRAM bandwidth is in the range of a hundred terabytes per second, and right now HBM uh bandwidth is about uh a tenth of that. It's about eight terabytes per second, right? So they want to break through this eight terabyte barrier, uh, which currently involves an HBM stack sitting next to a GPU or XPU and connected with maybe 2,000 lanes of uh interconnect between the two uh memory and the uh GPU chips, and it is packaged with an advanced packaging substrate like TSMC COWAS, right? So their idea is with high bandwidth compute is that you take the memory and you put it on top of the XPU die, and instead of connecting the memory through the side uh using, you know, on the shoreline density, as it's called, on the one edge of the GPU die, you can only put so many lanes before you run into advanced packaging limits. If you put the memory on top of the XPU, you expose the entire face of the chip uh between the XPU compute and the memory to have interconnects. So now you can put instead of 2,000 lanes going through between these chips, you can have, I don't know, tens of thousands or even a hundred thousand different lanes. So what that does is you if you increase your bandwidth uh number of lanes by 100x, you know, you'll get a hundred X more bandwidth kind of thing. So this is their main idea of doing uh what they call high bandwidth compute.

SPEAKER_02 18:36

Yes, yes. Okay, and so there's so much to unpack here. So I'm gonna walk it through again for people at a high level in case they didn't catch it the first time. Because I I will be honest, one, the naming might not do it justice, although it don't have a better name, because HBC, HBM, it feels a lot, it feels very similar, but there's actually a lot of differences going on. And and then two, there's actually some nuances that, again, weren't obvious to me like the very first time it came across. But as I looked back at it again as we prepared for this episode, um, there's some things that stood out. So I'll talk about it. So um, so one of the problems, you know, obviously the way everyone knows the way GPUs and HBM work today is you've got the HBM, it sits next to GPU, and you've got things like weights or activations, whatever, um KV cache stuff sitting in this HBM. And uh during decode, especially that stuff just has to fly back and forth. It's like if you separated like um your kitchen and you put your fridge in the garage and you just had to walk to the garage and get something, bring it back, chop it up, walk to the garage, get the next thing, bring it back, chop it up, walk to the garage. You know, obviously it's like, dude, what are you doing? Why are you spending all this time walking back and forth to your garage? Just put your freaking um ingredients right next to your chopping block, you know? Um, so so because of course there's lots of power loss, lots of latency time, that kind of thing. And to your point, you can't uh the analogy breaks down, but you you can't get as many lanes and as high and high of bandwidth. Um, so so one point is ultimately like how do we bring the compute as close to the memory as possible? And other people have called this like near memory compute, processing in memory, stuff like this. There's lots of startups doing this. I think it was is D-Matrix doing something like this?

SPEAKER_00 20:26

Yeah, yeah. This is D-Matrix approach, what they call in-memory compute. In-memory compute. D matrix also puts memory on top of logic. So there is a question of really are is the Qualcomm innovation all that different? Is another question to ask.

SPEAKER_02 20:43

Totally, totally. Uh so okay, so so that's the first problem is how can we bring the compute as close to the memory as possible? And then, and so so what you said is like, oh, well, what if you actually like the the closest thing is to stack the memory on top of the compute and just literally have you know have it as close as possible. Um now, there's obviously um thermal concerns there. These GPUs, XPUs are like crazy hot. So can you actually put memory on top of the XPU? And as I looked closer in it, and we'll try to include some pictures uh as well. Um my understanding is they are not actually stacking the memory on the XPU, but they are putting a logic chip under it and they're putting it close to the XPU, but they're actually offloading some of the workload to live on that logic chip. I don't even think it's necessarily like a programmable logic chip. I have a feeling that it's it's almost more of a true accelerator in the sense of accelerator. And so, for example, like my guess would be like maybe you have during inference some sort of primitive, some sort of function that's run all the time, like softmax or something in attention. Can you actually take that off the XPU? Put it in the logic that's right under the memory. And the idea is like uh fetch what you need from the memory, do those primitives right there, and then send the result back. And that would obviously help you solve like the thermal issue of you're not actually putting the memory on top of this crazy hot XPU, but you're still using advanced node logic and putting the memory on top of it, and then just accelerating an even finer grain piece of the workload. And so again, uh, and I'll let you jump in because I know you'll have thoughts and reactions on if I'm interpreting that right. That would be, this would be the name that we've always used before AI, which is like an accelerator. So, like um uh like a camera, like an image sensing processor or any other little like acceleration that we've done in the past, like with graphics, which is you're like, dude, we just keep doing this particular loop over and over on a CPU. Why not build a little, you know, Qualcomm does this all the time on their Snapdragon SOCs, like build a little thing to do some image processing that's just dedicated for that. And in my understanding, it felt conceptually like they're saying that. Take a little tiny piece of the inference, um, sort of primitives, move it off, but then also it's very important that it's very close to memory because it's very sort of memory bandwidth intensive. So I'll pause there. What is your reaction?

SPEAKER_00 23:18

So when I saw this being announced uh on the talk, it was very confusing the way it was drawn out. Because in the first example where they show how the current uh solution works, they actually had a GPU and they had an HBM there, and there was like this gold-colored like back and forth happening. It was very beautiful to look at. And uh you could see that data was like this gold data was going up and down the HBM stack and was like going around um into the X GPU. So that's how it works today. But then when they put in their solution, which is basically their high bandwidth compute, they still showed a stack of LPDDR, and then they said something about XPU, okay? And then on sitting next to it was an SOC, right? And then they were showing the same gold thing, which presumably uh presumably is data, going between the SOC and the LPDDR stack. So for somebody who's looking at this, it looks very similar to the HBM one. Like, so I'm like, what's different? And then uh they go on to mention that no, you don't need advanced packaging anymore, that you could do standard packaging, because you you you don't need that bandwidth anymore. So my interpretation of this is that it's not really the logic die under this memory is not really any simplistic logic die of any kind. It is a full full up XPU, right? Because the that's the only way that you will not need advanced packaging to make this work. You have to do the full thing there. And that's what Dmatrix is doing. They have a they do all their compute right under memory.

SPEAKER_02 24:59

Okay, so when you say XPU, what is your definition of XPU then in this particular use case? This is the thing that does matrix multiplications.

SPEAKER_00 25:07

That's what I call it.

SPEAKER_02 25:09

Yeah, yeah. So do you think they will do all any matrix multiplication anywhere in the workload right there under the LPTDR?

SPEAKER_00 25:18

Yes, I think so. It's going to me, it looks like it's very similar to what Dmatrix is doing because D matrix uh digital in-memory compute and their Raptor, which is the next generation of uh 3D DRAM, which they call 3D DRAM. Uh those slides are out in there. They they previously announced this stuff, so it's nothing that's unannounced or anything. They actually have a full-up XPU chip that it does all the compute under the DRAM die, and that's it. That's their Raptor chip. So this has to be something like that. This SOC sitting next to it is doing something else, right? It could be a mobile SOC. Like if it's used, if you're using AI on a phone, it could be a mobile SoC. If you're using AI in an in a car, it could be an automotive SOC, but it's not doing matrix multiplications. It's doing SOC stuff.

SPEAKER_02 26:09

Sure, sure. I mean, so I agree with you. I like conceptually, the idea of essentially like a tensor core coming and moving over and doing all the matrix multiplication. I totally agree. That's what makes sense. That's what you would want to do is get all of this multidimensional data, put it right down into some sort of like tensor core type thing, and have it do all the mat molds. It would be nice to understand the clarity of what else is in this SOC. Obviously, um, right there, there's die-to-die communication, there's there's orchestration of the whole workload, there's communication with CPUs. So there's all these other sort of like blocks, if you will, that don't need to live right there. But but I do agree with you that it seems like conceptually, anything in the workload that's related to the specific matrix multiplication stuff would live under the memory.

SPEAKER_00 26:54

Yes.

SPEAKER_02 26:54

And and you mentioned earlier that this technology could actually come to the edge, come to phones, come to auto, come to wherever. And I think that's also the key point here is if people are thinking like if people think like GPU with LPDDR stacked on top, then that might be a little confusing because it's like, oh, are we putting big GPUs into a car now? But the point here is that the technology, the concept of the technology, which is if your car is gonna run um neural nets, wouldn't it be nice to have the matrix multiplication under a big stack of not HBM, but you know, LPDDR, something that may be like lower power, cheaper, higher capacity, but that would still let you do essentially generative AI at the edge, presumably with even bigger models, for example.

SPEAKER_00 27:44

Yeah. I want to talk about this memory stacking part first because we will definitely talk about uh the automotive IoT and the robotics aspects, which they really want to get into in the future. And I think they're well positioned to do it. But there are like several issues with the way this memory works to begin with. First of all, you already mentioned the thermal aspect. It is pretty challenging to stack logic on top of uh GPUs or XPUs like this. The second thing is that there is a fundamental size mismatch, right? Uh these XPUs that sit under any form of memory, like even when Dmatrix does their Raptor, they are pretty big dies because they are reticle-sized GPUs that sit there, right? Because Nvidia's even like packages multiple GPUs in a single, like you know, Rubin or whatever nowadays. So it's mini it's easy to assume and it's okay to assume that the XPU will be a reticle-sized uh compute unit, which means it's like 850 square millimeters or anyway, it's above 800 millimeters squared of chip. Now, if you have to stack DRAM on top of that, can you imagine how big of a DRAM die you need to put on this chip? And can you imagine the planarity requirements of actually hooking this up and keeping it planar in a thermal environment? Like stuff is heating up, and the planarity between the top die and the bottom die needs to be maintained across with without having the bumps rip off each other. This is a challenging problem.

SPEAKER_02 29:16

Yes, yes. You're saying that like things move and warp and whatnot when it's hot, but obviously you need physical connections, wires, pipes, so to speak, going up and down. So you can't have all this moving and the pipes moving around. Like it needs to be all kind of rigid, stable, heat up together, cool off together, or dissipate all the heat so that it doesn't move around.

SPEAKER_00 29:37

Yes, yes. So it's a challenging problem. And as much as they say, like, oh, we don't need advanced coast packaging anymore, you know, yay, we solved packaging, it's okay, it's enough to use a standard pitch package for this. They have not really solved the packaging problem. Okay, they moved it to a different place. They moved the packaging problem from next to the XPU to on top of it, which is arguably even harder. And one more thing I want to point out is that the picture deceivingly shows you like multiple LPDDR stacks. Okay, it's not that easy because even the Raptor first generation will have is probably going to have one die layer. Okay, the expansion is going to happen later. Um, people want to stack more DDR die on top of each other, but it's not simple by any means because as it is, it's hard enough to stack one memory chip on top of logic like this. Okay, because like this is like logic on logic stacking in a sense, if you think of it like that. Um it's like stacking SRAM on top of uh compute, like like AMD did with its vcache. Similar stuff.

SPEAKER_02 30:46

Well, right, yeah, okay. Well, talk to me about this then. Okay, so obviously um Qualcomm's gonna need need to work with the memory vendor, but then there's a a logic accelerator under it. So obviously have to work with like uh the logic foundry as well and sure. TSMC, exactly. Um then uh as far as like the thermals and the stacking and making the logic and the memory sort of behave nicely, like ultimately who does that fall on? And um HBM4 has, or maybe it's HBM4E, has memory, HBM stacked on top of a custom logic die now. So I'm wondering if the if some of these thermal problems are already being solved for HBM4 stacked on, you know, like the HBM stacked onto uh custom base logic die. And if there's any like learnings that can be applied here. So it's like, oh, the industry's already solved this, or is it like, no, no, no, we're talking about different memories and and and a different level of compute under the memory?

SPEAKER_00 31:51

I don't think that the logic die and the HBM die sizes are as big as a GPU die. They are smaller. Oh, true, true, yes, yes. So that's what I was pointing out earlier, that it's not as easy as that. Now, if you want to stack something the size of a GPU die, that's difficult. So the logic dice under HBM4 are smaller, and they don't nearly do as much work as something doing matrix multiplications uh to infinity, which is what XPUs really do. They do a lot of matrix multiplications. So it's like a computationally much more heavy workload when you try to do it under so presumably running hotter for more often. Yeah. There's another catch to this. When you do DDR7 on top, let's say, I don't know, DDR7, LPDDR on top of LPDDR, the whole idea was like DDR has more capacity than HBM per layer, right? And therefore you're getting more capacity. No. But now when you stack DDR on top of DDR, that's pretty much HBM. Now what you're doing is you're stacking HBM on top of compute, in my opinion. And the reason HBM per layer does not have that density, memory density, is because there are through silicon vias. And when you have through silicon vias, you can't put as much memory cells around it. There's like a keep out zone. If you put a true via silicon via here, you can't put memory cells around it through some distance. Sure. So per die, your memory capacity drops. Right? So you it's you get high density if you stack only one layer because you don't have to put VRs through that memory layer. The moment you start stacking two or more, you have to put VR through the layer, and which means that the capacity per layer will start to drop, right? Uh let's say by half. And so you need to stack like four to make it even useful. You can't stack two because you lose it on density and keep out regions. So you see the complexities here that nobody ever like talks about. They just say, look at this, yay, high bandwidth compute, let's go, and that's it.

SPEAKER_02 33:54

Totally, totally. On that point, this was obviously an investor day, and so they teased some technology, but they didn't get into the technology weeds. I would love to see Qualcomm have a like data center day or something that's focused on like the technical architecture and all the technical details, or ultimately just publish some papers on this, show it at hot chips, whatever, because there are lots of um technical questions, but if they can answer them and answer them confidently, then it gives a lot more credence to what they try to say at Investor Day, which is we're late, we've got a new approach, we think it can ultimately be competitive, it's HBC, and we've shown you all the technical bits and answered all of your questions with satisfaction so you can believe that it's possible, it'll work, it'll scale, it's and so on and so forth.

SPEAKER_00 34:49

Yeah. So I think we've spoken about the technical complexities of HBC. Uh, we don't know any more details, like you say, unless they publish something. But I think the first HBC-based chip is expected in 2027, is that's what they said. Uh, and then they showed a chart that shows that they will do the AI 300 in 2028, and they will use like UA Link and uh ESun for scale up fabrics, right? Uh and then somebody in the audience asked, like, when are you doing scale up CPO? Or CPO, right? Obviously, CPO. Uh, they said, Yeah, it's going to be after that. So it's like maybe a 2029 thing would be CPO. So it's interesting that uh this is in the works, but we really have to see the silicon show up for us to understand anything more than what it already has been announced here. Totally.

SPEAKER_02 35:40

So on the on the accelerator roadmap, so they said yes, we've got a multi-generation accelerator roadmap. Historically, they had the AI 100, that was kind of like whatever, it was a long time ago. Maybe I'm sure they learned some lessons, but it you could argue it doesn't really count. The AI 200 um sampling in 2026, um, it does not have this HBC that we're talking about. It is the AI 250 that has the first generation of HBC technology, and that's in 2027. Um, and then to your point, the AI 300, which isn't until uh fiscal year 28. And I'm not, you would know. When does Qualcomm's fiscal year start?

SPEAKER_00 36:23

It's a good question. I never followed the financial side of things when I was at Qualcomm. I was doing engineering work. No, but I think it starts in 2027, actually. When you say financial, because why I said that is at the end when the CFO was talking about FY 2027, and he just mentioned, hey, FYI, that's actually in calendar year 26.

SPEAKER_02 36:44

Yeah, right, right, exactly. Yeah, yeah. Well, I don't know if it's a full year, but it's probably like it starts in you know June or something. Starts in the later part of the year. Qualcomm fiscal year start. I'm Googling it, audience. Um Quantom's according to AI overview, Qualcomm's fiscal year begins on the last Sunday of September. So they they kind of pull it forward by a quarter, essentially.

SPEAKER_00 37:08

Right. Um okay, so I did pay attention a little bit.

SPEAKER_02 37:12

So uh the AI 300, that's when they get the second generation as HBC. That's also when they add, yes, um, scale up UA link or eSun uh and copper and optical scale out. So they started to have the buzzwords. And so I I think this is like a nice transition to talk a little bit about alpha wave semi. Um, so one of the things nowadays, if if you're coming to compete in the data center accelerator market, you can no longer say I have a chip. You have to say I have a rack scale solution, which not only means the accelerator chip, but it means the interconnect. So obviously, going all the way back, Nvidia bought Mellanox, and that was their way of using MA to say we want to be more than just the GPU, we want to do the whole system, and Mellanox gives us all the networking, um, scale up, scale out, switches, all the good stuff. Um, so then the question is what is Qualcomm's Mellanox, so to speak? And Alpha Wave is an acquisition that they made, I think about a year ago, maybe a little bit less. Um, and Alpha Wave historically is well regarded for their IP. So a lot of communication IP, you mentioned it earlier, and we can get into it a little bit more, um, from CERTs and essentially taking an accelerator and making it able to talk at high enough bandwidth and get data in and out and communicate in and out, I should say, um, with uh sort of industry standard networks, talk to like say a Broadcom switch on the other end or whatever. Um, I will also mention that Alpha Wave Semi actually also brings custom silicon customers to Qualcomm because Alpha Wave was pretty smart, which is to say, hey, if we're good at the hard part, Certes in I.O., we can help people do custom XPUs. I mean, that's essentially like what Marvell does and what Broadcom does. It's saying, like, we could license you the hard part, or we could say, like, no, no, no, we can also do the quote unquote easier part, like the front-end RTL stuff. So Alpha Wave Semi was actually also in the game of custom XPUs. So not only did Qualcomm acquire the IP and the talent, but they also acquired customer relationships and like already uh like roadmaps that Alpha Wave Semi had with these huge hyperscalers. Um, they didn't name them, they said there's two, and then also separately on the call, they had Satya Nadella spoke and Mark Zuckerberg spoke. So obviously that implies that that Meta and Microsoft are working with uh Qualcomm on data center, but they did specifically call out two um custom silicon customers, which are presumably heritage legacy alpha wave ones. And I didn't think about this until just now, but Alpha Wave was public, so a person could probably go back and look and figure out who those customers are.

SPEAKER_00 40:05

Uh the CFO also said that they expect one billion dollar of revenue in 2028, I think, where they said that the total uh revenue would be five billion. Um, but one billion will come each from one of these hyperscalers. So two billion is already spoken for. It may I don't think it's actually Alpha Wave's customers, uh, because he said that on top of this, we have like the Alpha Wave business as well that will add on top of this. So he says, yeah, things are looking very good. That's what the CFO's whole thing was. But in, you know, talking in terms of acquisitions, actually, their modular acquisition uh that was recently announced is very interesting too because you can't build all this stuff, a rack scale solution, without providing a software layer to go with it. And basically that's exactly what modular brings to the table. They have something called like Mojo, which is somehow equivalent to Nvidia CUDA, I suppose, in the programming uh layer. Uh then uh they also have something called Max, which is the equivalent of like a Triton or TRT LLM, which is for model serving. And then uh they have the cloud one, the cloud uh product, which is uh distributed. It's like Nvidia's dynamo. It handles like KV cache offloading, I suppose, and data movements and all of that stuff. So the modular is basically uh that you can uh use this software layer to run what they call multi-silicon token factories, which means that this software layer will tie together hardware from multiple vendors uh in a single solution so that you can mix and match. So that's a powerful thing where Qualcomm solution can be slaughtered in with others and still work. Right, right.

SPEAKER_02 41:58

So this is like um the unicorn that everyone's always talked about, which is like write it once and run it everywhere. But actually, when you peeled it back, there's always trade-offs of like, oh yeah, you're writing in some high-level abstraction, and then something's happening and it's trying to like um you know translate it down to something that runs. And usually what happens is it's it things are like unoptimized, you know, there's a translation layer higher up and it's unoptimized. And and so the that the natural question is like, okay, uh, is this different? And actually, um, this was a very surprising and strategically good decision by Qualcomm because Modular is started by Chris Lattner. Um, he's one of the co-founders, and he is sort of like like a legend in the space of compilers and programming languages. When he was in his PhD, I think he created uh Clang compiler and LLVM. When he was at Apple, he uh invented the Swift programming language. Um and he also has history with the MLIR compiler. And at the end of the day, what Chris and Team did is make it so that you can build in, I think if you use their language, it's like a superset of Python, maybe, but they they have the hooks to go all the way down to like uh the MLIR, it's like multi-level intermediate representation or something. So get to kind of go way down the stack and and hook in there. And so like at a most high-level conceptual level, I would think of it as like other approaches that it's like, yeah, right using our software and it'll run on Nvidia's and ours, um, would have to do some translation stuff at a much higher level and it was unoptimized. And and Latner's team is able to go kind of like much further down the stack. Um, so ultimately in the data center, this is this was this would be great because you could say, hey, um use mojo, uh use modular tools, write it for the Qualcomm and It will also run on NVIDIA. Now, Chris has before said that they can actually squeeze more performance out of NVIDIA's chips than NVIDIA. So it would be like we can also make sure that you're getting as much as possible out of your existing NVIDIA infrastructure, potentially maybe even more. So that's compelling. But then another really compelling angle is just Qualcomm infrastructure to say, hey, what you deploy in the cloud on Qualcomm hardware or someone else's hardware, you could also deploy on premises, you could also deploy on your desk. You could also deploy on your phone. So that's interesting too. It's not just cloud multi-vendor silicon that you could deploy across, but in theory, you could easily deploy across from cloud all the way to Edge. But not obviously NVIDIA is doing that too, where they're bringing right with their um laptop and stuff that they've come out with recently to say, like, oh yeah, you can write CUDA and it can run in the cloud, or it can run at the DGX little box on your desk, or it could even run in your laptop. But that is still a proprietary environment. And so if you use Mojo and Modular, you would be able to get all those same benefits, but in a true open uh kind of modular, hence the name environment.

SPEAKER_00 45:22

Okay, so that's good about that. We should uh move on and quickly mention what they called for their CPUs for data centers. They call it the C1000. Uh and uh it's supposedly a five gigahertz per core and a 250 plus core count, uh, which is like runs a lot of PCIe Gen 7 with LPDDR and all of this. I think they have like three different versions of this, uh, one for agentic CPUs, one for general purpose, and one for AI head nodes, which is which is nice. And it looks like they had like Mark Zuckerberg come in and do a video clip saying that like Meta is planning to deploy the C1000 in the data center and that they have a multi-generational agreement to supply to Meta. So good, they already have like a business customer there in Meta. Uh but uh the one question from the audience at the end on this topic was like really funny because uh that analyst asks, hey, your CPU is is like for 2028 or something, it's not like this year. So the analyst asks, like, hey, you say like five gigahertz, but is your CPU any good in 2028? Like uh now I know everybody knows there's an agentic AI CPU shortage now, but you don't have a product for two years. What's with that? And so for that, you know, the Alpha Wave CEO, uh Tony Pialis, right? His name was? Yes, yeah. Uh yeah, he was like, Oh yeah, Qualcomm has engineers, like the best engineers. And so when they design stuff from the ground up, it doesn't matter when it hits the market, it's still going to be the best. I'm like, dude, I mean, I thought Amon was like the bullish Qualcomm guy, which makes sense because he was in, he's been a Qualcomm guy since he was an engineer and now he's CEO. So it's I'm like, Tony Pial is like showed up like six months ago with the Alpha Wave acquisition, and I'm like, what's with the bullishness about CPUs? I don't know. I can't be that bullish, okay? I'm gonna say that. I'm gonna take that with a grain of salt. It's good, maybe we still need CPUs. I don't think we are anywhere near the peak of what hardware is required for AI, agentic AI, and all that. Maybe CPUs will be required for another five years. Because if you see like AMD's forecasts, uh there's like a massive growth of the CPU industry all the way to 2030, 2031. So, in a sense, yeah, I buy that. Qualcomm's not really late to the CPU game, but uh it's it's not that easy to just say that, yeah, we're going to be the best. Yeah, it'll be one of the CPUs. And if they can provide it at scale and with volume and you know, with have sufficient uh supply chain capacity, then people will buy it, right? So that's the level take on it.

SPEAKER_02 48:03

Yeah, to your point, there is a wave happening right now and they're missing that wave, right? Where it's just like we need all the CPUs we can get, they're missing it. So interestingly, what happens when they come back in? Well, if CPU capacity is a lot higher, then you are going to have to compete more on performance or power or cost or something. So I guess it's good to, if you know it's gonna be late, make sure that you feel very confident that you know where it slots in, which by the way, I appreciate them calling out. There's a difference in the type of workload and requirements that you need from the head node versus from general purpose, which is just running more of your SQL databases because now your agents are pummeling them, versus the agentic rack running a lot of maybe VMs with little agents and they're spinning up tools, and some of them are making web requests and some are compiling code, whatever. So I appreciate that they spelled that out. But if they're gonna try to compete at that agentic layer, um, yes, they need to obviously think really hard about what are the specs they want and make sure they get it right. It's too bad that they couldn't have brought it in a year earlier, but it is what it is.

SPEAKER_00 49:06

There are reasons for that, which we won't get into. But I want to quickly mention also their other big angle, which is the edge AI thing. Because Qualcomm is one of the biggest proponents of edge AI, other than I would say maybe NXP, uh, because it's very important from their whole IoT and automotive business, which they've been doing for the longest time. Uh, and so they expect that what they call the software-defined vehicle is now going to be called the AI-defined vehicle. And they provided like a really interesting example. Like a car drives into a parking lot and then it sees a QR code, and then it just looks at it, identifies what it is, and then pays for it. Right? And you don't get like parking tickets, so if time runs out, it can renew it, or it can like maybe get an uh notification on your phone and then you know take care of it all by itself. I thought that was a nice example. Apparently, that's already in deployment. So wow.

unknown 50:02

Yeah.

SPEAKER_00 50:03

So that's one use case they're looking at, and uh they they want to couple these uh high bandwidth compute units that we spoke about uh into automotive SOCs, which means that you can put AI into a car and so that it can think for itself and look at QR codes and identify this and that. And they want to put vision as a primary driver for their whole uh edge inference play. Because they say that vision is like the biggest uh uh way to get information and you can if humans can do most things with vision, uh AI is AI should be able to. So they're like heavily centered on uh vision as a major unlock for industrial AI. So all of this was really cool because they are talking about converting cars into basically token generators. If you can have this high bandwidth compute unit in a car, it is actually using local AI inference and connectivity may exist, yes, but you can use local AI to do a lot of stuff. And it's really fun. I think this concept is really nice. I don't know whether it'll materialize, but it's fun to talk here about all these applications.

SPEAKER_02 51:18

Totally, totally. And the nuance I will add is that many cars are already token generators, but they're captive. It's for ADAS in autonomy. And so you're you're you're Rivian, you're Tesla, you're whoever. And so you're running end-to-end neural nets these days on potentially NVIDIA chips or custom chips in the case of Rivian's Wrap One. Um, but it's captive. That is for uh essentially driving the car. Um, so what's cool is essentially if you can enable the cockpit or like the user experience, user interface to have AI and have enough sort of compute headroom to go experiment and do very interesting things, whether it's just like, hey, next next song, please, you don't have to take your hands off and it actually works, or actually the car itself doing interesting things for you.

SPEAKER_00 52:13

So yeah, a lot of these edge IoT things are really fun, actually, because they if you remember the recently acquired Arduino 2, uh you know, and they have the Dragon Wing uh platform, which I believe they are launching later this summer. So what you can actually do is you can buy like an AI card uh off of Amazon, and then you can use that to run like cloud locally, right? Um, those those are cool. Uh I don't know what kind of uh model like sizes or whatever these they had said. You can run cloud code locally. That's what uh Qualcomm said on their Investor Day presentation. So I assume it's like a capable model. But I think it's nice. You know, they showed all these applications in like retail where you can like stock shelves and then it automatically like restocks, and you have all this intelligence within the department store and uh the how they use it in energy, oil, gas, and everything. But their long-term play at this moment is basically robotics because that is what they view as their absolute future opportunity, which they mention to be like a one trillion dollar opportunity in 2040, though. If you see it. Oh, is it 2040? Yeah, yeah. It's 2040.

SPEAKER_02 53:20

So it's like, okay, whatever. It's a we'll retire by then. But uh, no, I mean, what do you need to do interesting robotics? You need a CPU that that is constrained by power, because it's probably battery powered. Um, you need accelerators, you need memory. Well, and one of the things that we don't have today is cheap enough, high capacity, high bandwidth enough memory at the edge. So we have to run these crappy little models with very little context. If HBC or other innovations can bring us much more memory and much more bandwidth, um, and still do it in a way that makes sense from a power envelope, um, that's really compelling to be like, what if today's frontier level model, not even Fable, Fable, I miss you, come back, give me a call. Um, but you know, Opus level three or four years from now, if you could get that at the edge and and you can fit it in. Um, I think the opportunities are very compelling. Um, oh yeah. And then of course, what else do you need? You need connectivity, right? So they kind of have the whole portfolio: CPU, accelerator, some memory innovations, connectivity. It's very promising for Qualcomm. Oh, you need the software stack, they're acquiring that with Mojo Modular. Wouldn't it be interesting if they acquired it for the data center? But actually, the best place for that to capture the most value was robotics in you know, seven years from now.

SPEAKER_00 54:47

That's my whole uh outlook. You know, I think we're coming up on time anyway. So I just wanted to mention that my whole outlook on what the Qualcomm Invested A came away with was their data center entry point is a little bit late, yes, but we are nowhere near the peak of what compute and AI is anyway. So maybe, like you say, it's just a blip in the long uh time horizon here. But that's a short to medium term play. I think if edge AI really becomes uh you know uh something that is widely adopted, uh Qualcomm has a very, very strong portfolio and they could come up and dominate edge AI like they did communications for like a good decade, right? So I think they have a very strong edge AI play, but that whether that's going to come anytime like next year or whatever is the question. It's it's gonna take a while. So I remember that you uh in the beginning mentioned you got to ask Cristiano a question. Uh I don't know if it was like on on recording or off recording, but doesn't matter. But now I'm curious, what did you ask him?

SPEAKER_02 55:53

Yes, yes. So it was not in the recording. I was in a room with um Akash, the CFO, and Cristiano, the CEO, and industry analysts had a chance to ask questions. And this one was on the record, so I can talk about it. Sometimes it's on background, so you can't talk about it. Um, but I basically said, hey, you you introduced HBC, which needs less HBM but more DDR. Um, DDR uses fewer memory wafers than HBM. Um so on the one hand, that could shift the mix from HBM back into DDR's favor. Um, on the other hand, uh this actually opens up the opportunity to bring much more memory to the edge. So um, Christiano, from your perspective, how should we think about the memory market in 28 and 29 and 30 when more supply is coming on, but new innovations like this are gonna shift the mix, but also it's just gonna grow the pie because the edge is gonna need more. So, how should we think about it? And then he was like, that was very many questions. That was a big question, but it's very good and very interesting. And um, I think he was basically like, HBM's not going anywhere. Don't think HBM is dead. We're gonna need HBM. But he's uh he said, you know, essentially what's interesting is the amount of increased demand there will be from the on-premise infrastructure from the workstation on your desk to your laptop to your car for LPDDR and more memory. So he also was like giving the M-O-A-R memory vibes more memory.

SPEAKER_00 57:29

Amazing. And in general, was everybody really excited about all the Qualcomm announcements from what you could see around you?

SPEAKER_02 57:35

Oh, yeah. Yeah, I think all the other analysts in the room were excited. I think, you know, investors were excited. There's always the technical questions. Some people are naturally more skeptical than others. And so it's like, okay, you you introduced a lot, you talked about it, we want to see the technical proof. And and some of that is just like you have to know the context of this is an investor day. What are we gonna talk about? How detailed and in the weeds we're gonna get. And also these things take time. So, you know, you I think you can give Qualcomm the benefit of the doubt, but they do eventually need to follow up with technical details, especially if stuff's not shipping for a few years. The best thing they could do is write a paper, give a presentation, whatever, to sort of prove that at least in the lab, it's real and it works.

SPEAKER_00 58:19

Yeah, so we still need more memory. Good to know.

SPEAKER_02 58:22

Yes, totally. Um, okay, maybe here's last one last bonus thought. We won't even talk about it. We talk about it more some other time. If HBC can bring much more memory close to the accelerator. So, how about let's not call it HBC? Let's just say industry-wide, umpute, what are the implications on the memory market, right? Like obviously, we're still gonna want HBM, but now all of a sudden, maybe even more DRAM, and not even from a data center perspective, but when we're talking about bringing more data to the edge, we're talking about robotics, we're talking about cars and stuff, like it just feels like even more wafers are gonna be needed. More memory. M A memory, yes. M-O-A-R. Exactly. Don't let anyone deep seek you and tell you, like, oh, HBM's dead because near memory compute. No, M-O-A-R, more memory.

SPEAKER_00 59:17

All right, we should call it with that, yeah?

SPEAKER_02 59:19

Yeah, let's call it there. Uh, everyone, thanks for watching. Um, we appreciate you curious, intelligent, uh, interesting people who love semiconductors and listen all the way to the end. Um, check us out on YouTube, leave comments, check us out on Spotify, send us emails, whatever. We have a daily if you like our takes, you can get them daily for free. semidope.com, check it out, and thanks for listening.