Bryce

It used to be back in the day that when a hardware manufacturer, like the day that they announced a new chip, the day that we said, Hey, here's this chip with this name, you know, when when the day in 2017 when Volta announced when Jensen announced Volta, he had the chip available to hold it up. And like, you know, that was exciting. It was like you you you didn't know about this thing before the conference. And like now it's like uh, you know, we we show our roadmap there, and it's like we show the road, we sh we announce the next chip like a year before it's gonna be out, and then we say something about the next, next chip, and then we even like say what the name of the next, next, next chip is gonna be, which is Feynman.

Conor

Welcome to AD SP The Podcast, episode 226, recorded on March 20th, 2025. My name is Connor, and today with my co-host Bryce, we chat about Bryce's experience at both the NVIDIA GTC 2017 conference and his recent experience at the NVIDIA GTC 2025 conference. How's it been, buddy? You are live.

Bryce

Oh god, Connor. Connor, my butt, my my soul has left my body. Oh, it is it is day four of the GPU technology conference, and I am an old man. I do not know if I will survive. This is just because it's from morning tonight's non-stop stuff, and you're uh It's from morning tonight nonstop stuff, and uh me, uh uh uh Jake Hemstad and some of our other coworkers were at a uh a nice team dinner last night, but we got home pretty late, and uh I I have had to oh oh they scheduled my talk at 8 a.m. uh yesterday morning, which me meant I had to be there at like 6 a.m. Oh man, this is a this is a a crazy, a crazy conference. This is like there's like 25,000 people here for GTC, which is NVIDIA's uh uh tech conference where we talk about AI things. And um like there's like this crazy exhibition floor, and it just like takes over San Jose. Uh and like it's it's too large for the convention center, so it's spread across the convention center in like three or four other locations, and it's a it's a madhouse. It's a ton of fun. Oh, they have a they have a candy wall, which is a giant wall with you know, you know those things that they have coffee beans in that you can like open up and then the coffee beans spill out into your bag.

Conor

Uh what?

Bryce

Like you you take it like like you know coffee. Oh, like at a grocery store. You know. Yes, like yes, where you'd get like nuts or coffee beans or something like that. Like in the bulk aisle kind of thing. Yeah. So they had one of those for candy. Ah, I was doing so good with my diet, buddy, but the candy wall really did me in.

Conor

Yes, it is uh uh free stuffed in general. I don't I don't know if a candy wall would do it for me. A donut wall, they had a donut wall at the end of 10k race I ran once, and then I took a bite like right after finishing and almost was sick. Yeah, can't can't recommend that. Oh yeah, by the way.

Fire Announcer (Interruption)

May I have your attention please?

Conor

This is the weekly or monthly fire alarm test that of course they're gonna record while we're doing this. So you continue telling your story while I mute my mic.

Bryce

Uh no, no, no. It's it's it's interactive. My mother said she listened to episode 221 of our of uh of your podcast, and she says, please make sure to tell Connor that the Inception slash Juno actress he mentioned is now a dude, Elliot Page, Page.

Conor

I mean, I was actually uh one, I knew this, but two, I was actually pretty careful, uh, I think not to uh misgender Elliot because I was referring to the character that they played who was a she. I mean, in the Juno movie, uh if yes, memory calls correctly, that was before um they had transitioned to Elliot. So I was aware, and I was also like cognizant when I was making the point, and then when I couldn't come up, I couldn't actually remember uh I can't remember if I was trying to remember uh Elliot or the name of the individual, but I I think I don't know. Someone can go, I'm not gonna go back and check. But I think while editing it, I was I was careful to make sure that I didn't uh say the wrong thing. Um Right. Oh yeah, that's that's right. Okay, why don't why am I saying um it's Juno? The reason I brought her up was because it was specifically from Inception, right? That uh the dream folding itself over, or like the the world folding itself over. Which is it's less important the actress or actor, I guess now. But uh more was it important that the character in the movie is capable of folding worlds over on itself. But thank you for the feedback.

Bryce

I don't know why my mom is now listening to the podcast. I think maybe she's she's stalking me or something.

Conor

Um yeah, I mean I those two back-to-back episodes, I mean, especially the one after that, there was a lot of what do I call it, um, inserts of other media, clips from whatever, movies and whatnot. But yes, thank you, Bryce's mom, for the feedback. Uh also the lovely note on one of the postcards you sent me. Uh, she had written something, I believe.

Bryce

Oh, that's right. That's right. It's funny because like postcards have such an unpredictable latency. We sent we we we like collect postcards now, and so we send a lot of postcards. And um the worst ones was we sent some postcards from Japan that took like between six to twelve months to get like fully delivered.

Conor

Damn.

Bryce

And like they they also have like a pretty a pretty substantial drop rate. Um like uh uh I only think like 80% of them get through. Because I know we sent you other postcards, um, and I think at least one set of the postcards did not arrive. And I think part of it is is like, and look, look, no offense to Canada, okay? But like we're in the world, like, you know, we're in Morocco, and we ask someone, like, you know, okay, like we we want to send a postcard to to you know to Canada or to the US or somewhere. And we just have to trust that they know what they're talking about. Um and I think that maybe there's been a couple times where we've tried to send a postcard to Canada, um, where they've they've given they've been like, hmm, maybe Canada's in Europe or something, and then they've given us the wrong postage for uh for sending something to Canada.

Conor

I mean, the topic of uh Canada may or may not come up later when we talk about um Twitter, Tesla, and Elon, because we have to revisit a topic. I'm not sure if you recall what we talked about, but we're gonna we're gonna shelf that for maybe part two of this. Because we should get back. I mean, and so and also part of the problem is is I moved in between. So I think you sent it uh and if they had shown up on time, I think we would have gotten those postcards, but I think just the the delay.

Bryce

I suppose that's that's that's impossible. Do you have someone in the US there's like this service where you can like do mail forwarding, which uh Um which is great because I I still haven't updated my address from when we moved last year, so uh we still get a lot of mail at our old address and they just route it to us.

Conor

Who does that? Is like uh is it the mail postal service?

Bryce

Yeah.

Conor

That is something that Canada may or may not have. I have have never looked into that.

Bryce

I just assume that like if it's is the is the Canadian Postal Service called the Royal something something?

Conor

No, it's called Canada Post. All right. Anyways, back. We so we should stay on the topic of GTC because one, uh I've never been, and uh I don't think you did You've never been to GTC. No, remember we've we've told this story a couple times that I was gonna go the very first year I joined, and I joined in October of 2019, which was for those keeping track, like five months before the pandemic went full swing, GTC got canceled. The next year it was online. I think maybe it was two years in a row it was online, and then it's been two years in person and just both of these years. I haven't been presenting uh a ton.

Bryce

Yeah, I I actually I had been to GTC bef in 2017, like four months before I joined NVIDIA, but I have not been to in-person GTC since then. Although somebody maybe pointed out that that may not have been true. I may have been to one other one, but I don't I really don't think so. I don't think I've been to in-person GTC since uh since 2017 since I joined. Well, you know, we should have had you give we we gave all these talks about like uh uh intro intro to CUDA C ⁇ , like how to write GPU code in idiomatic C ⁇ . And uh uh we should have just had you give the talk that I was gonna give because uh I killed myself over the past two weeks uh, you know, trying to get all my slides ready on top of all the other all the other GTC related things I was doing.

Conor

Well, so before before we get to your talk, because it is it is March 20th, very early in the morning for Bryce, 7 a.m. It's it's 10 a.m. local time because I'm still on the East Coast. I watched the keynote on Tuesday and I've seen two and a half talks since then. I watched your talk, I believe, at like 2 a.m. last night. And I've lost many hours of sleep due to having watched your talk because I just I I've I just have I have so many follow-up questions. Um but before we get to my my comments and my questions, because there was a couple questions live, and now uh I it we'll link it in the show notes, folks. It is available via replay on uh the GTC website. Unfortunately, you do have to log in via email. We are working on getting them just posted on YouTube. But by the time I'm done my career at NVIDIA, we will just be posting stuff to YouTube like every other technology company. NVIDIA, we're still we're still getting there. You know, we uh the market cap's there, but we're still working on some uh the edges of some stuff. Uh but before we get to your talk, or we can skip straight to it, what are your uh, you know, having been, at least according to your memory, this is the first time you've been to GTC in person. It's obviously much different. What is it like? Were you at the keynote in person where there's like a stadium full of people? Like uh what are the highlights?

Bryce

So I I tried to be at the keynote in person. I really did try. But um, you know what happened, Connor, is a very Bryce thing, which is I did not read any of the prep material. I read the one email that said don't try to um like drive to the in park at the convention center, uh, because there's gonna be no parking. And so I uh I and that there will be a shuttle from Nvidia's headquarters to the convention center. So I, you know, reasonable person, got up at the keynote was at 10, I I left the um the hotel at 8.30, got to headquarters at like 845, 850, and I I parked in the building that I used to work in, which is, you know, the new building, uh uh Voyager or Endeavor. One of the two. We named them both for space shuttles. And point is those two buildings are uh across a uh three-lane highway, and uh uh I I go and ask the receptionist, hey, where's the shuttle to GTC? And she's like, Oh, it's in building S. Building S is as far away from where I parked on the NVIDIA campus as possible. And so I'm like, ah, great. Now I gotta walk 20 minutes to get to the shuttle. I walk over to the shuttle, it's now like 9.15, and uh I I wait for the shuttle. The shuttle leaves at 9.30. I'm like, all right, great, I got plenty of time. I get to the convention center, I ask someone at the convention center, hey, where's the keynote? And they're like, oh, the keynote's over at the SAP Center at like the sports uh stadium, uh where the or like where they do like concert events, and like, and I'm and and that's like a 20-minute walk away. So I'm like, alright, wonderful. So then I have to take a quick Uber over there, and I get there, and uh the line is like out the door and around the corner. Uh, and so I get in the line, and the line, and I start watching the keynote on on YouTube. Because we do we do live stream the keynote on YouTube, and uh I make it like a fourth of the way through the line, and then some guy comes out and says, like, alright, that's it, folks. It's uh it's uh you know completely packed to capacity, you should go to the viewing party in the convention center. And at that point I was like, you know what, it's maybe it's not my day, but I will tell you the story of the first GTC that I went to, which was before I came to work at NVIDIA, when uh I was being seduced by Olivier Groot to come over to the green side. And uh he convinced me that this was that that GTC 2017 was a a GTC to come to. And uh I've been telling part of this story all week that, you know, it used to be back in the day that when a hardware manufacturer, like the day that they announced a new chip, the day that we said, hey, here's this chip with this name, you know, when when the day in 2017 when Volta announced when when Jensen announced Volta, he had the chip available to hold it up. And like, you know, that was exciting. It was like you you you didn't know about this thing before the conference. And like now it's like uh, you know, we we we show our roadmap there, and it's like we show the road, we sh we announce the next chip like a year before it's gonna be um out, and then we say something about the next, next chip, and then we even like say what the name of the next, next, next chip is gonna be, which is Feynman.

Conor

I heard that as well, and I I was surprised. Like uh I I'm not sure that was the first time I've ever heard him say like next, next, but I he kind of said it as like a throwaway comment almost. It was like, oh, here's Ruben, and also the next one will be named after uh Feynman. And I was like, wait, we make that stuff public now?

Bryce

Yeah. Well, I mean, part of it is because it's gonna leak. Part of it is because if you're buying, if you're building a 100,000 GPU cluster, you need to do your planning a little bit further in advance than you used to. And like for consumer products, for consumer products, we still don't announce things, you know, three years ahead of time. Because like with consumer products, you know, you can still have the magic. But I feel like we've lost a little bit of the magic because I think there was something magical about you know the CEO going up on stage and you're not knowing if there's going to be a new chip or not. And then he says, Oh, here's the new chip, and then he he does the showman thing, and then he he holds it up. And uh it was really magical, GTC 2017, when he uh launched Volta, which was I'm I'm incredibly sad because Volta is uh now approaching uh the end of its life. It's been it's a it's an eight-year-old GPU now. And to me, it will always be the new GPU because it was a transformative architecture. Um it was the, I believe, the first architecture where we had tensor cores. Um it was also the architect the first architecture where we could fully sub and properly support C, because it was the first architecture where we had independent threading on the GPUs, where each thread had its own independent program counter. And so you could write proper um uh uh uh synchronizing code between threads. Um and there were just a bunch of other ways in which Volta was just um a huge leap in terms of programmability and performance over the previous GPUs. And really, we almost divided the world um into pre-Volta and post-Volta. Um and now it's like it's approaching the end of its lifetime, and it makes me sad. That was the GPU that brought me to NVIDIA. That I went to the keynote in 2017. I was living in Berkeley at the time, I woke up at like 5 a.m. I drove down, I found parking at the convention center. This was back when the keynote was at the convention center. Before NVIDIA was a big thing. And I got to the keynote and I was there real early. I got into the room, I was one of the first people in the room, and there was a section up front that was labeled press only. Connor, do you do you know where I decided to sit?

Conor

I'm gonna go out on a I'm gonna go out on a real thin limb here and say that you sat in that section, Bryce.

Bryce

I sat in the press only section, and I I live tweeted the uh uh the keynote. So I guess arguably I belonged in the press section because I was live tweeting pictures of the keynote. I tried to find the Twitter thread um to retweet, you know, to be like, hey, this is the last time I was at GTC, but I I think I built up quite a critter following from live tweeting the uh 2017 GTC keynote. And it was a magical, that was a magical GTC, and then I went to watch Olivier's um talk called Inside Volta, where he um him and uh our our former colleague Luke Durant talked about all the architectural details and it was just a magical thing. But now, oh man, now the conference is magical in very different ways because it's this giant event, and everyone's here, everyone wants to learn about CUDA and about NVIDIA GPUs, and uh, if you you walk around with an NVIDIA badge there and you're a rock star, and it's really something else. It really is. Um yeah, and we this year we've been we've launched this whole new curriculum for how we teach modern C ⁇ programming for our platform. Um because our platform's been around for 20 years, and so the problem is that a lot of the a lot of the teaching material that's out there is older teaching material, and it's teaching people uh how to write C ⁇ code for our platform in a non-idiomatic way. And like I a lot of the teaching material was maybe great, you know, 10 years ago, but um it's just become outdated, and so we've completely um uh changed our approach for how we teach um GPU programming. Um and that's part of what my talk covered. Um we've put out all this course material that teaches this new way. We're also now like very big on Python, and so we're sort of trying to figure out how can we teach C how can we teach our APIs in the same way in C and Python and in other languages? We uh we starting to care about other languages too.

Conor

Alright, perfect. You've brought up your talk. This is a perfect transition. So now, if you would like listener, feel free to pause this podcast. You're probably not going to. I wouldn't personally if I were you and I was in your shoes right now, not having watched Bryce's talk. Maybe you have already. Uh, but if you want, if you are so inclined, you can pause this. Go watch Bryce's talk. Uh link will be in the show notes. And then you can come back. Uh there was just a couple questions asked in the talk, if you did go and pause it. And now I shall ask my questions. First of all, it is not a question, but a comment. I always I always forget, and I'm always reminded when I watch one of your talks, Bryce, of the shared passion that we have for beautiful you beautiful slides. You make some of the I I mean, I I usually uh to some extent write a quote unquote trip report for every GTC ever since I've been at NVIDIA, um, when there was a GTC, because obviously COVID year they canceled it. Um, the first one. Uh last year, I think they because it was the first one back in person, they had some recording issues, and so my trip report kind of fell apart because I was waiting for Jake Hemstad's uh talk to be released to like complete it, but then it never ended up having a recording. Anyways, one of my things I like to do is I like to take screenshots of like the highlights. You know, if I watch a Jax or XLA talk and they've got some nice, you know, slide, I take a slide. I think I took eight or nine uh screenshots of your deck. What are some of the I mean I'm I'm gonna compare. I haven't finished the what's Cuda All About anyway talk, but there's two different slides. One from I don't actually I don't actually know him, but his last name's Armstrong. I think Rob Armstrong, he's giving that talk.

Bryce

Rob Armstrong, yeah.

Conor

He has a GPU versus CPU slide, you have a GPU versus CPU slide. I don't need to I don't mean to dunk on Rob's slide, but yours much Chef's Kiss, much more beautiful. You also have that host slash device with the uh the gradient from beautiful, beautiful. I I saw that and I was like, this is Bryce at his finest. Um anyway, so just you said you spent the last two weeks on your slides. Uh it definitely shows.

Bryce

Yeah. It it's interesting now that um a c a couple years ago I made the decision that I was going to speak a lot more frequently, but give the same talk more regularly. I used to make a talk and I would give a talk once or twice. And what I decided was I I don't want to make three or four or five talks a year. I want to make one really, really good talk, and I want to give it 10 or 20 times a year. And part of the philosophy is I want to give the talk so many times that it becomes just like like that I master the talk. And so I think I make fewer talks, but I put so much more effort into the talk because like I made this talk knowing the reason I was so stressed and pushing so hard for it is I'm gonna like I'm immediately turning around. I gave this talk what's what's today? It's the 20th. I gave the talk on the 18th. On March 27th, I'm gonna be giving the talk in Paris. On March 31st, I'm gonna be tip giving the talk in London. Um, and I'm gonna be giving the talk like two or three times a month for the next uh, you know, uh eight or ten months. And then the other big decision I made is to script all my talks. I have a script of what I'm gonna say. And it always amazes me where like I, you know, every talk there's a time target of like how long I'm supposed to make the talk for. And what I what I always do is I go through and I make all the content, and then um if I have like I I I very rarely test how long it takes before I give the talk. Um because I usually give 60-minute talks, and I usually just make the content and it happens to fit in 60 minutes, just without any thought about it. Um and sometimes I can just look through and I can just eyeball it the like maybe this is too much, maybe this is too little. In this case, I needed to give the talk in 35 minutes. And you know, I was a little worried about that because that's not my usual time target. And so I did one dry run through where I went through and ran through my script, and it it ended with like 36 minutes and 24 seconds, and it was just like it was perfect. It was just, it somehow hit the perfect length. Um, but I I love I love scripting the talks because I I feel like for the most part, after I've given the talk two or three times, I can do the delivery without it sounding too scripted. Um it'll still sound a little scripted, but I can do the delivery without it sounding too scripted. And and then I don't have to worry about going off on tangents. I don't have to, once I've got the timing down, I don't have to worry about how long it's gonna take. I can make it more consistent and a better experience for the people who I'm giving the talk to. And one of the reasons why I love giving the talk 10 or 20 times is by the time I've gotten to, you know, the tenth time I've given the talk, I've memorized most of the script. And so now I'm just I'm just consistent. It just sounds natural and it's just consistent.

Conor

Alright, food for thought. I mean, I do not script my talks, but um the result of this talk, it's it's very good. And like I said, these and I actually I went and checked. I I think I said it was seven or eight. I took one, two, three, four, five screenshots. Five screenshots. So not not eight or nine, but definitely the most I've taken of any other talk I've watched is one. Which leads me to, I mean, I'm not gonna share screen share because uh we don't need to. You'll know the slides I'm talking about, and maybe we'll include it in the show notes. By far, my favorite slide. Be sure to check these show notes either in your podcast app or at adspthepodcast.com for links to anything we mentioned in today's episode, as well as a link to a GitHub discussion where you can leave thoughts, comments, and questions. Thanks for listening. We hope you enjoyed and have a great day.

Bryce

Low quality, high quality, that is the tagline of our podcast.

Conor

It's not the tagline. Our tagline is chaos with sprinkles of information.