We Spent Decades Building Programming Languages. Now We Think They're Obsolete.

Vibe coding is rewriting the rules — Two veteran engineers who helped build the internet explain why LLMs and AI coding may render the programming languages they spent decades building obsolete.

Keith Adams (HHVM, Facebook) and Julien Verlaguet (Skip Labs, ex-Facebook) go deep on how coding with AI is reshaping the craft they've spent decades mastering.

This isn't a surface-level AI hype conversation — it's a systems-level
argument about TypeScript's dominance, the hidden genius of linear types, c++, why reactive programming will be the next bottleneck, and whether Rust's borrow checker is the future or a dead end.


⏱️ Timestamps:
0:00 — The big question: Do LLMs change programming languages?
1:17 — Why TypeScript is winning the AI era
4:28 — LLMs and C++: surprisingly good, but why?
8:47 — TypeScript might evolve based on what LLMs need
11:07 — The TypeScript security bomb nobody's talking about
12:36 — Code reuse in an LLM world: precious or cheap?
17:01 — Keith's war story: Replacing PHP at Facebook with Hack
20:55 — How your programming language shapes your thinking
25:03 — The "system programmer" mindset and why their APIs suck
35:21 — Julian's OCaml conversion: "It just worked the first time"
38:34 — Types as lightweight formal methods
44:10 — Linear types, garbage collection, and the Curry-Howard correspondence
55:39 — Reactor programming: the next frontier for LLM coding tools
1:00:07 — Advice to a 19-year-old programmer in 2026
1:08:15 — "LLMs are like steroids" — they amplify what you already are
1:13:47 — The ugly truth about big company engineering incentives

🎙️ Computers, Coffee, & Beer is hosted by Keith Adams and Julien Verlaguet — two engineers who helped shape the modern internet and have the war stories to prove it.

🎧 Full episodes on YouTube + your favorite podcast app.
 @ComputersCoffeeandBeer  

📩 For Business Inquiries: don.broida@gmail.com 


=============================

Channel About: 

Computers, Coffee, & Beer is hosted by Keith Adams and Julien Verlaguet — two engineers who helped shape the modern internet and have the war stories to prove it. 

Keith built the HHVM JIT compiler at Facebook, helped virtualize the world at VMware (employee ~80), and went on to Slack before founding Pebblebed, an early-stage VC firm for technically ambitious founders. 

Julien created the Hack programming language, spent years writing safety-critical compiler software for Airbus and nuclear plants, and now runs Skip Labs. 

Together they dig into the systems, languages, and decisions behind the technology that actually runs the world — told by two people who were in the room when it happened.

=============================
#vibecoding #ai #programming #llms #coding #softwareengineering #airevolution

Transcript

SPEAKER_00 0:00

I'm pleased to be physically hosting you in a physically co-located in San Francisco edition of Computers, Coffee and Beer. This time, no coffee, no beer, but all computers. Decaf. Decaf copy. I guess it counts. That counts. It's a pleasure seeing you, my old friend. Uh very good to be here. Yeah, thank you for being here. There's tons of stuff we could talk about. We are we're physically here in Pebble Bed Intergalactic HQ in San Francisco's mission district. Um we're investors who build. You're the CEO of Skip Labs. You've got a million things going on. Um, but our default equilibrium over the last couple decades as a you know aficionado of programming languages and a programming language professional. Um usually talk PL. And uh one of the things that I find myself really confused about these days is what it means to do programming language research or or to build new programming languages in this kind of post-LLM world, right? Because I've made sense of programming languages as an intellectual discipline, as a kind of applied psychology so far, right? You're you're bridging this gap between these very inhuman machines and the kind of limited psychologies of human beings, and we have our strengths as well. Um, and you're trying to translate those strengths of humans into the these machines. And now a lot of the code is going to get written by very different psyches, it seems like. And does that change PL? Should it change PL? Does anybody know?

SPEAKER_01 1:17

I think nobody knows. Let's start with that. Then I think um there will probably be different phases, right? And I think phase one, uh, what's going to matter is going to be where's the data coming from. So right now you see an uptick in TypeScript, for example. And that's because LLMs are very good at TypeScript. And then it's uh it's you know uh it's it's this virtuous or vicious circle, depending on how you want to look at it. Positive feedback where um where the more code there is out there that is written in TypeScript, the more you know AIs are gonna pick up that data and are gonna be trained on that data, and the more and the more there will be um TypeScript out there. Now, the two languages that seem to be really doing well with LLMs, the Python and TypeScript. I could see how I think TypeScript is gonna win that race. Uh, and that's because TypeScript is faster. It has a JIT that makes you know the the execution kind of reasonable. I've been briefed, yeah. Uh you've been briefed. You've heard about this stuff. This JIT stuff's gonna be hot. You heard it here. And the other thing is that it has uh somewhat some tooling around types. So there's a culture of typing. Um TypeScript is not sound, so that means there are a lot of things that you cannot do, but you can still describe intent in terms of types a lot better than you would in Python. And those two will I think give enough uh an edge to TypeScript to beat, you know, to win that race between Python and TypeScript.

SPEAKER_00 3:01

I mean, do you think do you think that's the equilibrium? It's just sort of TypeScript eats the world and I think for now, yes.

SPEAKER_01 3:08

As long as LLMs um need a lot of data to be trained, um, I think TypeScript is gonna eat the world. But it might not stay this way forever, because what's going to end up happening at some point, I think, is that LLMs are gonna become good enough at programming in general that the programming language probably will not matter as much. Uh and when we get to that point, that's when things are going to become really interesting.

SPEAKER_00 3:36

So, one of my dreams here, if I can, if I may, because like I'm impatient to start banging this drum, is like, why hasn't this made a big fluorescence in C and other kinds of um you know machine-friendly languages, let's say, right? Because if your story, you know, the sort of age-old push-pull inside of a company was always sort of C versus your programmer productivity languages. And the programmer productivity language people, right, the people writing PHP or the people writing Python, the people writing uh TypeScript these days, or whatever your kind of productivity-oriented language is, they do lay a lot more track than the people sitting there kind of, you know, puzzling over 20 pages of template errors and a compile failure, right? But if I'm an LM and I love type systems anyway, right? Type systems are just sort of awesome feedback for me. I eat them for breakfast. Why not also do something that's generating amazing machine code the first time, uh, like C.

SPEAKER_01 4:28

It's a little bit of a mystery to me. I think it really boils down to data that isn't source code. So I think basically, and I'm just speculating here because it's not my job to train these things, but I would imagine that what's very what's going to be very useful is annotated data where you have small snippets with somebody explaining what this thing does. And you know, with something like Stack Overflow, um, most of the data was related to TypeScript and Python because these are programming languages where you can actually program this way, where you're trying to put something together quickly, you need feedback, um, you don't know how to do something, you type it in Google, then Stack Overflow. Well, now they're you copy a chat GPT, but you know, back, you know, pedal back, you know, 10 years.

SPEAKER_00 5:22

What was Stack Overflow? Exactly.

SPEAKER_01 5:24

Um, and so there's probably someone on Stack Overflow who already discussed that, and there's a snippet of code that you can copy paste, right?

SPEAKER_00 5:32

For future listeners, possibly on a lunar base or whatever's gonna happen in the next 18 months, Stack Overflow was a was a seat do web. It was a it was a HTTP service where uh people would ask questions, usually about the programming problem they're having, and people would say there was like a social gamified aspect of like trying to answer lots of questions. And so people would compete to have highly ranked answers to these questions. And so you would Google for you know the problem you're seeing error message you have or the thing you're trying to do or something, and there'd be some little snippet of like, oh, here's how you you know handled this edge case in parsing YAML and blah blah, whatever.

SPEAKER_01 6:09

Exactly. So chances are very high that if you're trying to do anything in JavaScript, you basically type it with words, and there will be someone who is going to explain to you how to do it and give you a snippet of code. Right. And I think that is going to be much harder or impossible in C. Right. Because if you program this way in C, it's never gonna work. I mean, it's it's probably gonna be worse than that. It's gonna look like it's working, yeah, and really it isn't. Yeah, you know, I've I've seen that a lot where you know, when you you watch somebody experiencing C view C code, and every single line they're like undefined behavior, undefined behavior. Oh, I see another undefined behavior right there. So I I if I had to speculate, because then again, I I was not involved in the process of training those things, I would imagine that it has a lot to do with data on the internet uh that is available to train the model.

SPEAKER_00 7:07

I mean, this is a I do have a bunch of like a lot of other people, right? I spent you know last winter finishing all my side projects in GitHub, basically, right? And being blown away that Opus 4.5 could do it. It's gotten a lot better since then. It wasn't that long ago. Um definitely they're pretty good at writing C now. And I and I think one of the things that I was surprised at, you know, my shrinking margin, my shrinking little island of like stuff that only Keith can do with computers or whatever over the last couple years. Uh a lot of the stuff that was defended there until then was lock-free data structures in C. Do you know what I mean? Like trying to do correct stuff with STD Atomic where it really is concurrent and a really state machine. I think until like about Opus 4.5, it was as if the LLM didn't really think about concurrency and sort of write a serial version that would work. And it like just clearly wasn't really thinking about races or anything like that or hazards or anything like that. Um it does all that now. It does all of it sort of reasonably marvelously. Like there will be bugs, you do have to review it. Um, but it's more like, you know, a colleague on a pretty senior colleague on your team, honestly, right? Because like none of us write defect free code, a sort of defect rate of human written C is estimated at one every 10 lines. It feels higher to me, honestly, than than one, you know, one line in 10. That's a defect. Uh and so even for the data poverty, you know, whatever's going on with sort of abstract reasoning and RL and you know, maybe some of these companies have a big have big C corporate that they don't share that that make this easier. Um but I'm pretty impressed for if you'll grant me my premise, basically, right? Like grant me my premise that uh that this generation of LMS is also awesome at writing C.

SPEAKER_01 8:46

So it could I think there's a big difference between writing from scratch and modifying something existent, right? If we're talking about writing something from scratch, Python and TypeScript are just really a step above. Yeah, if you're talking about looking at an existing code base and modify it, LLM is good at a lot of languages, you can use them to do a lot of things. But really from scratch, I would go with TypeScript or Python, and also depending on the size of the project that has to be dealt with from scratch. So for example, if I had to write one function in C, that would work really well. Yeah. If I had to write, tell the NLM to write me an entire project with let's say 10,000 lines of code in C, I what I've seen is a big difference between a project, even with other languages like Go, um, it seems to be working a lot better in uh TypeScript or Python.

SPEAKER_00 9:48

Yeah, yeah. And uh you may have tried more things than me too, but uh that's interesting. I actually thought you were gonna go the other direction with this, that it's like the big existing project that it struggles with.

SPEAKER_01 9:56

Uh but no, I mean it's um it struggles with yeah, it struggles still with understanding what the large project is doing, and uh also it can introduce really subtle bugs because it doesn't understand the environment of the bigger project. But this is all a given. But I was more saying it's not so much that LLMs have a good time or hard time working on big projects, it's more like the comparison between the the difference in in performance between TypeScript and C. You can see it a lot more on a project from scratch.

SPEAKER_00 10:33

So let's let's imagine that you're right of you're right about this. I find it's pretty persuasive, to be clear. This is like matches my experience as well as it seems just want to write TypeScript and Python to some extent. They divide up the pi between them, right? Like like most new projects get built in one or the others of these languages, unless you've got some burning need to build it, you know, to the metal for some reason. Do you think um do you think TypeScript evolves in ways that are LLM dependent? And if so, what are those ways, right? Are there are there things the LLMs like wish they had in the language, so to speak?

SPEAKER_01 11:07

Uh I think it will get to that. I think it will get to that for many reasons. The first one is we're going to run into probably into security problems. Um TypeScript already has security problems. I mean JavaScript community in general already has security problems because there are a ton of packages out there and you know they're evolving very, very fast. And it's a culture of programming that's very different from C or C. It's a culture of programming where taking on new dependencies is not frowned upon. When you're writing a C project, every single dependency is very carefully thought about, and C as well. And that's because um the interaction between those libraries and your own code can lead to very subtle bugs and can lead to things that are very difficult to debug and maintain. And so that's why people are very careful about those dependencies. In the world of Python and TypeScript, not at all. Like whenever you need something, just install the package, and then that package installs another gazillion packages. And before you know it, you depend on a ton of code, and some of it is vulnerable. And I think what's going to happen is that this problem is going to become exponentially larger for two reasons. One, we're going to see a lot more code, and the LLMs will also have that kind of instinct. They will install whatever they need to get the stuff. That's what they've seen, right?

SPEAKER_00 12:37

That's what they've learned to do to get functional programs.

SPEAKER_01 12:40

Number two, we see models, especially like ones like Mythos now, who are really good at injecting, you know, finding security holes, injecting them, and people are going to use that to find new holes and find new ways to um, you know, hack into the system.

SPEAKER_02 12:58

Yeah.

SPEAKER_01 12:58

So I think just for that already, the system will have to evolve because there will be pressure to maintain that ecosystem in a in a place that is sane, yeah, to bring in tooling and guardrails for for those system.

SPEAKER_00 13:15

So let me try and interrogate this like in so okay. I have two completely opposite and incompatible with each other reasons that might not turn out to be the case, right? So in one scenario, almost everything about the way the software world is set up uh is is there to maximize code reuse because the prior is software is precious, right? It took the sweat of some poor programmer in San Francisco's brow to get this thing to work in the first place. You'd be a fool to think you can do better in most instances. So don't. You just import the thing, whatever. Um in a world where software is not cheap, or software is not precious, rather, and where you can throw everything out and start over again whenever you feel like it, why all the code reused? Like, like why not, you know, why not make a little JSON parser when you need to parse some JSON? And why not make a little YAML parser when you need to parse some YAML and so on?

SPEAKER_01 14:08

Still cheaper to share. You know, it's it's it's like uh imagine I made the same argument in programming, right? I told you, you know what, RAM is cheap. Uh why, you know, reuse some things that you've computed when and not just record so obviously there are trade-offs.

SPEAKER_00 14:24

And output tokens do cost something, it turns out too. So yeah, yeah.

SPEAKER_01 14:27

So I think there are trade-offs. There are cases where there are these algorithms on you know, SAT solvers and things like that. And um, I I used to be really interested in those in the early 2000s. And my instinct as a functional programmer was always, well, these things need a lot of backtracking, right? So it needs to try something, and then once it is tried, it needs to go back into the state where it was to try something else. So it's like obviously you should be using persistent data structure for this stuff. And I tried, you know, getting some of those algorithms, like you know, um Davis Putman and these kind of things to work with functional data structures. And it was way, way, way slower than undoing the work you'd done. Right, and just redoing it. And just redoing the other, well, uh minus one basically of your operation and then go back to do to to try something else. Super frustrating for me. I was like, so really this whole persistent data structure makes you know taking making checkpoints cheap is yeah, it's cool in theory. I mean, no, there are some places where it works really well, but there are some others where the trade-off is just not good. So I definitely think that for LLMs and code, there will be things like that. There will be moments where the LLM thinks, you know what? It's going to be too complicated for me to figure out how to set up this or that. So let me just regenerate the code that does this. But in general, no, I think you still want to share code because tokens are not free. They might uh they might be cheaper than a human being, you know, typing things with their little fingers, but they're still not not free.

SPEAKER_00 16:09

Yeah. There's a uh and then I I guess the opposite, I think you're right, probably, right? Um, I think there's a one of the things that people seem to get confused about when talking about artificial intelligence in general, especially sort of extreme extrapolations of it, right? Like way better AIs than we have today, is to assume that uh just because it's very smart and and perhaps also very plentiful and getting more plentiful, that it's literally free. You know what I mean? Like there will always at least be the energy required to run the model and so on. And so you will have to kind of out the other users of those resources to get your thing through. So there still is gonna be an economy of effort, even if the intelligence is way, way better and and way more abundant than we can imagine right now. Um so I think it's still gonna matter to save effort, and for that reason, there will still be code reuse just because like we're using salt problems as useful to artificial intelligences too. At Slack, so you guys uh had a hack conversion. We did. We went through a we did go through a uh so hack kind of is to is to uh to PHP as TypeScript is to JavaScript, if you'll allow me to say so. Although you got there first for the record, because you're probably too modest to say it. Um but as a gradual typing system that's that's sort of oriented around how PHP programs tend to be written. Um and Slack, a little bit like Facebook at the time I started, was a grown-up Lamp Stack app, was uh you know, Apache MySQL PHP. Um the founder, one of the founders, Cal Henderson, had had a bunch of success with that at a previous startup, Flickr, right, when the consumer photo sharing site on the internet um basically saw no reason to change that. And around the time I started as as chief architect in 2016, you know, we're starting to see some. Wait, chief architect. I love these titles. Sorry, sorry.

SPEAKER_01 17:52

I love these titles. I I'm gonna can I call you chief archite architect for it. Yeah. Oh well. I love those titles. It makes me uh think of, you know, um uh what what what religion is this? Is it Franc-Maçon who have that? Oh, the the the the Han Masons, yeah, the big architect, you know, and so I I it reminds me of that, so that's why I always find those titles funny.

SPEAKER_00 18:16

But please, no, no, no, it's fine. I had a weird, I grew up in a Marine Corps household, by the way, and like the modifiers you put in front of like enlisted ranks pile up very similarly to the modifiers you put in front of individual contributor titles, basically, right? Like if you're a staff engineer, obviously.

SPEAKER_01 18:32

Ah, yes, there's that true senior this, senior. Can I be a public static staff engineer?

SPEAKER_00 18:36

Yeah, can you be a senior staff chief? Whatever, right? Um, so I mean though the way that we've like I when when that offer came through initially, like when Cal kind of sat down and negotiated like what that meant in the context of Slack, like can mean different things in different places. Um and when we kind of like workshop this definition that was like, I'm accountable for Slack's technical strategy, right? So I'm accountable in the sense that like I'm prepared to give account, right? Like there are other parts of the company that want things, and and when things break, I need to be able to articulate what happened and what we're changing. Um and then the technical strategy part of it is like I need to be kind of aware of what the heck we're trying to do, what what you know business challenges we're trying to address in the medium term, and have the tech roadmap at least align with and hopefully, you know, anticipate and improve that course of action. I'd say like in hindsight, um I give myself like a B minus or something. I like I think like I didn't, you know, I don't think I drove the Ferrari into the ditch completely, but like uh there are definitely things I would do differently. But the hack conversion, uh since since you're asking the PL question, I don't know if we've like ever sort of gone blow by blow through much of this, right? Um around the time I was starting in 2016, some of the problems that you tend to run into in big PHP code bases were starting to show up, right? And so So how big was the code base? Oh, uh good question. Uh order of a million lines. Right. So uh in the hundred to million order magnitude, I'm confident. And it was running HHVM already or it was running PHP, it was running Zen, PHP 5, I think we were stuck on 5.3 at the time, 5.4 maybe. Um and uh and changing PHP versions was a whole thing and so on. Um so if you've never worked on, and I guess people these days like might have had their whole careers kind of in a TypeScript code base or something like that. If you've never worked on a big, entirely dynamically typed code base, it's really, really daunting, right? If if you've never done it, you good for you.

SPEAKER_01 20:32

Yeah, yeah. Good for you, you should be happy about it. That's right. That's right. And strive never to have to do it. But yes, it's daunting.

SPEAKER_00 20:39

So let's say, for instance, you um, you know, the code base written nine times out of ten does the order of arguments, you know, user ID, then permissions object, then you know, object you're acting on or whatever, right? And there's just like some place in the code base that did it some other weird way.

SPEAKER_01 20:55

What about test coverage? Because usually that's what determines if it's possible to it, you know, work in a big dynamic language code base. So, how was the test coverage?

SPEAKER_00 21:07

Test coverage, I think around the time we were talking mostly in terms of line coverage. Um, it was order of 20%, and we had sort of a continual like quarterly OKR around increasing test coverage all the time. This ended up sort of uh making me cynical about the sort of Goodhart's law properties of tracking that metric too too closely, right? So, Goodhart's law, of course, right, is like when you make something, when you attach incentives to things, it stops being a useful metric. Um, right. So if we start attaching sort of the proxies of success, including like raises and you know, getting pats on the head from people you respect to test coverage. Well, one way to cover the code is to like have all unit tests just be a trace of the implementation, right? Like Yeah, people are gonna gain the system. Yes. So the the the thing is like I'm not I actually that'd be one thing. It's one thing if people are cynical and they're just like, ah, I'm just gonna get my bonus and blah blah blah and write these bad tests. That's actually not what's going on. I think it's actually A more subtle thing where if you sort of drop testing culture on an engineering culture that hasn't had it before, people are like, okay, we're supposed to write tests, and they don't really have like the um I guess the sort of interior model of like what's desirable about like what makes a test good or bad.

SPEAKER_01 22:17

Um, I think there's that, and there's also the tools that kind of shape how you see the world and the programming language as well. So this is, you know, I've I've I've sorry to go on a tangent. Yes, I've been in in in these debates a million times, right? Like you've got a dynamic programmer who tells you, well, I don't need a type system because I don't make typing mistakes, and anyway, I write the tests, I have to write tests, and the tests are gonna catch the typing mistakes that I would be making. So, really, you know, the the the type system is kind of wasting my time. Right. And you know, to somebody who comes from a language like Java or C or whatever, that seems to be incredible to hear. You're like literally not credible, right? Yeah, literally not credible. They're they're thinking, you are lying to me. Like you need the type system, I use it all the time, it catches mistakes for me. And the thing is, the tool that you use shapes the way you think and also the way you see the world. Um, so if you are using you know dynamically typed language all the time, what is going to happen is that you know that things are gonna fail all over the place, and you you will write code with tests that are very defensive. So you're gonna really test everything. And if that's your style of programming, and I give you Java, you're not gonna change your style. So you're gonna write all these unit tests, you're gonna check for all the types of everything, and then you know what you're saying about Java, which is that the type system is a pain in the ass that gets in the way, is true for you because you're actually in the habit of writing all those unit tests, right? And the same is true the other way around. So if you take somebody like who comes from a statically typed language, they're used to not test the types for the input parameters or things like that, because all of this is checked by the system. So they come into Python and they don't change their style of programming, and they realize that you know things break all the time and things don't work the way the way they're supposed to. Now, I would argue that in a large code base, there will always be a large enough fraction of people who are not disciplined enough that every dynamically typed code base I've seen is bad. Like I I I've seen small projects where this works because they're disciplined enough to make it work. But in larger code doesn't scale, right?

SPEAKER_00 24:55

Like just saying like be a good person, care a lot about the code, being right, like that. You can't enforce that in a huge organization.

SPEAKER_01 25:02

And I mean, I've I've seen that in other places too. You know, there's the functional versus OO stuff. So I've seen that a lot where you see people fight because they see the world in different ways, and they don't realize that the tools that they use and the problems that they use to solve has shaped their understanding of programming. Another good divide is system programmers versus non-system programmers. So let me give you a few examples. So if you are a system programmer, you see the world systems programming.

SPEAKER_00 25:35

Yeah, we're growing, yeah.

SPEAKER_01 25:37

Uh you you see the world through memory, right? Like what you see in your head when you're programming is memory because the memory layout is basically everything. Like, this is what you have to care about because that's how you're gonna write a driver, that's how you're gonna write a garbage collector, that's how you're gonna write all that stuff, you know, or a kernel, all this stuff. You need to know exactly how memory is laid out. So that is what you see in your hand when you're programming, right? And so now somebody comes along and tells you, yeah, we're gonna abstract that all the way. You know, this is gonna be called prologue or hassle. And and you just, you know, define what you want it to do, and and then the system programmer goes like, that's not programming, like this is light. Yeah, exactly. Like this is wrong.

SPEAKER_00 26:23

It feels like you're yeah, nobody can tell me like what what what machine instructions correspond to any of this?

SPEAKER_01 26:28

Like, and so the system programmer comes in and you give them a problem, and what they're going to do is think in terms of memory layout, and then they're going to structure things this way, right? And so if you give them a garbage collector, it's they're like, This is already too much, right? I I want to know more. But you see that divide also with uh functional programming versus old programming. So, for example, to a functional programmer, everything is about types. You need to know the types of things, and that's what's going to shape your understanding of the program. Once you know the types, once you have the signatures of the functions, the the functional programmer kind of feels like, yeah, this is this is my understanding, my abstract understanding of the program. Except that there are different ways to abstract the same program. So in the OO world, they don't care about types, they care about functionalities. So what they want is they want to know what is the API, what can I call, what can they do with this thing? But what the shape of this thing is doesn't matter to them. That's abstracted away. That is, and so when you see those two kinds of people trying to um basically um they switch to a different world, you see exactly the same frustration. So the functional programmer arrives in an OO code base and is like, what is the type of this? You know, what does this program do? I'm calling this method, I don't know which method I'm calling. That's right, what it's inputs, what it's outputs, it's like all just going on here. Like, help me, you know, and the OO program is totally cool about it. It's like, no, but you don't need to know what's calling, you just need to know what it does, you know, right? And the other way around, right? Like you've got the the OO programmer who shows up in a functional code base and who'll who have similar frustrations. And what is very interesting with uh all these cognitive biases that come from the tools you're using and the language that you're using, is that I've rarely seen any programmer who understands those biases, who's conscious who consciously aware of those biases.

SPEAKER_00 28:44

Yeah, like you can know this the same way you conceptually know about yourself that you have other cognitive limitations, but then you go out and just like enact in those cognitive limited ways anyway.

SPEAKER_01 28:53

Exactly. So what I've seen is you know, the C guy who really thinks that everything should be written in C, and anybody who doesn't use C is an idiot. Right. And it's just no. We call those people correct, right?

SPEAKER_02 29:06

We call those people who can program, people who actually know how to program and know how computers work, right?

SPEAKER_01 29:11

I mean, system programmers are really interesting because there's also some kind of hierarchy of you know how noble the software is.

SPEAKER_00 29:20

Yeah, the less feedback you have, the more.

SPEAKER_01 29:23

But I think it has to do with the execution stack, okay, which is that the if I'm using software that you have written on a daily basis, but not the other way around, in the software hierarchy that makes you more important.

SPEAKER_00 29:37

I'm a linear person, I agree.

SPEAKER_01 29:39

At least that's how system programmers see the world. That's right, that's right. And so for system programmers, they're thinking, well, I write the kernel, bitch. You know, I'm I'm I'm the man, you know. Like, what are you doing? Everything you do uses my stuff, right? I I don't care, but and so it it's it's really interesting, those hierarchies that there's also the um I'm sort of a reformed kernel person.

SPEAKER_00 30:03

Like I expected to end up being doing kernel stuff.

SPEAKER_01 30:05

So, for example, for kernel persons and and system programmers, one thing that drives me absolutely crazy is how shitty their fucking APIs are. Sorry for my French, but basically the it all comes back to the it's for good reasons. It's basically what happens is they optimize for speed, uh for memory consumption, for basically the hardware. They don't optimize for the human. Let's say you're in Java. If I take a library in Java, I look at the name of the methods, um, and I read the docs quickly or whatever, chances are high I'm gonna get it right without reading all the time.

SPEAKER_00 30:47

This is a game engine or a you know trading AI thing.

SPEAKER_01 30:51

I mean, even going that uh we can go more into details, but let's say you know I have a uh a method that starts a process, I assume that if I just call this method without any of the optional parameters, it's probably gonna do the right thing, right? And in in in system programmers' APIs, it's almost certainly never gonna do the right thing. So you've got two ways, two annoying ways. You've got the one where you're trying to do something very simple, and 99% of the users will try to do these very simple things. To get that simple thing to work, you need 25 different calls and setting up, you know, and and and everybody copy-pastes the same code that they found on the internet to get that set up. So that's create process in Windows, for example, if you ever use that.

SPEAKER_00 31:41

Um IOctol and like the family of IOctols on Unix, right around.

SPEAKER_01 31:44

And and the other example is the API has just all the wrong defaults. You know, so for example, I discovered the last instance of that was I discovered recently, not that recently, maybe 10 years ago, that you could now use locks within shared memory. So this used to be something completely S-Rex that nobody was doing was definitely not supported by the code.

SPEAKER_00 32:11

You guys aren't doing this, are you? You're not doing this sounds really scary. We're not doing that, okay. But you can do that, right? And so and if this is System 5 shared memory segments that we're talking about, and uh which is like a little bit of a wort on the Unix API to start with, arguably. Well, yeah, yeah.

SPEAKER_01 32:27

And now you can tell at the initialization of the mutex that this is actually in shared memory. And so if you have different processes that m-mapped that same same part of memory, they can actually use that as a lock. Okay. And so this is now supported by Linux. Um, this is via m map, not via system five shared memory or it's uh it's no, it's uh system uh shared memory.

SPEAKER_00 32:50

So it's an Mmap. Mmap map shared, but not the special funny namespace that's not the file system that is the that's the shhmem calls. Okay. No, I'm I'm showing I'm showing you. It's just a regular Mmap. So relieved, okay.

SPEAKER_01 33:03

Regular Mmap with uh shared. I I misunderstood what you meant. So regular Mmap, that is just shared that you can share across different processes. And it used to be that you if you try to allocate a lock in that memory, the lock was basically not working, right? And so now it's supported and it has been supported for a long time. So I was very happy about that because I needed that. And I, you know, do the stupid thing, which is that I did not spend my whole day reading the man page. And you know, like the the system programs were there to punish me because they don't like that. That's right. You are supposed to read the whole man page and and read it multiple times. And and I didn't do that. And and then so what I would expect is if one of the programs crashes, one of the processes crash, it releases the lock. Right. And it doesn't do that by default. Yeah, yeah. Of course not. Yeah, yeah. Why would it? Why would it do that by default?

SPEAKER_00 33:59

Well, might that be the safe behavior? Maybe it needs to still have exclusive access until it restarts.

SPEAKER_01 34:04

And I mean, when you think about it, what probably happened is that it probably costs something extra to do that extra work when the process is is uh is um killed. And so they decided to not make that the default for those who want, but it's fucking stupid. Like that's not what people want. And so to go back to how system programmers think, they do not think in terms of what do the people who are gonna use my API want. They're not thinking those terms, they're thinking in terms of I think that's the library author's problems.

SPEAKER_00 34:38

Exactly. That's actually the that's actually how you do think about this. Is like you uh because I've been in the position of having to build these horrible systems that you're describing in various ways. And like the thing is like you never get performance back if you give it away, right? Like there's no there's no lever that lets you like be like, oh whoops, we like this inevitably, and it's like a store, right? Maybe that maybe it's literally one instruction, but the semantic of the way we define this means we just absolutely have to do this work in this situation where lots of people don't need us to do that work, and so you want to say, okay, it's all a libraries problem. And I mean, I grew I'm um I am one of these people that you're talking about, obviously, first of all, right? And I grew up in kind of uh, you know, you look you were educated in O'Camel, I was kind of educated in C, not because because that was how these countries were doing it at the time, right?

SPEAKER_01 35:21

Like you just my first language was C and Pascal. Okay. And I discovered so I So you but but you so you understand how computers work. And yet I understand how computers work confusing opinions. Then what when that then what happened is that I learned OCaml much later on. Uh and back then, uh the first time I tried it, I didn't like it. So first it was Scheme. So Scheme, when I learned Scheme, so I I got you know uh that oh god, the name the name escapes me off the Lambda book.

SPEAKER_00 35:51

Um yeah, uh the Sussman book. Yeah, exactly.

SPEAKER_01 35:54

Uh interpretation and exactly uh and I structural interpretation computer programs, that's ICP.

SPEAKER_00 36:01

Sorry, it's gonna drive me crazy.

SPEAKER_01 36:03

Uh and so the first time I got my hands on that book, I was maybe in my first year at university. Okay, and back then it was still in Scheme. I think they have a version in Python now that MIT does.

SPEAKER_00 36:14

Okay, interesting, interesting.

SPEAKER_01 36:15

I I don't know. But and I discovered Scheme and I loved it, and it was you know abstracting. So what I loved was the simplicity of it, and then I got hooked on functional programming, and after that came O'Caml. Yep. At first I didn't like OCaml because I didn't like the syntax, so it was a very shallow way, you know, shallow reasons not to like OCaml. And then I got hooked on OCaml, and so this is going to be interesting to a C programmer because when you're writing C, what you do basically is you write your program and then question for C programmers out there do you start it when it's the first time you start it? Do you start it from within a debugger? And to tell the truth, like I just started from within a debugger when it's the first time I tried it because I'm pretty sure it's gonna stay full. You know, if I wrote something non-trivial and then you spend a ton of time debugging, and I never like debugging, like I and some people love it.

SPEAKER_00 37:13

Oh, you're right. This probably is actually there, this is an interesting cleaver that might sort of separate the world of sort of natural systems programmers from yeah, yeah. It's like, would you rather just work? Or is it actually kind of fun to investigate?

SPEAKER_01 37:24

I mean, they are people who just love this stuff, they love to debug. Uh, they love to you give them, you know, um the secfold file where is that called the core dump. Yeah, yeah. You give them a core dump, they investigate that.

SPEAKER_00 37:37

I don't know how little languages have core dumps. These are useful. It's the most core debug things, right? You want a user to be able to give you here's the serialization of the program state when it failed. Why is it all just a stack trace?

SPEAKER_01 37:48

It's true. Uh I it's a good question. It's probably too expensive when there's a VM, maybe getting the states of the VM, because I think it when you have a VM, you also need, you know, the core dump or the VM, and that might be too large. I I don't know. But basically what happened for me is that I was used to programming this way, and with scheme even more so, because in Scheme it's all dynamic, so you have to debug all the time. And the first time that got me hooked on OCaml is that I wrote something non-trivial in OCaml and it just worked, you know, the first time.

SPEAKER_00 38:26

For me it was Haskell in 1998, by the way. Like it, but yeah, the similar thing of the same thing. I wrote all this crap, and like once I got to compile, it just worked. Exactly.

SPEAKER_01 38:35

And the first time this happened to me, I was like, oh my god, there's no going to know this is what I need now. And then then I got and and true, I think I went a little bit too far with types. So I I I like types because they they helped me write correct programs, and then I started trying to encode everything with types, and then I get to got to the point where I did discovered dependent types. So I was like, well, wait a minute, I can have any spec I want in the type. And then I was like, I remember bursting into the the the office of my my uh my professor back then telling him, we're going to prove everything. That's right. This is gonna be awesome, there will be no more bands. That's right, it's going to be a wonderful world. And he was, you know, smirking back. He was like, So what it is you're gonna prove exactly like well, everything.

SPEAKER_00 39:32

And this is actually sort of one of the things I do, like circling back a little bit to LLMs, but not in a super hard way. You know, types are a kind of lightweight formal system, right? Like they're a lightweight formal method, right? They're a formal method that's kind of built into your tool chain, and so you're using it kind of by default. Yeah. But at some level, the type checker, that part of the semantic checking of your program passing is a theorem about your program, right? Like is it is a sort of claim about? Yeah, there you go, there you go. Um yeah, people are smart, man. Like lots, lots of smart guys. Um one of the things that has been interesting to me about this like last six or nine months is that people that I think of as extremely resistant to or extremely skeptical of you know, both static types, but also other kinds of formal methods, including like building lean forebroofs and stuff like that. Some of them are trying to do this, right? And and and some of them are having some amount of success, right? So like uh like mutual friend Ken Peck, who of course like is one of these unit testing people, like you know, has has a lot to do with the fact that we call him unit tests like built J unit and kind of popularized TDD. TDD. Yeah, and uh I'm sure it's one of those things with many fathers, but like is certainly one of the people who is out there, you know, influencing people's thoughts about these things. And uh, you know, it's like last time I I had a conversation with him, I was actually on his podcast still burning a couple of weeks ago, and you know, he's was building a B tree and and building a little lean for just that the B tree doesn't supposed to do, and kind of feeling like that was working, kind of feeling like this was helping, even though he, you know, didn't know much about lean four especially and didn't know much about formal methods especially, kind of was able to get up to speed along the way. And I'm curious like whether that ends up being what we expect, right? Because I mean, yeah, like you, when I review a ton of even you know, fable code, right? Like there are defects, there's things where it misunderstands your intent or misunderstands what it's supposed to do. And if like everything we did was accompanied by some sort of proof that that the things I care about are actually upheld by this program, you know, does that become a thing that either else wants, that their users want?

SPEAKER_01 41:38

So, first of all, for what I'm about to say, I think half of the employees of my company are are gonna want to kill me because they love formal methods and they want more formal methods everywhere. The other thing is that I think formal methods is super useful. I and I find them even beyond useful, I find them beautiful.

SPEAKER_02 41:57

Right.

SPEAKER_01 41:57

Like I think, you know, the first time I used um uh I used uh cock to prove that you know um some algorithm on an AVL tree was actually correct.

SPEAKER_02 42:10

Right.

SPEAKER_01 42:11

Um so it was actually the uh in the book called The Cock Art by uh Yves Berthault and Pierre Castellan. So if you have it's just French for chicken children. So if if you have yeah, it's uh French for chicken, it's not the kind of cock, it's the good kind of cock.

SPEAKER_00 42:26

It's the one that's in Q.

SPEAKER_01 42:28

C O Q. Yeah, and I think they changed the name recently. Yeah. And so if you haven't read that book, this is a really, really good, it's my one of my favorite computer science books out there. And I think not not very many people know it, but there's something extremely satisfying in getting the proof that whatever you've written is is correct, right? Now, the thing is, my experience with those things is um depending on the so there are two two limitations, two things that I think are going to be a problem for LLMs, um, because they are problems for human beings. Number one, there are certain classes of problems where uh uh trying to specify them formally is actually harder than solving the problem itself. So, of course, if you're trying to sort numbers or writing a B tree, like you just said, the properties are nice mathematical properties, right? So if I want to write the spec for a sorted array, that is really easy, right? For all i da da less than uh uh i plus one, and you know, done. It's and not only is it easy, it's actually really readable. Like another human being will be able to look at this spec and go, like, yeah, that's a sorted array, right? Now they are in B tree, we all know the properties of B trees or red block trees or whatever it is that you're trying to build. So I think for those kind of problems where you have a well identified data structure and well identified properties that you want to prove. Those formal methods are going to be bomb and they're going to be used all over the place, probably by LLMs more than humans, because it's a lot of work.

SPEAKER_00 44:09

What is something crazy you believe? Like what's what's something where you're mad like on the one hand this, on the other hand, that Julian Verlegay is the only sane man in the world that's gone mad?

SPEAKER_01 44:18

I should have prepped that. I don't know. Uh I don't know if there is something like that that I believe that is really controversial.

SPEAKER_00 44:27

I think you were trying to get there with local LLMs.

SPEAKER_01 44:30

No, because I don't think that's really crazy. You're not maxed out. I know, no. And when it comes to LLMs and AI, I don't think I have enough context. I don't think of myself as enough of an expert to have those kind of crazy ideas. But um no, I used to have a crazy idea, which was you know, placade a type system on top of a dynamic language and make it work. Yeah, yeah, sure. And back then I was, you know, for most people at the company, I was classified as certifiably insane.

SPEAKER_00 45:01

It was a pretty limited project, yeah, yeah. You were like the fifth, it was like like uh mathematicians and angle trisectors or whatever, right? Like I had you know a stream of people saying I'm gonna put a type system on top of PHP coming to talk to me. You were like the fifth one, and like you were the one that had to be. You were the one that had like the answers to like the first question, right?

SPEAKER_01 45:24

It was like, yeah, yeah, kiddo, yeah, yeah, we we've had it all before sure. Exactly. But no, I mean that was definitely a crazy idea. I used to have I before that idea, another crazy idea I had was I was obsessed with linear types, and I really thought that linear types were gonna solve the problem of garbage collection. Yep, and it was before Rust was popular, so I wrote this compiler called Linear IML. And so what's interesting is that um I think history has proven me right, like Rust is a thing, but I was hoping so I was I wasn't hoping to only do system programming with it, and I was hoping that the scope would be much larger. And for me, I when when I was working on Linear ML, it as I was using it, it became pretty clear to me that this was never gonna become mainstream to the point that it was going to replace Java because it was just too difficult to use. I think that around that time Rust was starting to pick up. And then I looked at Rust, which had a scope that was uh um more narrow, which was we're going to replace system programming. And I looked at what they're doing and I was like, I think that's right approach. That's that's what they should be doing. Um, and I think also what Rust had going for it was um C11 was picking up. Well, no, C11 was Rust was already had been there a couple of years, but when C11 came out, there was a huge boost for Rust because Rust was basically a formalization of all these principles. And so, but I think yeah, so the crazy idea I had in 2004 was hey, we we should use linear types um to manage memory statically. Right. And then went full in on that. I wasn't the only one having that idea. In fact, you would you could claim that you know Girard would be the first one. Girard was saying that in the 80s. Okay.

SPEAKER_00 47:32

So I actually maybe not the philosopher that Peter Thiel's into, but uh Girard.

SPEAKER_01 47:39

He's um French um uh logician.

SPEAKER_00 47:42

Okay.

SPEAKER_01 47:43

Uh and um so he is the one who invented system F. So I don't know if you heard of System F.

SPEAKER_00 47:49

I do know of it, but uh that's a deep cut. So like why don't you bring me back up the system?

SPEAKER_01 47:53

So yeah, so maybe I can explain to you. So basically, when there was first Lambda Calculus, right? And Lambda Calculus um Church came up with a very basic uh type system for it, which we now refer to as simply type lambda calculus. And that type system for Lambda Calculus was uh very restrictive, like you could barely type any program with it, but it was just checking that the shape of things was correct, and just writing a fixed point operator with it was was already out of the reach of what the type system could do. And then so there was a stream, uh there was all sorts of people who worked on building more expressive type system. Um, one of the famous ones is the one from Girdle called System T. And System T was a good extension. Um, and the the main thing that they're trying to do is add looping and well, add a fixed point operator that works. Um when Girdle did that, um the people started looking at it and thinking, well, is this expressive enough? And that's how they came up with uh Ackerman function, which we now know very well because the Ackerman function was there to show that system T, Girdle's system T was not expressive enough to express all sorts of looping. And basically, there was a lot of back and forth, there was a lot of trial, and it all basically finished with system F. And system F was basically what's still considered today, I think, the most expressive Thai system for Lambda calculus, where you can pretty much express whatever you want. And so that was one of the main contributions of Girard. But after that, he worked on linear logic, and linear logic for those who are not familiar with it, is basically something where you it's a different kind of logic because whenever uh so how can I how can I talk about this? Basically, what you can notice is that if you use natural logic in certain circumstances, it will lead to incorrect results. And usually the way that happens is whenever you talk about resources, right? So let's take a resource that everybody knows, let's take money, right? And I'm gonna make um statements that involve uh money, right? So I'm gonna do the classic thing of A implies B. So I'm gonna if A is gonna be a statement, B is gonna be another statement, and I'm gonna say A implies B. So let's say A is a statement, I have two dollars. So you have two dollars in your pocket, the statement A, right? Then B is the statement about what you can buy with it. So let's say it can say B is I can buy a chocolate bar, right? Then I make another statement, C, which says I can buy you know true in gum. Right. So if I follow the rules of that natural logic, if I actually say A implies B, which is if I have two dollars, I can buy a chocolate bar, let's say this was true, then I have a second statement that says A implies C. If I have a chocolate bar, I can buy true ingum.

SPEAKER_00 51:04

That'd be B implies C though, right? Oh, if I have a chocolate bar, no, uh A implies B, A implies C.

SPEAKER_01 51:10

Okay. If I have those two things together, in natural logic, I can deduce that A implies B and C.

unknown 51:19

Okay.

SPEAKER_01 51:19

So I can deduce that if I have two dollars, I can buy a chocolate bar and I can buy a chewing. I'm following. And that is obviously incorrect. I mean, you can go try out the gas station and tell them, show them your little reasoning, they're gonna say, no, sorry, get out of here. You know, we're not we're not giving you um those two things. Um, and the reason is two dollars is actually a resource. So whenever I write, if I have two dollars, I can buy a chocolate bar. Whenever that statement, that uh implication has been used in my proof, it has to consume the two dollars, right? And that's what Girard was interested in, and he developed a logic for that, but his secret plan was not to manage resources, well, it was uh he was he was particularly interested in a in a particular kind of resource, and that's memory. So basically what happened is he developed a logic where all implications are um changed, um uh modified to have linear implications, and a linear implication consists in taking the statement and consuming it. So now when I write A and price B, if I decide to use A, A has not used, I used up A. Okay, so now I cannot go back and use it for something else. Right now, how do you express it's it's actually strictly more so then you have operators to say when you want to duplicate A or shrink A. And because thanks to those operators, you can actually express all of natural logic. Okay, because in natural logic, all you have to do is say um duplicate A and R times explicitly, and then you'll get to deduce all the things you would have deduced in natural logic, right? So it's strictly more expressive than natural logic. Then then at this point, you're probably thinking, what is the relationship with you know memory? Where are we going with it? And so the thing is, so I have to jump to programming through Curry Howard. So for those who are not familiar with it, Curry Howard is a bijection um that was invented in the that was discovered in the 70s, that basically says that a type is to a program what um a theorem is to a proof. So what that means concretely is that whenever I tell you, hey, I have a function, it takes an a and it produces b's, um, and you with your little fingers write down that function, you are actually doing exactly the same thing as if I told you, hey, I have a theorem that takes a's and produces b's. Can you write the proof? Writing that proof is the same job as writing the the function. And the the the good example to convince yourself of that is to take basic examples involving implications. So uh we we can do one together so that you you get the idea. So let's say let's say you know I have a implies b and a implies c, right? Um, and I want you to write me something that's uh a proof that a implies c. Uh I want you to write the no, wait, you have a implies b, b implies c and well, actually we can do simple. We can take you take an a, you take a function that take given an a produces a b, and you have to produce b's. So let's say you were writing that function with your little thing, right? Your first parameter is an a, your second parameter is a function. Well, what you would do is you would apply the a to the function and return that result, right? Right? It turns out that this exact thing that you're doing has an existing rule in a proof theorem that would be called modus ponens that does exactly that, right? So I can take your program and derive uh uh a proof from it. It the only thing you have to remember is proving that A implies B is the same thing as writing a function that takes takes the given an A produces a B.

SPEAKER_00 55:39

Right. That's a really interesting uh usually like when I ask people for like something crazy they know or or know is true. Um it's sort of a hot take, basically, right? They're sort of like, well, you know.

SPEAKER_01 55:49

So I didn't have a good hot take. So what I did is I took the hot take of somebody else in the past, which I thought was really cool.

SPEAKER_00 55:56

And no, but you did real effort on it, right? Like you actually like committed a chunk of your intellectual work.

SPEAKER_01 56:01

That is true, but it wasn't really my hot take in the sense that there were people who were you know already believing in that. Uh, but yeah, I I tend to do that though. Once I have a hot take, I tend to just go, oh no, I have a hot take. So here we go. I think um reactive programming is going to be uh a very important part of the success of LLMs, especially in the field of coding. And so my hot take, well, and I'm cutting my money where my mouth is, obviously, because I'm building uh all the ecosystem to make that work. But basically, my take is LLMs will need to train on larger systems, and to train effectively on larger projects, they will need to be able to iterate on those larger projects faster. And so the latency of the tooling is going to become the bottleneck. And so right now I think nobody cares yet. I mean, a little bit like uh anthropic, but I think that goes in that direction that they start to think uh maybe we should own some of that tooling to help, you know, the feedback loop of the LLM. But I think it's going to become apparent, much more apparent once um the we want to train LLMs to do much longer tasks and work on much bigger projects by themselves. So right now they perform tasks on large projects that they usually didn't write themselves. But once we get to the point where we wanted LLM to write a five million line code uh in a five million uh line code base and maintain it themselves, once we get to that point, then to train those kind of models well, you will need to run experiments. And if every time you run an experiment, you have to wait for circle CI for 30 minutes, the training is not going to be very fast. So if you take all of that and shrink it. So, and what I mean by all of that is really the whole thing. So, what I mean is the first step is obviously type checking and linting, all these things that has to come back fast. Once this comes back, then you need to do static analysis to figure out which tests you have to run and only run the tests that matter. Once the tests passed, you will have to also make the program itself incremental so that you don't have to restart everything, fill up all the caches and everything, invalidate you know just the parts of the code that needs to be invalidated and re-recompute only the parts of the cache that needs to be computed. And then if if you can do all of that, which is what we are building at Skip Lab, so you can't have problems in computer science.

SPEAKER_00 58:47

Uh so yeah.

SPEAKER_01 58:48

If you can do all of that, then the change whenever you're doing so. Another thing that's going to be important, I think, is the capa the cape, um, the ability to branch. So you want your memory to be you want to be able to run like a hundred experiments at the same time. And kind of like copy on rip fork the big thing underneath it. Yeah, yeah. And so once you have all of that, that means that when the the the AI wants to try, you know, let's say a hundred different things that would have taken half an hour each, and all of that takes, you know, step second. That the model who has this kind of tooling will beat all the other problems. And so, but it's it's the history of optimizing pretty much everything. There's always you know a bottleneck, and then all your resources go on that bottleneck, and you only start investing in the next bottleneck once you know things when when you basically get to it. And so I think we haven't actually gotten to it yet, but we will get to that point where the bottleneck is the tooling, and when that's the case, then the latency of the tooling is going to be the key, and the only way to build tooling with low latency is with reactive programming.

SPEAKER_00 1:00:07

If you were uh if you met a 19-year-old version of yourself, except it's 2026, um, and you're yourself and you're brilliant, and uh you're in France somewhere, you're you're you're a brilliant man, seriously. Um we'll skip over the false modesty. Um and you know, you've got a lot of potential anyway, and and a lot of opportunity. And you love all this stuff, right? You're really interested in PL, you're really interested in in computers, you're really interested in things like computers, the machines, but also computers like the kind of conceptual frameworks that are unlocked. What would you advise that younger version of yourself today to go do? Is it still just like do your informatics degree, you know, you're on the right, you're on the right path to keep going? Um, or does something about kind of this you know, whole LLM milieu massively change the right way to allocate your energies?

SPEAKER_01 1:00:57

Yeah, I don't know. I mean, we're talking about a younger version of myself who would be young today, who would be starting to. That's right, yeah, yeah.

SPEAKER_00 1:01:05

So you're 2026.

SPEAKER_01 1:01:06

Okay, Julian. And I'm 19 and 2026. Yeah, so I would definitely add more time, you know, working on data science and spending more time figuring out how all this stuff works.

SPEAKER_00 1:01:22

All this stuff being like kind of machine learning.

SPEAKER_01 1:01:24

Yeah, machine learning. Yeah, figuring out how machine because I I came pretty late in the game for these kind of things. Like I learned seriously about them in 2016, 17, something like that.

SPEAKER_00 1:01:36

Um and you waited for me to leave Fair before you started working on it. Thanks, man.

SPEAKER_01 1:01:40

Yeah, I appreciate you joined Fair after you left. So that was interesting. There was a time where I was thinking joining Fair in 2013, uh, right after Hack, where you were telling me all about these uh this cool stuff happening at Fair, and I I was very curious about it. So I would definitely tell the younger version of myself go learn what Transformer is, how this stuff works, go build an intuition, because that's the stuff that takes the longest time. Because for example, when I joined Fair, the way I like to learn things is usually by using IT. So the first thing I did is I rebuilt a framework for you know AI and rebuilt um uh an autograd and all that stuff because I wanted to understand how all the formulas were working.

SPEAKER_00 1:02:26

I had to make autograph in C, or I had to make a template system where it's like these things are, you know, and then you just I don't and I don't think anybody ever used it for anything. I hope they didn't use it for anything. But point is like we're engineers, we learn with our hands, you kind of have to make stuff to know.

SPEAKER_01 1:02:39

Same instinct, but basically what ended up happening is that I do know how these things work and I don't, in the sense that I do know what steps need to be taken to get a transformer to work, but I don't have a good intuition. I have a little bit of an intuition of why it works and why not, but I think what takes a very long time in it is to build an intuition where you know what kind of things are going to encourage the model, for example. And that that takes time. So tell my younger self, go educate yourself on that and become better at what along those lines.

SPEAKER_00 1:03:19

Like when I started doing this stuff at Fair, I think that part of it felt unlike any other kind of computing to me. Yeah, that's true. Like when I was I I remember I was working on a word embedding model that um was sort of inspired by WordDev, but was like, we're gonna use hashtags as supervision. So it's sort of a bilinear embedding system. We're trying to minimize the distance between a bag of words and the hashtags it co-occurs with or whatever. And like random CPUs and stuff, like very simple thing, right? And I kind of had something kind of working. You know what I mean? It was kind of like you'd put it up the tweet and it would kind of like get a hashtag that sounded plausible some of the time. And then we hired Jason Weston, right? We brought Jason Weston in, who's like a great M N great NLP guy, already very experienced at the time, now even more so, has done a bunch of amazing stuff since then, of course. But he kind of like looked at this and was like, yeah, this is a cool project. This should be better. Like for all the things you're looking at and all the parameters this thing has, this just like should be more impressive. Try like these seven things. And it was kind of like one of them was like, you know, clip the vectors to the unit hypersphere and you know, do uh use this learning rate schedule and well, and a couple of other things, right? And like all and you could do it in one step. It wasn't sort of like sit there and like retrain for the first one, refrain for the second one. It was all just like an improvement, and it was almost and it reminded me, I kept using the analogy of like, it's like a chef like tastes your chocolate chip cookie recipe or something, and it's just like you need more butter, you need more salt, oven needs to be hotter, but then at the end you should add some water or something, whatever like crazy stuff. And you do all the weird stuff they say, and it comes out way better. And it's not a digital kind of way better, it's this weird analog kind of way better, where just like these results are subjectively much more impressive, and it's still kind of like this. I mean, that we know more about the underlying math. Like we know, you know, you wouldn't just like clip to the hypersphere now. There's like smarter ways to do it and everything. Um distributed systems are a bit like that too. Actually, that's a good point. That's a good example. That's a good example.

SPEAKER_01 1:05:08

People who are good at that stuff, they will be able to architect something from the get-go that's just going to be much, much better than somebody who hasn't been bitten by those problems before.

SPEAKER_00 1:05:18

We can still build like a TLA plus model though. Do you know what I mean? Like there's there's still things in distributed systems where there's like a digital notion of correctness that saves your butt, you know. The performance part of it is true. There's a lot of taste and a lot of a lot of intuition around the around getting it to run well.

SPEAKER_01 1:05:31

And it also feels the same that they're when they want to optimize the performance, oftentimes they run experiments. So it feels like experimental science, right? Where they run experiments and in fact, performance work in general feels a little bit like that because the hardware is so complex now that you know nobody really knows what's going on in there. So you have to treat it like you know a black box and run experiments, and including the people who made the hardware, right?

SPEAKER_00 1:05:56

The hardware is so complex it has weather, the hardware is so complex that it you know gets. In the states, you know.

SPEAKER_01 1:06:01

But there's still people with good intuition, good taste who will know this kind of stuff is more likely to work. So it's not going to be a hit 100% of the time. But they will tend to have better ideas than you know the layman who just doesn't know anything about it. So it feels to me that I I would have liked if I was starting software today, I would try to build that taste A ASAP for you know these kind of systems or um anything related to AI.

SPEAKER_02 1:06:32

Yeah, yeah.

SPEAKER_01 1:06:32

Um and then I don't know. I mean, there are some people who say that you shouldn't learn how you shouldn't, you know, learn programming, pick up programming. Um, but I don't know about that, you know. Like uh I think pretty prominent people in that space have said that you know learning how to program is useless now.

SPEAKER_02 1:06:51

Yeah.

SPEAKER_01 1:06:52

I then again I'm super biased because I like to program, so maybe maybe uh I'm wrong and I'm just not acknowledging my bias, but I find it hard to believe. I I still think that we're pretty far from you know having the LLN just completely takes the jobs of all the programs out there. What I think is much more likely is that the program is actually going to become much, much more interesting, where you have an army of LLNs, an army of agents that are working for you, doing all the grunt work that nobody likes doing, like refactoring and you know, cleaning up some some stupid stuff that anybody could do, it's just a time consuming. Uh it will also help, you know, try out ideas, but I think that at least for the foreseeable future, I don't think it's going to replace the the engineers. Uh but you know, uh again, I've underestimated LM before, so I'm not because in my book I think we need a breakthrough. I think I said it in in another in another I've I've said it many times. I think we need another breakthrough like Transformers where like you know, uh where to like this kind of breakthrough to something that really changes the game to get to that last step. And many people think no, uh if we just keep on pushing LMs, we'll get there. Yeah. So I don't know, is it good advice to tell someone today to learn how to program?

SPEAKER_00 1:08:21

My best guess is still yes, to be honest with you. My best guess is that like even if uh so one of the one of the things I've observed with people using coding agents and LMs and these other kinds of you know massively overpowered, super high-leverage software construction tools is the more I think of somebody as having already attained real mastery of of engineering as a craft and science, the better they're able to use these tools, which is not a priori what I'd have expected, right? Like you might have expected that it kind of lifts the floor, but the ceiling is wherever it was. It goes both ways.

SPEAKER_01 1:08:52

Yeah, yeah. Those who were mediocre engineers are much, much more mediocre now because the LLMs are basically it's LLMs are like steroids, so they'll supercharge whatever whatever you are, right? And so if you were mediocre and you were pushing out shitty code, well now you're pushing out a ton of shitty code. That really sucks for everyone. And if you were a good engineer pushing out good code, then you know now it's it's like leverage, right? That's that's what the LLM does. It's an amplifier, it's an amplifier. And so it's true that I would tell people to go learn how to program, but I would put a caveat in it, I would say, if you're truly passionate about it. Because what I mean, I uh you probably heard this month before when I joined uh uh I was at Facebook and I was frustrated, so it was late state Facebook, and I was frustrated because there was a lot of people who I was interacting with who didn't care about um something relatively geeky, but you know, some programming stuff, and who like do you know, and and I was telling you, I'm frustrated because you know, imagine you're a kid and you like to play basketball, and your whole life you dreamed you dreamt of playing basketball, and you finally made it make it to the NBA, right? You've been in the big leagues, and you know, and and you have a plan, you want to talk about things, you want to talk about new ways about training, about new ways of this, new ways of that. And the rest of the team, bar maybe one, is like, dude, we're just gonna throw the ball in the basket, all right? Nobody gives a nobody gives a fuck. This is just a game, we're just going to throw the ball. You're annoying us with your new techniques, nobody cares, right? And there was a lot of that happening in software in Silicon Valley, in you know, because there was a lot of people who chose the job because it was a well-paid job, right? And so I think now with LLMs, having this kind of attitude is going to be very, very risky. So if you go into software because you think that it will be a well-paying job, but you're not really passionate about what you're doing, I would say be very, very careful. Because what's going to happen is that if you're looking, so typically if you're not passionate about something, what you're gonna do is you're gonna try to find a niche where you get to do always the same thing. And that's all gonna get automated away.

SPEAKER_00 1:11:26

Yeah, that's all in the in the day and age of it used to be you could have a whole career that was like I'm the build system guy. Exactly. I know everything about make, I know everything about CMake. Exactly. Uh, but maybe there's some new Fangal build systems, do they even work anyway? Who knows? I'm the make and C Make guy. Exactly. And that's what I do for this company, and it's enough, right? And I think that kind of is is it is structurally never gonna happen again. Exactly.

SPEAKER_01 1:11:48

So my advice would be if you are truly passionate about it, and in which case, my advice is completely useless because if you're truly passionate about it, you're gonna do it anyway. And I mean, that's why I started computers. I started programming because I loved it.

SPEAKER_02 1:12:04

Yeah, yeah, yeah.

SPEAKER_01 1:12:04

And then back so in the 90s, like computing was a subculture, right? Like and back then, you know, nobody would think that you would make big money out of it, especially in France. You know, it was more seen like a subpar engineering field where you would, you know, nobody saw this as a as a smart career choice. Uh at least not when I I I started like in the 80s when I was a kid. Yeah, yeah. And and I think, yeah, so the same should apply here. Like you the the right question to ask yourself is would you do it anyway? Then then you are you know you know you are you're in a great place. Uh the same is probably true with music. There are other things like that that I I I think you should only start if you're passionate about that. Yeah, yeah. So yeah, so one one of the differences with music though is if you start it for the money, well, that's probably a bad idea. The people who are passionate about it are probably gonna be so much better than than than you, Ali, that you're not gonna find you know a good job and you won't be able to make a living off it. And I think now computer science is probably gonna become like that. Computer engineering. It used to be that we needed so many little hands to write these little shitty things that there was space for a bunch of people who didn't care.

SPEAKER_00 1:13:26

Yeah, you'd have your 10 million line code base at your gigantic company, and there'd always be some project to move from this data store to that data store. And it was like some director somewhere has an army of people who are you know doing nothing but doing these relatively mechanical migrations. And if if that project was going slowly, that director could ask for more people and probably get them and so on. Yeah.

SPEAKER_01 1:13:47

Yeah. And and actually my theory is that directors love to have um useless projects. So I I've I've worked in enough big companies now to know that you know the director wants a big hat count.

SPEAKER_00 1:14:01

Yep. That's I mean, that is actually, and in fairness, that's the measure of impact.

SPEAKER_01 1:14:04

That's the measure of man's control if you're a manager. And and ideally, the problem is when you have people and those people are doing real work, it creates friction. Because real work means change and people are reluctant to change, and so it will create human friction that you then have to oil, put oil on, which is basically the job of a man the marginal chain, right? And so ideally, if you're a director and you want to grow your org, the best thing to do is to uh hire as many people as possible to work on something useless that you've convinced upper management that somehow was useful. That's right. That is the idea, that's where you want to be.

SPEAKER_00 1:14:47

We have this class of bugs that we've had in the past that really hurt us. It's because we're on data store fribble, and we need to get to data store boggle to get it, and once we get to boggle, this whole class of bugs goes away. Or we could take millions of lines of code. Rewrite is is is also a good one. And the Rust rewrite that's happening in the case.

SPEAKER_01 1:15:08

We can have a Rust rewrite that makes sense. So for example, um the one that Bun did recently, they had a bunch of uh memory bugs. You switch to Rust, you don't have them anymore. That makes sense. But when you go from a safe language to Rust, and that safe language didn't have performance issues to begin with, you're like, why are we doing this again? And and it's not gonna be done. It wasn't the plan, it was not to do it with an LLM.

SPEAKER_00 1:15:36

An army of people doing that. For those that have not followed the rip from the headlines bun reference here is it used to be in Zig, they landed a Rust rewrite at the PR. That's what you're gonna say. Um, so you know, probably uh probably to the chagrin of some open source contributors, I imagine, that might have had a PR open LLM author or otherwise. I mean, rebasing after rebasing after that one is gonna be it. I guess you'll let the LLM rebase it. You let the LLM figure out how to how to express your intent in Rust.

SPEAKER_01 1:16:06

So, you know, I mean, I mean, and and so for directors, I mean, talking about bad incentives, they have those those really weird incentives that number one, they need to get as many people as possible because that's how you know that's how they make more money. And so they have an incentive to uh hire more people than they need. And then they have an incentive for those people to do things that are gonna cause as little friction as possible, which usually means to work on things that are as you know not disruptive and that are useless. The more something is useless, the less people argue about it.

SPEAKER_00 1:16:45

And for those listening at home, by the way, Julian and I have worked at some big companies before. Uh, the that does tend to be a little corrosive to your uh idealism about what a job and technology can be like over time. I do want to say like those are also there is good work that happens there, and there's good teams that do awesome stuff. Well, thank you once again for joining us, Julian. It's been an honor and a privilege. Next time we'll probably be virtual, unfortunately, but we'll have to do this again. Um, you know, for those of us that that for those of you that listen this far, thanks for listening. And remember to smash, like, and subscribe into stratosphere, like Jose can say, go write it out of his mind.

SPEAKER_01 1:17:18

Of course.

SPEAKER_00 1:17:18

Um, and uh hopefully we'll see you next time on Computers Coffee and Beer. Thank you.