Dave Abrahams 0:00

The STL, in my opinion, is a singular work of genius.

Bryce 0:05

What do you think it was about the STL that made it so so novel?

Dave Abrahams 0:12

A sense of searching for some underlying truth.

Bryce 0:18

Yeah, no, no. We we have more time than that because it's just my mom and I do yoga twice a week, as Connor knows.

Dave Abrahams 0:24

Oh, that is very cool.

Bryce 0:25

She's uh she's pretty flexible.

Conor 0:39

Welcome to ADSP the podcast episode 48, recorded on October 3rd, 2021. My name is Connor, and today with my co-host Bryce, we have on special guest Dave Abraham, and we talk about how C and how aka BootCon got started, and his work on the C and Swift programming languages.

Bryce 1:01

Well, you know, it's funny, the um the one one of the things that I am best known on the committee for is my um uh my passionate speech um for why we should not standardize the 2D graphics library. And um during that, I said at some point we have to draw a line, uh like meaning like let's draw a line about like what we're gonna include or not. And I did not mean for it to be, you know, uh a cute pun, but everybody in the room just groaned. And people, you know, there I got such praise afterwards for my cleverness that I was like, all right, I just won't tell people.

Conor 1:42

Of course, now now I've ruined it, but only for only for our our listener.

Bryce 1:49

Yes, our listener who who may or may not be a fan. Um so kind of I guess you should do a little bit of an intro for now.

Conor 1:57

So this is this is how I was gonna start this is say that uh Bryce, you've known Dave. Um, I I mean Dave and I have, I think, peripherally seen each other on Twitter. Um, I don't think we've ever actually spoken in a meeting directly to each other. Um, although obviously I I know who Dave is by reputation. Uh so I think we should let Bryce uh do the introduction of Dave. And then the first follow-up question will be Dave, take us back to whatever year you want to take us back to. Um to give us the I'm super looking forward to hearing the history of everything you've been involved in. Because as folks that are familiar with Dave, I won't spoil Bryce's introduction, but you've worked on a ton of cool projects, started conferences, etc. And you've you've you've definitely been around. And um, yeah, so I I I uh I'm super excited. I've I've been looking forward to this to to get to hear the behind the scenes of not just the the the bullet points, but um, yeah. So Bryce, I'll throw it to you and then and then we'll throw it to Dave.

Bryce 2:57

I'm I'm I think I'm I knew Dave from like my earliest days in programming because I got involved in Boost very early on, and that was the time that Dave was you know was still Mr. Boost. Um, and that was you know when when Dave was working at Boost Pro. But I think we first must have met at BoostCon 2011, which was the last BoostCon before it became C now. Um, yeah, because that must have been where I met Dave first, um, and uh you know, all the other C folks, John Cow, um Marshall Clow, you know, everybody else who I've known now for like 10 years. Um and then uh, you know, I guess Dave, how many years after that did you go off um to do Swift?

Dave Abrahams 3:48

Uh I started that in 2013.

Bryce 3:51

Yeah, yeah. So it was true, yeah, because you were at the first C now, and then uh and then you you departed. And I guess I I didn't have the full perspective at the time because you know I wasn't on the C committee. I didn't um you know, I wasn't involved in C11, but you know, Dave was uh you know a large part of that, but then uh uh went off to sorry about that. We talk about lambdas are amazing.

Dave Abrahams 4:19

Went off to build yeah, I didn't have anything to do with lambdas. I I I gave you uh you know the those exception uh static exception specification thingies, the no-throw things, and uh move semantics. Yeah, yeah, move move semantics are big.

Bryce 4:35

Yeah, like I would argue the biggest feature in C Alama.

Dave Abrahams 4:41

Yep. Yeah, what about auto? What about auto? Sorry about all that, but I I made some mistakes.

Bryce 4:47

So so I I guess maybe so I think this is actually an interesting place to start because um I heard a story about the origin of Boost um at some other C event recently, and I'm not sure if it's historically accurate. Um it was in particular, it was regarding the origin of the name of Boost.

Dave Abrahams 5:11

Yeah, well, so you know I wasn't there. I mean, I've only heard this story secondhand, also.

Bryce 5:17

Well, you you weren't you weren't there?

Dave Abrahams 5:19

I thought that I wasn't there when when Robert Clarer and Beaman Dawes had this conversation.

Bryce 5:25

Oh, I for some reason I thought that you were one of the folks that was there that that was one of the initial founders.

Dave Abrahams 5:32

Well, I mean, I you know was involved immediately after that. So but this this first conversation that they were having, uh yeah, I wasn't at that.

Bryce 5:45

So so when when was that?

Dave Abrahams 5:47

Uh I don't remember, frankly. Let's see. Uh it must have been uh just after 98 because we standardized C. And the reason that that boost came about was that Beaman, who was the chair of the library working group, um was thinking about where our news our new libraries were going to come from. You know, we had just standardized C with this amazing thing called the STL, which it really was uh a groundbreaking piece of work, but it was also not the way we were supposed to do things. We were supposed to be standardizing existing practice stuff that had, you know, actually been in wide use uh beforehand, and the STL had had at best marginal use at that point. And in fact, they you know, based on what Alex Stepanov said he needed, there were a bunch of features added to C that no compiler had even implemented yet when we standardized C. So um, yeah, so we didn't want to repeat that, and and so Beaman's thinking was let's let's get some libraries out there that that you know are high quality, standardizable, um, and uh go into wide use, and you know, they're basically try to lower the barriers to entry as much as possible so that a lot of people use them. And I think we succeeded in that. Boost seems to be everywhere.

Bryce 7:33

Yeah. Well, and not only is Boost everywhere, but um uh a large chunk of the C standard library editions in C11, 14, 17 all came from Boost libraries. Um although it's it's interesting, you know, the point that you make that that standardizing the STL was not really what the committee was supposed to do. Um, because you know, it was successful, it was a risk, but it was successful. And um I I think that the committee frequently um uh oscillates between being far too conservative and being far too bold in what we do.

Dave Abrahams 8:20

That uh yeah, I mean I I think like one of the one of the frustrations I've had with the C committee is the is a kind of uh unwillingness to form a consensus about about how the committee would operate and you know how bold to be, what you know, how important is breaking code, how important is performance, any of these kinds of decisions, you know, to set a framework for future decision making. Um no one has ever wanted to do that. And and that's I think a a major a major problem with the committee. But in fairness, uh with respect to being too conservative, you know, the STL, in my opinion, is uh uh a singular work of genius. I mean uh you know it has in you know, with how many years has it been since it came out? 20 years? Uh with let's see, 98 to 2021, 30 years. Okay. With 30 years of hindsight, right? It's definitely, you know, I would have designed some things differently. It's got some ugly corners, whatever. But uh, you know, those things worth gambling on like that come along once in a generation, maybe.

Bryce 9:57

Yeah, yeah. I certainly think that's true. What what what do you think it was about the STL that made it so so novel? Or or what what what is yeah, what what what what made it so novel or successful or so so genius?

Dave Abrahams 10:16

Well, I mean, you know, Alex Stepanov, uh, you know, uh you don't need to say much more than that if you've if you've heard Alex talk and and really know about what he what he knows about and what he thinks about. I mean, aside from being an amazing polymath who you know can read things in the original ancient Greek. Um and uh his I think the the fundamental thing that that he brings brought to the table that I've you know carried with me ever since I I uh met him basically is a sense of searching for some underlying truth. Um rather than rather than designing something. You know, Alex the the genius of the STL was that Alex didn't say, you know, I'm gonna come up with a beautiful design that's that accomplishes these things. Look at what I've done. He says, look at what's actually there in in the world. Let's find let's generalize from that and let's let's find the the actual abstractions that are there. And that that's a kind of humility that he brought to design that I think is you know really crucial.

Bryce 11:56

Yeah. And so uh so as we said earlier, in in 2013, you um you went to join Apple and you got started on this, you know, this new project, Swift. Um, and so what uh what was that like? You know, you there aren't many people who can say that they were at the um, you know, the the real the start of a language that has grown as quickly to maturity as Swift has. Um and uh yeah, so so maybe so maybe tell us a little bit about your experience with uh with Swift.

Dave Abrahams 12:34

Yeah, well, I mean it was a it was a uh once-in-a-lifetime opportunity for somebody who's interested in language design, you know, because uh languages do come along every week and they die in obscurity. So uh, you know, uh so if you've been around a few years and and seen this, you only want to get engaged in a uh in a language project if you think it's really got some legs. And uh, you know, this one had uh Apple's future behind it. Apple was staking its future on it, although to what extent wasn't entirely clear at the in the early days. Um uh and it had an amazing group of uh people uh that I had faith in, you know, starting with Doug Greger, who was my close friend. Um and uh yeah, so what was it like? It was uh it was a thrilling adventure. Um you know, um there it was you know, it was scrappy. Uh we had a a group of about 10 to 12, I guess. Uh that maybe wasn't even the the starting size might have been might have been like eight. Um but uh you know we it Chris uh Lattner to his credit uh ran the ran the thing very much by consensus as much as possible. Um and uh and while you know some people denigrate design by committee, I have always found that the the process of coming to consensus uh inside of the C committee, for example, has been more beneficial than than harmful. Um and so we had a lot of fights about things. Um uh we all learned a lot from each other. There was an amazing amount of uh of different experiences in the group. Um yeah, and and it was it was a thrill. But when I got there, the the language was uh extremely immature. It was almost impossible to build anything generic um with it. And and uh yeah, so so the first the first few months I would say were quite frustrating because of that. It was it made it really hard to make progress. And you know, it was good to be part of the design discussions and and move the design forward, so that was great, but like I was also responsible for the standard library, and how do you build a standard library on a fragile foundation that barely works? Yeah, um uh and uh at some point uh you know the the pressure of not delivering anything became too much for me. And I and I decided, okay, I'm just gonna do what I what I have always done with broken C ⁇ compilers, which is figure out how to you know coerce them into doing what I need. And um who cares if it's if it's beautiful or or even correct, you know. Um, so that got me over a hump when I was first able to build an array. That was that was something.

Bryce 16:30

Yeah, I mean I I guess I guess that it really is early days of building an array is uh you know is is a challenge, but uh and and and you were involved a bit in the design of C of Swift's um generic system, right?

Dave Abrahams 16:46

A bit. Um uh yeah, definitely. Yeah I I had some influences, especially later on. I would say that the a lot of the inspiration for the initial uh design of the generic system came from work that Jeremy Seek had done on something called system G, I think. Um and uh basically what he was doing as far as I understand it was building a generic system with separate compilation. Um and in the in the sense that Haskell does it, except with associated types, which at least standard Haskell doesn't have. Um and uh and I think Doug took that as a foundation. And part of the reason that was so important at Apple was um or for Swift was that Swift needed to uh needed to not lose the one really killer quality that Objective C had for Apple, which is this ability to upgrade either side of an interface without breaking code. So and without recompiling.

Bryce 18:08

A challenge that continues to plague the C ⁇ ecosystem and the evolution of C.

Dave Abrahams 18:15

Yep, yeah. So that that's what they called uh uh resilience or ABI resilience. Um and uh and so to do that, you have to be able to separately compile generics into something that doesn't change when you when you invoke them with different you know uh parameter types. Yeah.

Bryce 18:39

Yeah, and I mean I I think I think this, you know, the this part of Swift is a really good example of, you know, we we have we have some talk in C about like the ABI problem. And some people are like, well, we should just go and you know break ABI or you know, not break ABI. And it's almost the the conversation is usually one that makes it seem like it's simply a policy question. Um, to me, it's never it's always seemed like there's been a missing discussion about the technical uh limitations that we have. That you know, we don't really, it's not something that was the language was designed for from the ground up. It's not something that standard library implementations were designed for from the ground up. Um and who knows, maybe, maybe it's not too late to bring some sort of resiliency to C. Um, but uh I think it's much easier, of course, if you design it from the start of your language. Um, you know, one of the problems that would come from almost any form of resiliency that we'd add to C is would it have some cost over doing things the non-resilient ways? And if it did have some cost, then how would you get people to to buy into using it? And uh one one of the problems, of course, is that some people who are making the choices that will that will impact whether or not you can change your ABI, they may not realize when they're writing their code that they needed resiliency.

Dave Abrahams 20:09

Uh yeah. Um so all of these things are are real problems for for Swift too. I mean, uh uh ABI resilience at generic boundaries is is a definite performance cost because what it means is that you end up with some kind of dynamic dispatch, even though it's static polymorphism, you end up with functionally like like um implementation in the implementation, a dynamic dispatch that that chooses one implementation or another based on the on the input types. Now, one thing you can do to to um mitigate some of that is to is to have a JIT, right? Uh or a or some kind of late recompilation that that erases the cost at the resilience boundaries. But regardless, you you have to think of it every resilience boundary with polymorphism behind it, even static polymorphism, as a virtual function call. And and you know, one of the things that that generic programming depends on being able to hide very small operations like the increment of an integer or a pointer behind an abstraction boundary, the you know, whatever the uh the concept requirement is in C terms. Um and so you know, when you when you pay the cost of a dynamic dispatch for tiny operations, your code gets a lot slower. So there are all of these uh these halfway points that you can get into in. with Swift uh with regard to things like saying that this thing is inlinable. If you say it's inlinable, then the resilience boundary goes away. But then again you can't change it. Right?

Bryce 22:13

Yeah, yeah.

Dave Abrahams 22:14

And and I think yeah I think the whole question of library evolution is something uh Swift hasn't yet really uh addressed. Um it's addressed the problem of how do you build an ABI so that you can extend a struct without breaking code. But what are like there's a whole there's a whole universe I think of tools and and processes that that library developers need to have in order to be able to to effectively evolve their libraries which I don't think I don't think Swift has done anything to actually dig into yet. They've had a lot of other stuff on their plate.

Bryce 22:59

Yeah.

Conor 23:00

I was going to say is there a language that you think is the good archetypal model for those kinds of tools or oh no uh I don't I don't think I don't as far as I know nobody has really tried to figure that out.

Dave Abrahams 23:14

And you know I think I think starting with with the ability to create ABI resilience is um is important. So I think Swift is in a unique position at least of of having that as a platform. But uh yeah um I think there's a lot of work left to do.

Bryce 23:37

I mean yeah a lot of a lot of languages including a lot of new languages you know struggle with this. I know that right now in the Rust world there's a lot of conversations going on about ABI um uh resiliency and um yeah um I'm gonna badly summarize my understanding which is that for a long time the model in Rust had essentially been we we're not you know we don't provide or we don't worry about ABI stability and now there's maybe some question that or there's some there's some folks who might uh uh want that and they're trying to sort out how exactly would they do that especially in a world where they sort of you know they sort of statically build everything um yeah yeah well one interesting answer for for performance oriented languages like C and Rust is is tell people how to build apis that can be extended and give them some guarantees but but don't uh don't make that the the default right right so there are things that we do in C all of the time uh which we get from C you know you build a C interface you you dynamically allocate your struct then you can extend the struct anytime you want those kinds of techniques actually work in practice to some extent they parts of them may be guaranteed I don't know yeah but yeah like a like a good example of this in C, which is um would be like std mutex. So on um on MSVC their std mutex implementation is like it it's eight it's 10 times larger than their std shared mutex and it's like two times slower and the reason is because their std mutex has to be be ABI compatible back to like the Windows XP mutex layout which is like this huge thing that uses this very old API. And um you know fortunately no one should ever use a mutex so yeah but but you know it it would have been a fairly simple thing to avoid this problem if they had just instead of having the the the um you know the the metadata for that mutex be uh just regular data members of std mutex they could have just put it in some opaque struct and just had a pointer to it. It's just that when they did their implementation they didn't um they didn't make that decision and there was you know there was a trade off to that at another level of indirection. But uh I think it's something that we perhaps should have designed for more consciously when we uh when we first started standardizing C. Yeah yep uh well I mean that's that's up to each individual implementation right yeah um were you were you around when the um when the infamous std string ABI break um came through the committee uh I I'm not familiar with this story so uh at least not in those terms this this was um so in C11 we we changed some of the complexity guarantees on std string in such a way that it it prohibited the GCC copy on write implementation from conforming to what std string was oh yeah I think I remember I I think I remember lots of complaints about the about the at that time I think I remember people saying that you couldn't actually make a copy on write implementation conform. Right right it what what what's interesting to me is that uh everybody who I've ever talked to from that era doesn't remember it as being a a controversial thing at the time that the requirements were changed. It was only afterwards that we realized how significant uh of a change it was and the impacts that it had on the GCC implementation. And uh you know there there's there's some folks that including myself that think that it's it ended up slowing down adoption of C plus 11 because uh if you upgraded to C plus plus 11 to the you know the compliant C plus plus 11 std string uh it would you know it would be an ABI break and std string gets used everywhere. And I think it's just you know it it's uh we would never today in the committee we would never make such a change because we've become super sensitive to ABI changes to the degree that almost any proposal that would in some way shape or form break ABI is is a non-starter. But you know it wasn't always that way um we just didn't know we didn't realize what the costs were at the time.

Dave Abrahams 28:50

Well at least I mean I to to your credit now I guess at least there's some consensus about about what's important. All right yeah I mean uh I think at the time that ABI break happened it was it was fairly random we would break we were willing to break ABI over here not over there for what reasons you know there was there was no uh uh sense of what the scale of uh values was um so but I I do think that that uh you know an unwillingness to break the uh the ABI of C ⁇ is is a probably a serious limitation on it being you know evolving into a better language.

Bryce 29:42

Yeah I mean I I think it does still remain sort of an existential um the existential struggle in question of the C library evolution um because uh as long as we live in a world where we cannot fix our mistakes after we ship them that that will make the committee very very cautious about what things we ship. And so I think that the standard library will roughly continue being the same sky size and scope as it is today. Because any sort of massive expansion would bring you know a degree of risk that would just be unacceptable. And so it's like we we we all understand the rules or not rules we all understand the the current you know policies and consensus um and like the implications of that today um but I think just because we we understand that it doesn't necessarily mean that the outcomes are the best.

Dave Abrahams 30:49

Well why can't you just uh standardize library components that are are designed with resilience in mind.

Bryce 30:56

Right. And I and I think that's something that we need to um uh be doing uh but uh it's it's not an area that we've really treaded into and I think that oftentimes there's the performance trade-off and always yeah but we not often always but but you have to if you want the resiliency you have to be willing to make that trade off like you know one example um there's been a lot of talk recently in the C about standardizing um networking and uh one question that always comes up is if we standardize some sort of secure networks in some sort of TLS layer well of course that would need to be standardized in a way where we would be able to roll out you know security fixes um and deprecate old you know protocols, etc. And I mean I think we could do it. I think that if we sat down and if we said from the start, hey, this library component needs to be something that's going to be resilient to change, we could design it in that way. There might be some some performance trade-offs but it could be done. And I think that oftentimes the conversation the C committee has just been well this is an ABI break so we can't do it and we maybe need to have more of the conversation of you know how could we build you know this new component to be resilient.

Conor 32:16

Makes sense. Thanks for listening. We hope you enjoyed and have a great day.