Bryce

I was making myself, I was making myself this very fine hot chocolate.

Sean Parent

My first uh MAC hack was uh probably 88. Uh so I was working at Orange Micro.

Bryce

There were some technical difficulties with the hot chocolate maker, and I had to debug it.

Conor

Guess what? I got a fever. And the only prescription is Sean Parent. Welcome to ADSP the podcast, episode 17, recorded on March 18th, 2021. My name is Connor, and today with my co-host Bryce, we are super excited to bring you a conversation with our first guest, Sean Perrett. How is it going, ABSP listeners? It is just Connor recording a introduction/slash prologue. Before we hop into the conversation with Sean, Sean Perrent, as many of you know, is a senior principal scientist working for Adobe. He's been at Adobe since 1993, with the exception of a year in 2009-2010, where he worked for Google, which is where the infamous That's a Rotate story comes from. Before Adobe, he worked at Apple for roughly five years, and before Apple, he started his career at a company called Orange Micro. That's all I'll say about Sean's work history, as I'm sure you'll hear a lot more about it from him, as he will be a recurring guest on ADSP the podcast. The one thing I do want to add though is that if you haven't seen any of Sean's talks, I can't recommend them highly enough. My two favorites are C seasoning from 2013 and generic programming from 2018. Listening to Sean talk is inspiring. And a part of the reason that I gave my first conference talk in 2019, Algorithm and Tuition, is because of watching Sean Perrin's C seasoning from 2013. So at this point, I'm going to splice in a quote from his generic programming talk in 2018 because I think it's absolutely fantastic. And after that, we will hop into our conversation. Super excited to bring you this conversation with Sean Perrin.

Sean Parent

Right. And Alex has this beautiful little solution in there for stable sort that builds it up out of all the other algorithms. All the algorithms within the STL fit together like puzzle pieces. Okay, right? I mean, keep in mind here, I'm working with a bunch of phenomenal engineers, right? Great people. These are some of the top people in the industry. And I've never seen somebody who writes code like this. Okay? This is not like the code that Alex published in his paper five years before. He's clearly grown and had some time to think about it. So it's just this phenomenal piece of work. And for me, it was like I want to write code like that. How do I write code like that?

Conor

Alright, we are recording our first episode with a guest. And if you've been listening up till now, you probably have a good idea who it is, the man, the myth, the legend, Sean Parent. How's it going?

Sean Parent

Hey, it's going pretty good. So yeah, I've been actually listening to your podcast. I think I'm current, so I've listened to all of them.

Conor

It was probably the highlight of my year so far when you tweeted out around Christmas the photo of you with a glass of eggnog with the caption that was a lot of APL. So thank you for that.

Sean Parent

Yeah, you're you're an easy man to please, I guess.

Bryce

Sean, do you do you remember your do you remember the keynote you gave at C ⁇ now like 2014? It was a it was a you gave a talk where you were talking about um getting I think it was like getting Photoshop or something like of that nature running on an iPad. And I what I remember about the talk is there's one part where you're talking about um you know what what's what's the compute that you have available on on the on the iPad. And like a small part of it is the serial processing power, and then there's a larger part of it that's the you know parallel processing power, and then you have the GPU. And yeah, it was just it I think it's I think it's my favorite of your talks. But anyways, I've been thinking about it a lot lately because I now obviously work at NVIDIA and work a lot on parallel computing. And uh I just remember net like like in hindsight, thinking that talk was really prescient.

Sean Parent

Yeah, it's uh I use that slide. That slide's actually from from Russell Williams, and it's it's not specific to the iPad, it's actually uh an Intel box that a few years ago he kind of broke down the numbers on. Uh but yeah, if you look at where the compute horsepower is in in a modern machine, if you're just writing single-threaded code on the you know, on the on the CPU um uh with no SIMD, uh you're using uh 0.25% of the complete compute capability of the machine. And uh you know, most of the MIPS are trapped inside the the GPU and increasingly now uh split between GPU and ML units.

Bryce

Yeah. Yeah, one one of these days I'm gonna one of these days somebody at the video is gonna teach me something about this machine learning stuff.

Sean Parent

Yeah. Yeah, definitely worth uh worth looking at. Uh I mean but the uh but uh uh but you know my short summary of of kind of modern machine learning is is if you think about um you know what we write on machines are just functions. And and we typically come up with some equation for the function. But any function can be approximated. And so if you have something where you don't know what the function is, but you can feed enough material to to uh a machine learning system, it can learn to to derive an approximate function. And and uh uh so you know that's kind of all all machine learning is, but uh but it does it does change things. If you look at in image processing, uh uh say take Lightroom, the the Lightroom Auto button, which you know kind of auto-adjusts all the sliders. Uh that used to be a very carefully hand-tuned algorithm and trying to look at, well, you know, statistically, how would we want to balance highlights and shadows, and what about this kind of image, and what about that kind of image? And uh the approach now is to just uh get a bunch of experts and have them correct a whole bunch of images and do a really good job and have the machine learn what they do. And that's the auto button these days.

Bryce

So so you've worked on a you've worked on Photoshop. What are all the things that you've worked on in Adobe? You've worked on Photoshop, Premiere.

Sean Parent

Uh I have not worked on Premiere. So at Adobe, um uh uh I started at Adobe working on Photoshop, and then we launched uh an internet division. This was way back when Netscape was really hot and and everybody needed an internet division. Um uh uh and so I I ended up working on uh a shared component. About the only surviving piece of that is something called XMP, which is a standard for metadata and how metadata is embedded in documents, so that you can you can pull it out. Um from there I did a stint in Adobe's ATG, which is their advanced technology group or was at the time, uh, a research team. And I was doing that's where I started research on something called property models, which I still is still something that I continue to work on in my what little spare time I have. Um I was in ATG is when Adobe hired and then fired Alex Stepanov. And that led to me uh and then hired him back, and the hiring him back led to me managing him, and together we formed Adobe Software Technology Lab, which spun out of our research team. And so I ran Adobe Software Technology Lab for eight years, and uh, you know, we wrote wrote the book Elements of Programming, we created Adobe Source Libraries, we did a whole bunch of stuff internally as part of that effort. And uh uh when ST Lab fell apart, uh I went to Google for a year, worked on Chrome OS, came back to Adobe to work on Revel, which was a consumer version of Lightroom for uh we shipped it for iPhone, iPad, and desktop. And it was a precursor for Lightroom Mobile. It was running the same imaging engine that Lightroom runs. And so I pivoted from working on Revel uh just before Revel was canceled uh to start Lightroom Mobile, and so I did Lightroom Mobile and then uh got the imaging engine going inside the browser, so that was Lightroom Web. And then after doing that, uh a guy named Dave Howe walked into my office one day and said, Could we just put Photoshop on the iPad? And I said, I think I know how I would do that. And that launched the effort to put Photoshop on iPad, so uh did f did Photoshop Mobile and and but Photoshop Mobile is somewhat of a larger effort if you can probably read the tea leaves if you look at at uh what people were hiring these days. Uh but part of that was taking uh uh the Photoshop code base and turning it into a library uh that contains kind of all the interactive logic. So so you can't just do batch processing with it, but you can attach a UI to this library and and put Photoshop in other places. And so recently I've been doing some work putting Photoshop in other places, which haven't shipped yet. And uh and right now I'm putting together a white paper and maybe looking to hire somebody and uh uh get back in in the research side of things a little bit. I've got an idea for uh a potential product that's a a little bit long lead and is gonna gonna require a little research to feed into it.

Bryce

Spicy. So what when you um when you f when did you first meet Alex? Did you know Stepanov before uh you became his manager?

Sean Parent

Yes, yeah. So um uh uh uh when I was working in Adobe's ATG group, one of my colleagues there uh was uh an individual, Larry Macenter. And Larry had worked with Alex at Bell Labs years ago and was friends with Alex. And Larry invited Alex to give a talk at Adobe. And at that time, Alex was uh vice president at Compaq uh computers, if you remember who they were. They made kind of you know portable PC devices, and Compaq was getting bought out by HP. And so Alex had been in HP Research before, but now he's a VP at Compaq, going through a transition to Hewlett-Packard. And so Alex came in and gave a talk on STL, and afterwards I went up to him and told him, Hey, I enjoyed the talk, and uh a fan of your work on STL, and it was a you know wonderful talk, and he asked what I did, and we started chatting about at the time I was researching property models, and he said, you know, that's actually interesting. So we ended up going up to my office and talking for like four hours, and I gave him a demo of property models, and we just chatted. And at some point in that conversation he said uh he said he said, you know, I've I've really missed being an engineer and doing engineering. And I said, Well, why don't you go back to it? Like, what's stopping you? And he just kind of laughed and shrugged it off, and you know, we wrapped up our talk and he went away, and about two weeks later he gave me a call and said, you know, I gave a lot of thought to what you were saying, and I would like to go back and be an engineer. And is there a position at Adobe? And I tried to get him a position in Adobe's research organization, uh, but failed. In fact, the exact quote I got from our then CTO was uh we don't need another C person like you. Uh to put this in context a little bit, at the time uh there was this belief that Adobe would at some point uh rewrite all of their applications either in in Java or in Flash uh um uh as safer languages. So uh I never thought that that would ever happen, and certainly I've been proven correct tonight. Um but uh I did manage to find Alex a position in uh uh uh one of our share technology groups, and unfortunately it was a really bad mismatch. Um uh uh this was kind of a uh uh uh generally a uh uh a mediocre team. There were some strong individuals on on the on the team uh uh working on a legacy component that just needed a bunch of bug fixes and maintenance. And Alex came in and was of the opinion that you know basically all of this is wrong and needs to be rewritten from the ground up, and and the engineers on this team don't have a strong theoretical education, so I need to start by teaching the engineers how to code, and then we'll start rewriting this product. Uh that went over about as well with management as you would expect. And and we tried hard. Uh we tried to make the classes very relevant, and you know, I helped Alex with his classes, and Alex was getting increasingly frustrated, and uh and at one point I told Alex, I said, you know, if it gets to the point where you're going to quit, please call me and and we'll see if there's something else that that uh you know, see if we can make something else happen. And and and he said, no, no, no. He thought that this project was important and he was willing to stick it out, and there was no way he was gonna quit. And about two weeks later he called me up and he goes, guess what? And I said, they f they I said I said you quit. And he's like, nope, they fired me. And so so I went to my my then boss, who was uh uh Greg Gilly, and uh a lot of respect for Greg, and said, you know, Alex uh was fired, and you need to to make that not happen and hire him back into your organization. And Greg said, I can't do that politically because he was, you know, fired in a different organization, and it makes the management chain in that organization look bad if I if I then just turn around and hire him back. And and I said, I said, well, at least do me a favor and take Alex out to lunch. So Greg took Alex out to lunch and came back and called me into his office and said, damn you, I'm going to hire him back. Um but here's the deal, you're going to manage him.

Bryce

So had had you uh had you managed people before Alex?

Sean Parent

Uh uh so yeah, I had been uh uh uh engineering manager on Photoshop for uh for a while fairly early on during uh Photoshop 5 timeframe. And uh uh when uh I had done the stint uh with uh when we launched the internet division, uh I was managing a shared technology team in that in in you know in that organization. So I had done some amount of management.

Bryce

And then But I'm I'm gonna bet that Alex was a bit different.

Sean Parent

You know, yes, Alex was a bit different. Uh the my wife will sometimes refer to me as Peter Payon because I have a habit of of uh collecting the lost boys. Um uh uh you know, when when I was running software technology lab, myself included, but almost everybody on the team, except for Alex, were kind of refugees from our our advanced technology group and and so kind of you know misfit individuals. Um even when I was engineering manager on Photoshop my management philosophy has always been that that you you know you kind of want to align your goals with where the company is going, but you want people to be working on what interests them. And and sometimes that means not taking a straight line path to to what the corporate goals are. You know, it means you'll you'll run a bit on a tangent. But if you have your people working on things that interest them, you'll end up going further faster uh than if you just force people to uh to stay on target and and walk a straight line. And uh uh so uh so y you know, I'm a relatively hands-off manager when it comes to to the individuals. I try to just quit, you know, make sure that I'm communicating with them well and and know what they're doing and know what they're working on. And you know, largely the role then of of for me of being a manager is is conveying that information up, right, right, making sure that that management is aware of the value of the team and and uh uh uh that things are going well in that regard.

Bryce

I I just realized we should probably explain to our audience who Alex Stepanov is, because some of our listeners are not gonna be C people and and won't know. Right.

Sean Parent

So who is Alex Stepanov? Right, so so so Alex Stepanov uh uh best known as the uh uh creator of the the STL, the standard template library, which is part of the C standard library, uh which was was very unique and revolutionary at the time that that it became part of the C standard. It didn't look like a like any other uh uh library in that time frame. And uh part of that was uh was inventing the the notion of concepts, which are now uh formal part of the language in uh in C20. So Alex wrote the the early papers and coined the term concept. Um Alex's history is he's you know quite uh uh uh a brilliant individual. He was born and and raised in in Moscow. Um uh was uh PhD equivalent and university professor at the age of fourteen uh was kind of identified early on as uh as a math prodigy and uh uh was uh kicked out of Russia in the in the 70s and found his way to the US. Um uh worked mostly in research organizations, in uh GE labs, uh uh Bell Labs, HP labs, um uh uh uh doing doing research in s in software development. And it was you know, his uh stint at uh uh Bell Labs that got him interested in in C and C. Languages, and then he developed the the standard template library at HP Labs.

Conor

So I didn't know Stepanov was kicked out of Russia. Have you not heard that before? Are you allowed to elaborate or shine some light on what happened there, or is that something we don't talk about uh anymore?

Sean Parent

You know, I've I've gotten the the story from Alex, so so apologies if I don't don't get all the details uh correct correct here. But his his parents were um uh or his father was a journalist and and that gave him uh uh uh access to uh uh uh foreign diplomats who were were were visiting Moscow. And he uh uh uh would uh smuggle information to the the the foreign diplomats and was arrested twice uh by the KGB. Um uh the first time he was sentenced to military service um uh uh and uh the second time they uh uh uh uh it was uh uh uh more of punitive measures. And the third time he was caught they uh escorted him to the border, stripped him of his papers, uh he and his wife and said goodbye. So yes, his uh he was uh uh I believe it was Austria. He was stuck in Austria for some amount of time, and he had friends in Sweden, so he was trying to uh uh uh get permission to leave to Sweden and just because of the bureaucracy couldn't make that happen, and one of his contacts said, you know, you need to contact the US, and apparently the US was like a Russian mathematician. You want to come to the US? Yep, we'll make that happen. So this was in the in the middle of the Cold War, and so that's how he ended up in the US.

Bryce

So Connor's dad is also a journalist, and apparently a fairly well-known journalist in Canada. That is true, yes. Didn't you once tell me, Connor, that you used to have people who would like look at your last name and they'd like pale their faces would pale and they'd be like, Are you related to whatever your dad's name is?

Conor

Yeah, this is a super random anecdote that probably our listeners uh won't care about. Um one time at a party um when I was in university, yeah, I'm I met one of my my ex's friends, and they said, Oh, Hookstra, that's uh you're not related to Gord, which is my dad's name, uh, are you? And I said, Oh yeah, that's my dad. And then she was just like, Can can you get me a meeting with him? And I was like, uh, for what? And she's like, I work for Fortis, which is like an oil company, and they were like, it would be oh my goodness, like the people upstairs, like there's flags, red flags that get raised whenever you know he sends out a story that has anything to do with us.

Bryce

And anyway, so uh it's he was an investigative reporter, so if you were doing anything sketchy in Canada, you'd be afraid of the sky.

Conor

Yeah, yeah. So uh I just think of him as like my dad. Um, but don't like people that work at oil companies and uh politicians, um they're they're always there's a bit of concern, I think, when uh when and I'm glad that there's probably like zero overlap between everyone that I'm talking about right now and the people that listen to this podcast. But anyways, um yes.

Bryce

Well don't worry, I think it's pretty unlikely that he's gonna get kicked out of Canada first.

Conor

But that is crazy that um I did not and that's it's so weird to think that uh Stepanov, like we all, or at least I we we think of him as as sort of like the father of the you know standard template library, um, and you know, worked at Adobe for a number of years, that at one point he was being sort of chased down by the KGB, and then at some point they said, okay, bye-bye.

Bryce

Uh I really kind of think of him as not not only the father of the standard template library, but also also sort of the father of a lot of what we consider to be you know modern C library design. Um you know, a lot of a lot of the um a lot of the fundamentals that we take for granted about how you should build a C ⁇ library come from work that Alex either did or Alex inspired.

Sean Parent

Yeah, I think that's that's definitely true. Um I think unfortunately, you know, I'm gonna say this because I'm I'm working on a standards proposal right now. I just finished a couple pages for for uh John Lakos's uh uh uh uh upcoming book. Um but but I'm I'm working on a standard proposal to try to fix the postconditions of a move from object and the standard, which I think are badly broken. Um but in kind of digging through why the postconditions for a move from object are badly broken, it uh you know, I realized that uh one of the key ideas from STL uh and concepts, which was about the domain of an operation, has been all but stripped from from STL. And the requirements in the standard right now, even though we still have the requirements tables where we define things like uh you know what what a quality comparable is or what less than comparable is. And now they're called things like you know CPP17 a quality comparable. We have all the tables there as as STL requirements, but they're no longer actually used as requirements. So so if you look, you know, if you if you just search the the standard for the CPP 17 a quality comparable requirement, it's never used as a as a requirement for any of the algorithms or containers. Stood find doesn't require that the value type is a quality comparable uh uh uh anymore, where it did in the old old STL documentation. Uh instead, there's just a couple of types within the standard library that satisfy the requirements, that you know, they guarantee they satisfy the requirements of of uh uh of a quality comparable, as my example here. Uh but that's you know the what value does that have if nothing actually has the requirement? Right? I have a type that satisfies these requirements that nothing actually requires. So so so I think over time the semantics of the standard library have kind of been stripped away. And what we're left with is is works as coded. Right? When I write std find, it's defined as as it applies equal equal to each of the elements in the sequence until equal equals returns true. Um uh uh and at that point equal equal is just a predicate. It's not it's not uh uh uh uh uh actually searching for for an equal element.

Bryce

Yeah, this has been um this is actually a problem I've I've become intimately familiar with um because I did a lot of work um on the algorithms when we were adding the parallel versions of the algorithms, we came to realize that you know a lot of the specification of the requirements for some of the algorithms, notably the numeric ones, but some of the others as well, were sort of under-specified or didn't really capture all of what the requirements were. And and in many cases they didn't uh you know, the we we didn't have concepts in the language when we first introduced them, and we didn't get them in C ⁇ 11. Um and so because of that, the the current corpus of algorithms, the ones that have been there since, you know, since the start, are not specified in terms of uh in terms of algorithm in terms of concepts, um, which is uh which is unfortunate. It's a Eric Eric Niebler likes to likes to say that you know concepts should should come from algorithms, right? That you you write the concepts um uh that a particular algorithm needs. But uh unfortunately that's just not how the that's not how the specification is structured today, although we've gone a long way till we're fixing that with ranges.

Sean Parent

Yeah, I you know, I think it things are improved in ranges, although we don't with in in C plus plus twenty anyways, we don't have much in the terms of algorithms with ranges. I haven't looked at at what's going on with C plus plus twenty-three, you know, where we where we might get more. Uh but yes, it is it is very unfortunate. And it's been uh uh you know, kind of this this this uh I think thing that has happened over time, this this weakening of the concepts because people want to say, well, look, if I if I call find and find is just this loop and I call it with my my equal equal, it does what I want. It may not do, you know, it may not be completely legal, but but what I want happens to have the same structure. And and you know, the way things are defined right now, it's perfectly legitimate to call std find, say, say, where the value you're looking for is a NAN. And that won't actually find anything, but it's completely legal as far as the standard's concerned, uh, because you can call operator equal on all those on all the things. And you so even if you're looking for a NAN in the sequence of NAMs, you can call it, it won't find it. Where in in say the HP or the SGI STL, the early versions of STL, and even in the early versions of the standard, uh uh uh uh that would have been undefined behavior because you were violating the preconditions for for find. And over time the preconditions have been weakened, and along with that the semantics of the operation have been weakened.

Bryce

So I think a part of it is also um we often at the surface we think about um concept requirements as being syntactic, but there's also semantic requirements too, and it might you know it's often difficult or impossible for us to enforce the semantic requirements, but it does not make them any less important.

Sean Parent

Yeah, yeah, and that's the point. It doesn't make them any less important, and the whole idea, you know, with concepts was was was they've never been particularly checkable because for them for the most part, you know, they they rely on on existential qualifiers, right? You know, for all or for any. Um, I'm sorry, for all or there exists, right, right, right, right, some element that that satisfies this. And so there's there's no way short of doing a proof that you can you can validate that that's met. So the idea with concepts was to say was to say we can still have the strict semantics, and what we do is we just associate those semantics with a name. We say if you use this name, then you have to guarantee that you satisfy these semantics. Um and if you do satisfy those semantics, then the operation works as specified. Um but yes, over time those semantics have been have been stripped away. So, like I said, you know, stood find no longer requires a quality. It only requires that operator equal equal be a binary predicate and be callable for all the items within the range.

Bryce

Yeah, and and I I think there there are some really good examples that demonstrate that sort of that property of that you know, a lot of these concepts, a lot of the concepts or things that you want to express with concepts involve these, you know, existential, you know, for all or for any that are really hard to quantify. Like the best simple example of this I think of is if you want to say I want an invocable that can take a single parameter that is an integral. Um, so not a particular instance of an integral type, but something that satisfies the integral concept. There's not really a way to express that in the concepts that we have today. Instead, what you can say is you can say, I want something that satisfies invocable with this particular integral type. But that problem of uh of sort of uh I think uh often what what in the C OX model it was called archetypes. It's very hard to check for that, to check, hey, does does this is this thing satisfy the invocable constraints for anything that satisfies this other set of constraints.

Sean Parent

Yes. Yes. And uh yeah, and and where this you know the the the damage that I think this has done with with move is in the old uh you know, HP and and and SGI STL. For people who don't know, so the the STL was originally developed in HP Labs, and so the the very first prototype of of STL was done at HP Labs, and then Alex moved to SGI and and so one of the early popular uh standard conformant implementations of the STL uh came out of of SGI. So you can still find the docs for that online. Uh uh somebody scraped them from the SGI website when SGI uh uh took them down. Um if you if you look at the at the old docs and even the the old versions of the standard, there was this notion of the domain of an operation. And uh and basically what that said is that the concept, you know, say a concept of a quality comparable, um uh only has to hold for the values that are reachable through the arguments to the function. So if you're talking about um uh uh uh std find, um uh uh a quality didn't have to be defined as as total for all possible values of the arguments past, only for the values that were reachable within the range of uh uh uh uh uh where you where you were finding elements. And that's how how Alex kind of dealt with nans and double precision. So you know uh less than is required to be a strict weak ordering. Uh uh less than on floating point types is not a strict weak ordering if you consider nan. And so so Alex just took nan out of the domain of the operation, right? Right, and being outside the domain of the operation doesn't mean that it has undefined behavior for a specific type. It just means that in the use of the algorithms, it's uh it it no longer satisfies the the precondition for for for the algorithm. And so that's how how things like NAN got dealt with. Um in the case of move, because now the domain of operation still appears in the standard, but it only does for operator equal equal on input iterators, which is like really weird. It's been like stripped every place else, and now there's just one section uh uh where we talk about domain of operation for equal equal for input iterators. And um uh the reason why why that affects move is basically what happens is when I I I move one object from one location to another ob location, the value that's been moved from is only required to be in an unspecified state. And that unspecified state uh uh uh uh has to be removed from the domain for most of the other operations. Um uh uh, you know, and basically my proposed wording is to say, you know, it's it's it's removed from the domain of all operations except for uh uh uh uh being used in the left hand side of assignment and instruction. Right. Right. And uh uh but it's it's it's a challenge because uh because domain of operation isn't used anymore in the standard. So That's where we'll end part one. Tune in next week for part two.

Conor

Have a great day.