Hey, it's Jason, host of the Code with Jason podcast. You're a developer. You like to listen to podcasts. You're listening to one right now. Maybe you like to read blogs and subscribe to email newsletters and stuff like that. Keep in touch.
Speaker 1:Email newsletters are a really nice way to keep on top of what's going on in the programming world, except they're actually not. I don't know about you, but the last thing that I want to do after a long day of staring at the screen is sit there and stare at the screen some more. That's why I started a different kind of newsletter. It's a snail mail programming newsletter. That's right. I send an actual envelope in the mail containing a paper newsletter that you can hold in your hands. You can read it on your living room couch, at your kitchen table, in your bed or in someone else's bed, and when they say what are you doing in my bed, you can say I'm reading Jason's newsletter. What does it look like? You might wonder what you might find in this snail mail programming newsletter. You can read about all kinds of programming topics, like object-oriented programming, testing, devops, ai. Most of it's pretty technology agnostic. You can also read about other non-programming topics like philosophy, evolutionary theory, business, marketing, economics, psychology, music, cooking, history, geology, language, culture, robotics and farming.
Speaker 1:The name of the newsletter is Nonsense Monthly. Here's what some of my readers are saying about it. Helmut Kobler, from Los Angeles, says thanks much for sending the newsletter. I got it about a week ago and read it on my sofa. It was a totally different experience than reading it on my computer or iPad. It felt more relaxed, more meaningful, something special and out of the ordinary. I'm sure that's what you were going for, so just wanted to let you know that you succeeded, looking forward to more. Drew Bragg, from Philadelphia, says Nonsense Monthly is the only newsletter I deliberately set aside time to read. I read a lot of great newsletters, but there's just something about receiving a piece of mail, physically opening it and sitting down to read it on paper.
Speaker 1:That is just so awesome, Feels like a lost luxury. Chris Sonnier from Dickinson, Texas, says just finished reading my first nonsense monthly snail mail newsletter and truly enjoyed it. Something about holding a physical piece of paper that just feels good. Thank you for this. Can't wait for the next one. Dear listener, if you would like to get letters in the mail from yours truly every month, you can go sign up at NonsenseMonthlycom. That's NonsenseMonthlycom. I'll say it one more time nonsensemonthlycom. And now, without further ado, here is today's episode. Hey, today I'm here with Dave Farley. Dave, welcome.
Speaker 2:Thank you, nice to be here.
Speaker 1:So I think my favorite three programming authors are Martin Fowler, steve McConnell and yourself. I own a number of your books and most recently well, second most recent book I bought of yours was Modern Software Engineering, and that one. It was one of those books where, like every section that I read, I was like yes, yes, yes, yes. And I often get the question from new programmers of what book they should get, and for a long time my answer was like I don't really know of one single book. That's a great starting point for a new programmer. But ever since I got Modern Software Engineering, that's the book that I point people to now as the book to start with. That's kind of the best overview of programming. So I just want to say I really appreciate your work.
Speaker 2:Well, that's great, thank you. So first I should say that's pretty good company to keep McConnell and Fowler. I'm very proud of that. Thank you, yeah. So modern software engineering it was a bit of a labor of that. Thank you, yeah. So modern software engineering it was a bit of a labour of love. I suppose I spent a while writing it and a while before that having the ideas that kind of eventually came together, and I wasn't quite sure whether it would work for anybody else. It was stuff that I wanted to write about but I wasn't sure whether other people would get what I was after. But I think it did land better than I thought maybe. So I'm very pleased with it.
Speaker 1:And it's a book that.
Speaker 2:I'm proud of. Yeah.
Speaker 1:Yeah, that's good to hear. It certainly resonated with me. Before we go much further, if anybody's listening who is not familiar with you, can you tell us a little bit about yourself?
Speaker 2:Sure, I'm a long-time software developer. If you're watching this with pictures you can see that I'm very old. So I've been doing professional software development for something in the order of 40 years and mostly at the slightly more complicated end of things. I used to build system software for computers and device drivers and those sorts of things and then I got into large-scale distributed programming of different kinds and so mostly at the slightly more complicated end of the spectrum, I suppose At the start of this century I began working at ThoughtWorks and at the time we thought it was probably the largest agile project in the world. We were building a point of sale system for one of the biggest project in the world. We were building a point of sale system for one of the biggest retailers in the UK and it was an interestingly complicated project. So there were about 200 developers working on it doing some version of Xtreme programming and that was challenging to everybody and Xtreme programming at the time, because Xtrememing was an approach that was designed for small teams. So we started, so we were kind of stressing Extreme Programming a little bit, I suppose, and started learning some of the lessons of what it took to make that really work.
Speaker 2:And I think of my first book that I wrote with jez humboldt continuous delivery as coming out of that. And continuous delivery was a very successful book. Um, my publishers told me that it was voted the 25th most influential book on software. Influential book on software, oh wow, which was pretty good going so, so so that that surprised and delighted jez and I when that happened. But that colored my career from then on. So I got involved in other projects and doing continuous delivery on those projects and so on, and then eventually I started consulting and so these days I'm a consultant, an author and a youtuber. As it happens, I start during the pandemic. I started a youtube channel, so so everything's kind of technical, everything's kind of rooted in in my foundations as a software developer, at least from my perspective. So I've come to the belief that I've come down very heavily on one side of the argument. Whether we are engineers or not, yes, we should. Well, mostly we're not, but we should be, is my argument. Yes, yeah.
Speaker 1:You know I'm quite familiar with that debate and when I was reading modern software engineering I realized that the answer to that question is software engineering, really engineering? The answer is it depends on how you do it.
Speaker 2:Yeah, absolutely. And when you do find something that I think really qualifies as engineering, as you might expect from other fields of engineering, you get remarkably better results, and that's what continuous delivery is. I think that's one aspect of it is that it's an engineering discipline that allows us to get remarkably better results.
Speaker 1:You know, I want to mention something other, something else that was very influential on me. Um, there was kind of a watershed moment in my life, um, I'd say in 2024 last year, as we record this, um I, I I read two books um, um, the fabric of reality by david deutsch and the beginning of infinity by david deutsch. Um, and what prompted me to read the beginning of infinity was that maybe this has happened to you before, um, you just keep hearing about a book over and over and I kept hearing about it, and when I read about you know, you mentioned the beginning of infinity in modern software engineering and I was finally like, okay, this is the last straw, like I finally just have to read this book because I've heard about it so many times.
Speaker 1:And then I read it and it just absolutely blew my mind, yes, it. And it just absolutely blew my mind, um, yes, which, which was really interesting to me because I'm 41. Last year I was 40 when I read this book and I kind of thought that like I had had my worldview, um, uh, turned upside down, um, all the times that it could. You know, I I read a few books in my mid-20s that I would consider life changers, um, and I thought, you know, once you kind of come out of plato's cave, like you can only come out of plato's cave once you know um. But reading these two David Deutsch books made me realize just how much there was that I didn't know, that I didn't know, and it completely changed the way that I look at life. And I'm curious. First of all, obviously you've read the beginning of infinity, because you mentioned it in your book. Have you happened to, have you happened to have seen david deutsch's other book? And my other question is how, how have, how has david deutsch's work impacted you?
Speaker 2:uh, yes, I've read the fabric, fabric of reality as well. Um, let's, let me start on the beginnings of infinity, because I agree with you entirely. It just kind of changed my world view. It made me, it gave me a richer, deeper understanding of things that I find fascinating and care about a lot. So so I, I am, I'm, I treat physics as a hobby. I love physics, and so I do spend quite a lot of my time reading, watching podcasts and YouTube videos and stuff like that on the topics of physics, and I just find it fascinating.
Speaker 2:So I can't remember I think it might be my brother-in-law, who has a PhD in physics, who might have mentioned the book to me as one that I might like, and I started reading it and I read a lot. I've always got at least one book on the go, usually more than one at a time, and so I read fairly efficiently. Efficiently, I usually read reasonably quickly. It took me about six months to read the beginning of infinity because I had to keep stopping and thinking about the implications of what I just read. There is so much, so many mind-blowing ideas in that book that you have to just stop and think about and, you know, crunch over the ideas a little bit before.
Speaker 1:Yeah, yeah, the it's very high density um the way I thought about it is like it's a book that can provide like years of intellectual nourishment. You know it's not a book that you read and you're just like, oh, that was pretty cool. Okay, I learned a thing or two. It's the kind of book that you do. You have to spend a long time digesting yeah, in the uk there's a.
Speaker 2:There's a well-known radio program that's been running for years and years and years decades even um where they interview somebody famous and then get them to recommend their top 10 songs of all time. It's called Desert Island Discs and the idea is you're going to go on a desert island and these are the only 10 songs that you're allowed to take, and as well as that, you're also allowed to take a book. It's the only thing that you can you know?
Speaker 2:So the book that I take is fabric is, uh, the beginnings of infinity, because I'd read it over and over again and get more from it every time.
Speaker 2:It's just so, but just to try and pique anybody's interest. That hasn't come across this book, but he's interested in this kind of thing. What I think david deuts Deutsch is probably, you know, the new version of Karl Popper. He's kind of come across with the. You know he's kind of like a new Darwinian. He's kind of revamped Popper's original thinking about how science works and what it really means.
Speaker 2:And Popper was the person kind of that really introduced the idea of falsifiability, which is a core concept in science these days. And David Deutsch takes that even further in terms of the way that he talks about it and he has this whole model about how it is. Science helps us to be able to create new knowledge and it's wonderful. It's just such a brilliant way of thinking about things and I find myself mentally referring back to those ideas often when I'm, when I'm trying to, you know, think about working, so on. So so the influence that david dosh has had on me is quite big, but only from those two books. There's other of his stuff that I can't get. So his current thing is something called constructor theory and I really can't get my head around what he's talking about.
Speaker 1:Okay, yeah, I, you ever listen to that podcast mindscapes with uh sean carroll yeah, david deutsch has been a guest on that show at least once. And you know I've been really frustrated because David Deutsch wrote the Fabric of Reality in, I believe, 1997, and then the.
Speaker 1:Beginning of Infinity in like 2011 or something like that, and it's like come on, like, like write some more books. I, I want to read. He's the kind of author who I want to read everything he he does, you know, but he's just not very um prolific in his number of books, but I suppose he makes up for it in the density of the few books that he does put out yeah, I, I.
Speaker 2:I think you have to give me a pass on that yeah yeah it's better than you know.
Speaker 1:Some authors put out dozens and dozens of books and they're all kind of the same idea, repackaged or whatever um.
Speaker 2:So you've read all. Sorry, you've read all my books um, um.
Speaker 1:so one of the things that I really okay so I want to tie this back to programming One of the things that I really took away from David Deutsch's books is the ideas of universality and reach.
Speaker 1:And it kind of put a name to an idea that I had thought about for a long time but maybe didn't know how to articulate all that well. Some ideas have more reach than others. For example, in programming, if you learn a certain JavaScript library, you've learned that JavaScript library, but maybe that knowledge doesn't have much reach beyond that particular library you might incidentally, learn a few universal things, um, but, but it's, it's pretty limited.
Speaker 1:But then if you learn something like um, automated testing, for example, um, that has enormous reach. It doesn't matter what you're working on, you you can always apply the principles of and I shouldn't even say automated testing, I should say test-driven development, because you can practice test-driven development. Even without automated tests, you can still apply a lot of the same principles. So that's an idea with enormous reach, and I've always encouraged programmers to focus in areas that have more reach. The way I used to put it in the past was focus more on principles and less on tools. Tools are necessary, of course, but again, they don't have all that much reach where, if you learn timeless principles, those ideas have so much more reach and so it's so much more profitable to spend your time studying those areas.
Speaker 2:Absolutely and to link those things together. Actually, that was what I intended my modern software engineering book to be about. Really was to try and talk about those fundamental principles that are always true talks about optimizing to manage the complexity of the systems that we build, and so I talk about five different practices or approaches that help you to do that. And they're the obvious ones when you stop and think about them. They're the ones that we already know. So modularity, cohesion, separation of concerns, abstraction and coupling, and so if you work to always optimize for those things, you will end up with a better result, a better design, a better implementation than if you don't. And I have said all of that and I haven't once mentioned a programming language or a toolkit or a library doesn't matter, those things are secondary. That's just as true in haskell as it is in visual basic. If you write modular, nicely structured code with good separation of concerns and all of those other things, it will be better than if you write code that isn't like that, whatever the technology. And those are the things that really interest me. And, and as you say, I I would certainly count test-driven development as one of those practices.
Speaker 2:And again, the link to um david deutsche's book is that it's. That's really about falsifiability. So so, if you know, I think, if we think of the role of automated tests in programming, what are they for? They're not there to prove that our solution is good. They can never do that. However many tests we have, we may always have missed one. In fact, we will have always missed one. But if we treat them as a falsification mechanism, if one test fails, then our solution isn't good enough. So we need to go back and think again and improve our solution.
Speaker 1:Right.
Speaker 2:So I think those ideas are operating at a kind of different level, it seems to me, and they are more foundational, and I think that they are based on my observation and experience of working with some very good programmers. They are the kinds of things, it's those kinds of skills, that differentiate the great programmers from the run-of-the-mill programmers, it seems to me.
Speaker 1:Yeah, wow, okay, there's a lot that I want to dig into there. Um, okay, let's see, you mentioned those five principles. I wanted to get into that. Um, man, I just had it but I forgot. Let's, let's do the five principles first, I guess. Oh no, I remember, I remember so this falsifiability thing. One of the things that David Deutsch addresses in his books is the fact that we can never know anything for certain.
Speaker 1:And that was interesting because I didn't really believe that before. I'm like, of course we can know some things for certain, but like and I guess maybe there's exceptions, like I think. Therefore, I am like, you can know that you exist, but as far as like hypotheses go, you can never prove anything right, you can only prove it wrong and all of our knowledge can only be accepted as true tentatively. All we have is our best theories so far. Yeah, and I thought that was really interesting because, going back to the idea of reach again and I thought that was really interesting because, going back to the idea of reach again I made that exact same connection between knowledge being generated via conjecture and criticism.
Speaker 1:First we come up with a guess, somebody comes up with a guess, and then we try as hard as we can to figure out reasons why that guess might be wrong. And then we try as hard as we can to figure out reasons why that guess might be wrong. And if we can't figure out any reasons why the guess might be wrong, then we tentatively accept it as correct. Same with automated testing. We can never prove that our code behaves as we intend. We can only subject it to all the criticism we can think of in the form of automated tests, and if it, if it doesn't fail any of our criticisms, then we can tentatively accept it as meeting the behavior that we intend it to meet yeah, there's, there's another, another great physicist, that that.
Speaker 2:I certainly learned a lot from Richard Feynman, and one of my favorite quotes of his is that science is a satisfactory philosophy of doubt.
Speaker 1:A satisfactory philosophy of doubt.
Speaker 2:Yeah, so in science, we are always skeptical. However good the theory, we assume that, as you say, it's good enough for now, it's the best that we can do for now, but it's going to be wrong somewhere, and so we're going to doubt it. But we'll use it because that's the best explanation that we have so far, but we'll doubt that it's true. So so your example, I think, therefore I am. You could, even you could easily question that now. So so when people started, when the llms first started coming on the scene, and people kept saying all they're doing is predicting the next word, I was thinking I'm not sure that that's not what. All that I'm doing, I might just be predicting the next word. I was thinking I'm not sure that that's not what, all that I'm doing, I might just be predicting the next word as well. So my, I could be working like an lm. Maybe that's the way that my mind works. Maybe the consciousness, the awareness that I am because I'm thinking, is an illusion of an, a more complicated version of one of those kinds of neural networks. I'm doubtful. So those might be alternative explanations. And so, you know, science is a framework that allows us to try and find the best explanations that we can, based on what we understand, and the job of a scientist and a good scientist is always looking for ways in which they're wrong, because that's where the really interesting things happen, and I think engineering is the same.
Speaker 2:So the first software engineer was Margaret Hamilton, but she invented the term software engineering and she led the team that designed the flight control systems for the Apollo program, and one of her things that she said differentiated her approach to software development for that project and you know others was that, to put it in her terms, the system that they were designing was man-rated, in that people's lives depended on it working, and so they spent an awful lot of time thinking about the ways in which stuff could go wrong.
Speaker 2:So they'd come to some kind of theory of what to do to make their system work design their software, whatever and then they think about how could it go wrong, and then sometimes they design things in to cope with those ways of going wrong, and it's one of the things that saved the moon landing.
Speaker 2:So there's a famous moment during the course of the landing, seconds before the touchdown of the lunar lander, on the surface of the moon, in Apollo 11, where they get a 1201 and 1202 alarms, and those were features that Margaret and her team built into the software that when the very, very limited computer became overloaded with work it would reset and prioritize the most urgent pieces of work, and that, and when that happened, they get a 1201 or a 1202 alarm. I can't remember the difference between the two now, but that's what would happen. So they thought about this event that they they didn't know would happen, but they thought possibly could, and then they designed their software to mitigate the consequences of that error going wrong. That's the kind of that's the sort of engineering thinking that I like that kind of skepticism over my own thinking, my own work I'm going to doubt, looking for holes in it all of the time and seeing whether I can either eliminate the opportunities of those problems occurring or, if I can't, then mitigate what happens when they do.
Speaker 1:I want to draw another connection there, just to say it explicitly. That again was another case of conjecture and criticism. It sounds like they were saying, OK, okay, here's our idea for this program. Now let's think of anything that could go wrong. Let's subject it to criticism and then update our, what we're building to meet that criticism yeah, absolutely, and certainly one of my formative experience.
Speaker 2:I got involved after I'd been involved. Well, while I was finishing up writing the Continuous Delivery book, along with Jez Humble, I was working on a project for a startup company to build one of, if not the world's highest performance financial exchange, and we were doing some pretty out there stuff. It was pushing the boundaries of software development in performance terms. So it was some quite cool stuff to be working on, and we had lots of problems that we didn't know how to solve. And you know, we had lots of problems that we didn't know how to solve.
Speaker 2:And so, applying that kind of thinking so that we could, you know, mitigate the risks of things going on, because this was, you know, if we were to be successful and we were this system was going to end up trading tens, maybe hundreds of billions of dollars worth of other people's money and so um.
Speaker 2:So you know it had to work as well, because otherwise, you know, we'd get out, we'd go bankrupt really fast.
Speaker 2:So so it's an interestingly challenging problem to have to try and figure out how to solve those sorts of problems and when, when you don't know the answers, when you can't just look it up on Google or Stack Overflow. You know you've got to try and figure out the problems yourself, and that's again an aspect of engineering. Is this, this discovery process, and so certainly part of my inspiration and part part of my inspiration for the modern software engineering book was that experience and realizing that what we'd fallen into was engineering, because that's what worked. So, and and part of that was being very skeptical, and one of the ways I always thought about it was that our job as software developers is to solve problems with software, and just writing code to solve the problem isn't enough. We've got to understand the problem well enough and solve the problem, and you know, and use code as the tool to implement that solution right, they're not exactly the same thing so I want to make a point about that real quick.
Speaker 1:Um, I've said before that most developers think about um software systems as being made out of code. Um, and superficially they are, but really a software system is made out of ideas, yes, and the code is just an expression of those ideas.
Speaker 2:Yes, yeah, it's like writing a book. You're telling a story. The story is not the words, but the words are the means of conveying the story.
Speaker 1:Right, and there's this like ghostly thing behind the code. What? You might call the conceptual model of the system. What you might call the conceptual model of the system and most sadly, I think most people don't even the existence of the conceptual model, doesn't even occur to them. But the conceptual model is absolutely the most important part of the system.
Speaker 1:And it runs through everything. It runs through the high level idea of what the product is and how the product is going to work. It runs through the UI. I always say that the quality of your code can never exceed the quality of the UI. The UI sets the upper limit on how good the code can be, because the UI kind of dictates the conceptual model. Or rather, the UI is downstream of the conceptual model and then the code is downstream of that. So your code can never be higher quality than the UI, because the code can never be higher quality than the conceptual model. That's an idea that I think everybody could greatly benefit from if they thought more about it.
Speaker 2:I'm not quite sure that I buy that 100%, but I think I see what you mean.
Speaker 1:I'm curious what part don't you maybe buy? I could imagine, think I see what you mean.
Speaker 2:I'm curious, what part don't you maybe buy? So? So I could imagine you know solving a problem nicely and then misusing that nice solution to write a crappy UI. I could imagine doing that being possible. So I think I see what what you're, I think I see what you mean, but I would think of it slightly differently. So let me first play back some words to where I'm at.
Speaker 2:So I've thought for a long time, and many people have said this. It's not original to me, certainly, but the usability of a piece of software doesn't stop at the UI, so it's an iceberg. You know that you're able to build good UIs because the design of the software lower down supports what needs to be there to build a good UI. So, if you know, if you think about an iPhone or something like that, where you're interacting with the user interface through gestures and so on on the screen, that's supported much more than just the widgets on the screen. That's part of the model of interaction with the bits of software underneath. So certainly at that level I think that what you're saying is true. But you can, you can have, you can have crappy apps on top of that model built on on here as well as good apps, even though the model underneath it is pretty strong yeah, I think I should have qualified my statement more um.
Speaker 1:So I think maybe what I'm saying is kind of true in a broad strokes kind of way, but but not universally um I warned you that I'm a skeptic.
Speaker 2:I'm sorry. No, that's good.
Speaker 1:I want my. You know, one of my favorite quotes is he who knows only his own side of an argument knows little of that, and so if my ideas aren't subjected to scrutiny, then I'm not sure if they're really any good. The scenario that I'm thinking about, which, sadly, I've had to live through many times, is when the engineer is given a UI design or a product design that is conceptually corrupt, and you have to write the code that supports this conceptually corrupt UI.
Speaker 2:That's the main case I have in mind, and I'm with you entirely there, and I'd go slightly further. I would say that's a rubbish idea. That's a terrible idea, because the design of your UI is not a requirement. That's part of the solution. It's your choice.
Speaker 2:If you're building some software that has a need for a user interface of some kind and doesn't in all in some form or another, then specifying those interactions in detail is not the same as saying what the software is meant to do. So I can come up with alternative designs. Let's be concrete. Let's say that we're going to build a calculator and so part of our requirements is that when I say one plus one, I want to get the result two. Okay.
Speaker 2:So I could specify that by designing the UI and saying I'm gonna have a one button and a plus button and an equal button. Or I could say that when I add one and one, I get an answer of two. The second way of specifying what I want the software to do, I would argue, is much more on David Deutsch's infinite scale than the other one, because now I can imagine a calculator with buttons or that I could type in the words one plus one equals two, or I could speak the words, or I could use some kind of thought control interface to my computer and just think one plus one and it gives me the answer two. And the requirement is still the same so the buttons are just part of the design.
Speaker 2:They they're part of the solution and I think part of the requirement should be to specify what we want, not how we get it, because part of the job of the software development team is to decide the best way to deliver those answers.
Speaker 1:Yeah. So this is where I think psychology comes into the picture, and I think, in order to be an excellent programmer, you have to be a competent psychologist. Excellent programmer, you have to be a competent psychologist. Um, you know when, when programmers are very junior, um, they tend to not realize that they are not only allowed to to criticize the instructions they've been given. But you have my opinion, you have an obligation to criticize the instruct, uh, scrutinize the instructions you've been given. Um, and you know if, if somebody says, well, if somebody gives you instructions to take an extreme example that are, um, uh, logically impossible, uh, then you shouldn't, yes, sir, and then get to work on trying to implement that, because it's literally not possible to do that. And less extreme is when you're asked to do something I won't try to use nice language when you're asked to do something stupid, you shouldn't just say yes, sir, and implement that stupid design.
Speaker 1:You should push back on that, something that's really frustrating, that I've run into so many times in my career. There's this natural fact of software that you can't get it right on the first go. Yeah, you have to, you know. You make your best guess early on as to what's going to meet the need, and then you know there's that saying plans never survive contact with reality. Software designs never survive contact with reality either.
Speaker 1:In about 100 percent of the cases, there's at least one fundamental concept, uh, which you realize was um, was a mistake, and you're like oh no, this thing I had this case recently this thing I called a build should have been called a run, and build was like the central concept in my application. It's like, oh no, now I have to go back and incur this enormous expense of changing every instance of build to run. And that takes a huge amount of discipline to do that.
Speaker 1:And sadly, it's an amount of discipline that most organizations don't have, and what really frustrates me is how many people won't say you know what, it would be better if we did that, but we're just not going to.
Speaker 1:We're never going to do that, so forget about it Instead what most people do is they say, yeah, that's a good idea, we should do that, just not now, and they're not honest with themselves. And they, they pretend that they're gonna make the change at some time in the future, um, but in reality, they don't ever intend to do that. That frustrates me I don't have a point. I'm just ranting at this point.
Speaker 2:No, no, no, I'm with you. Part of my solution, I suppose, like lots of other things, is to do the shift left thing, is to bring that to the front. And so if that's the truth and I'm with you 100%, it is the truth and not just software any complex system starts off with the simple system that works but isn't quite what you want. And the way that you get to a complex system is that you keep changing the simple system in ways that move you in the direction of what you want, and even then that's not enough, because what you will want is going to change, because the world's going to change, and that's true of everything, that's not just software. It's true of airplanes and cars, and roads and fences and everything.
Speaker 2:And so in software, the difference is that we, we move more quickly, and so it's more evident that we're making that, that those things are wrong, more quickly. They also also software's complicated stuff, and, and so I let me get around to my point. My point is is that if that's the reality and it is then there's only one defining characteristic for quality in software, that is, that it's easy to change, because everything else we're going to get wrong. And so, designing our software and our software development process, so that it's easy to change things, is the number one priority, the prime directive of software development, and so for me, that's what continuous delivery is all about what TDD is all about, what modern software engineering is all about.
Speaker 2:Why modularity, cohesion, separation of concerns, abstraction and coupling are important is because they make it easy to change our software. Why the other practices that allow us to learn quickly, like iteration and feedback and experimentation from the modern software engineering book, are also important, so we can figure out where our ideas are wrong yeah, yeah that's.
Speaker 2:That is. That seems like the foundation of how we all ought to organize things. The rather extreme example of the financial exchange that I mentioned earlier that I worked on. The company was called LMAX. If anybody's interested, they're still around and trading and very successful. So the LMAX system.
Speaker 2:We got the architecture wrong fundamentally four times on the route to finding something fast enough. But we knew some of what we needed to do. Other things we need to learn and change as we went. But we started off knowing that we got to hit certain latency targets. Turns out later that they were wrong. They were too low and not specific enough, but they gave us a target to aim for. So we started off.
Speaker 2:We built the first version of the exchange along the lines of that we thought would work, based on some simple experiments. And we tried it and it wasn't even close to fast enough. So then we looked at it again and we measured it better and figured out where it could improve. And we tried again and it wasn't even close to fast enough. So then we looked at it again and we measured it better and figured out where it could improve. And we tried again and that wasn't fast enough either. We looked at it again and we got something that was just fast enough, but didn't have much head room, but gave us some really good ideas about what to do next. And then we did the new thing and that was way faster, way better just to give a little context to this.
Speaker 1:So people kind of have an appreciation for what you're doing I happen to have read a little bit about this and and this um, performance in the financial, financial industry is insanely competitive. I I don't know if I I don't know if this is true, I just read it and I'm probably garbling my memory. But like they'll do things like, for example, um dig a new path, like through the mountains or something like that. They'll move their cable from one place to another, uh, just to shave off like a 100th of a second or something like that.
Speaker 1:So like these it's worth making an enormous investment for what seemed like really small returns, but again, it's so competitive that these seemingly tiny returns are actually quite meaningful.
Speaker 2:One of the companies, one of the companies that I worked for, later spent I'm going to hundreds of millions of dollars at least building a microwave link between Chicago and New York for exactly that reason. So Chicago and New York are two key trading hubs in the states and so if you could reduce the latency of communication it was worth that hundred millions of dollars investment because you would now get the signal about trading a fraction of a second sooner than your competitors. And yes, so we do all sorts of crazy things to get those kind of speeds. To put this into context, some kind of context, some numbers against it so the LMAX exchange, the core of the exchange, the matching engine, could process 6.5 million trades a second.
Speaker 2:Wow. So the round-trip latency from the edge of the system across the network into our system to the matching engine carrying out a match and back out to the edge was 40 microseconds. Wow. So I mean unbelievably fast. If you kind of figure out network latencies and the cost of a network stack and all that kind of stuff in that code, there's not very much time left for the code to do anything useful in that amount of time. So it's an interestingly challenging kind of problem domain to work in certainly and unusual. Not everything has to be that fast, but I think the challenge of being pressured hard by the constraints of the problem is the interesting one, because that pushes you in the direction of trying to find a more disciplined approach to solving problems.
Speaker 1:I think yeah, and I think it's good to have the experience of having very um diverse challenges, because then you'll discover the universalities that run through all these challenges and then you can take them back to programming, if that's your main thing. And you'll be better equipped to do a better job of that.
Speaker 2:Yes, and a range of experiences is useful. I don't think that you have to be a deep expert on a wide range of topics, but having an appreciation. I used to work at the consultancy and if you wanted to get into the senior levels of the of the consultancy in terms in technical roles, you had to have input, successfully implemented a system in three different technology stacks. So you couldn't just be a Java expert or a dietnet expert or whatever else it was. You had to have done it in three different, completely different tech stacks. So you had an appreciation of some of those common principles and commonalities and how they spread across. I I've not come across another organization that do that. I thought it was an interesting choice it is and I don't I don't completely disagree with it.
Speaker 2:I think it's probably quite a good idea. It certainly teaches you something different to just deep expertise in one technology yeah, the yes, um.
Speaker 1:Another quote that I share on this podcast. A lot is or it's more of a parable, I guess, but there's these two fish swimming in the water, a young fish and an old fish. And the old fish asked the young fish how's the water today? And the young fish says what's water? Illustration of how you need to get some perspective in order to understand things. There's an I don't know why I'm spouting so many quotes today but there's another one there's nothing worse than the worldviews of those who haven't viewed the world, which I find really interesting.
Speaker 1:You know, people complain a lot here in the US about the weaknesses of America. I'll jump right into a controversial topic because it's a big one. You know, people complain about the racism in America and how America is so incredibly racist, and I'm like well, have you ever visited any other country? Because I got news for you. Like racism is worldwide, worldwide, and in a lot of countries they're just totally open about it.
Speaker 1:Um, it's, it's totally nuts, um, but it's like that what's water kind of thing where if you've only ever experienced one thing, then you think that phenomenon is unique to that one thing you've experienced. Um, or rather, you know a lot of people make the assumption that that phenomenon is unique to that thing they've experienced. But then, if you go and experience other things, you're like oh okay, now that I've gotten a sample of several other things, I see that, yes, these few things are unique to my country or whatever, but these other things are just universal things. It's like this everywhere, and here's why this is different, here's why this is the same. All that kind of stuff.
Speaker 2:Yeah, absolutely, and that matter of perspective is an important one to be, able to start to have enough experience to build those broad, to spot the spot the water, to spot the broader, the broader issues that you know that you know that really matter. And I was reading something today that was talking about the difference between. It was an article that was it was pointing at a number of different differences between junior programmers and senior programmers and one of the things it was talking about was the focus of junior programmers on code. Because when you're a junior programmer, it's the translation into it's understanding and wielding the programming language, it's the hard part and you know, a senior programmer doesn't really care very much to the same extent. They don't care because that's not a problem. That's like you and I talking now. We're not busy analysing every word that we're going to speak. We're trying to express our ideas clearly, but that's kind of a secondary, lower order part of our brain that's doing that. It's the same as right.
Speaker 2:When I'm writing code, I'm not worrying about the syntax of the programming language that I'm working in, working in or anything like that, or what the library does or whatever else. I'm thinking about the shapes that I want to paint in the design of the code that I'm building, and this is a very different kind of, a very, very different kind of perspective and a very, very much more valuable kind of perspective, I would argue. But so how do we get junior programmers to be thinking and talking in those ways? Part of our problem, which I think you kind of touched on and we didn't talk about before, is that I don't think that as an industry, as a profession, we talk about design enough. We don't teach it. We don't. You know, we teach programming, we teach computer science, we don't really teach design. We don't teach what's good about design. How do we optimize for better designs or worse designs and and I know that's a slippery topic, but it's deeply important to building good software?
Speaker 1:I totally agree. And what's crazy is we talk so much about good code and bad code and stuff like that, but we talk so rarely about what good code is.
Speaker 1:I think I've been programming for a good 15 years before it even occurred to me to ask what good code was. And when I asked myself the question, I realized that I didn't have a ready answer. It took me a little bit to figure out what I thought was the right answer and just to totally put you on the spot, but I have a hunch that you do have an answer at the ready. What, to you, is good code? Easy to change? Yeah, yeah, I suspected that that's where you're gonna say. My definition is it's code that's easy to understand and change, which- I cheat because you can't.
Speaker 2:it's not easy to change if you can't understand it, so I'm getting two for one.
Speaker 1:Yes, but to be maybe even overly precise, there are often cases where you need to understand code even if you don't need to change it.
Speaker 2:Yeah, but if it's easy to understand, it's going to be easy to change as well actually, you know what I think.
Speaker 1:That might not be true, because I used to think the same thing, but then I realized some things. There are ways, okay, so I hope I don't have to explain myself too much, because I don't know if I can, but I think there are ways of structuring code where it's equally easy to understand in two different structures, but one way is easier to change than the other maybe.
Speaker 2:I'm sure that there are examples. I suppose that part of my I've been doing a YouTube channel for five years now, and so part of my psyche is one of trying to be a teacher and trying to find ways of expressing ideas more clearly, and one of the things that I've kind of come to is that it's hugely about the way in which we communicate, and often it's more about communicating in plain, straightforward ways than being perfectly correct.
Speaker 1:Yes.
Speaker 2:So I probably use easy to change as a shortcut to a lot of other things that are going on in my mind when I think about what that entails, what's included in that, which again is partly what, to my mind, the modern software engineering book is supposed to be about to try and be clearer about what some of those other things are, yeah, but fundamentally yeah but fundamentally, the whole book is about being easy to change.
Speaker 1:Yes, and as a fellow occasional teacher, I think it's so much better to successfully get across a sufficiently useful approximation to the truth than to unsuccessfully get across the perfect truth.
Speaker 2:Yes, yeah, yeah. So I think that's so part of what I mean by easy to change. You know, I've certainly said this in my writing in some places. It might have been in the modern software engineering book, but part of that is within any given context you should be able to almost just glance at the code and understand what it's doing, because then you can hold the whole context in your head. That's all here. It's abstracted away from the bits that are around. It's modular enough that I can change this without impacting anything else and and so on and so on, and so now I can see, I can understand what it is, and now I can change the stuff that's in front of me and all of the consequences of doing that are in front of me. That's that's.
Speaker 1:You know, that's kind of that would be good code, if you would like that yeah, yeah, um, for for a class that I taught a year or two ago, um, I would ask my students to scrutinize their own code and before I gave, any suggestions.
Speaker 1:I would say well, what do you think of this code? Do you find it easy enough to understand? Do you think it needs any more work? And I realized that I kind of had to help them adjust their standards a bit, because they would look at the code and they'd say this is easy enough to understand. Because they would look at the code and they'd say this is easy enough to understand. And then I challenged them well, like if you could lay eyes on it for only three seconds, would you understand in three seconds what it does? They'd be like okay, well, maybe not in three seconds. And then I would say well, do you think you could make some changes that? Made it so you could understand it in three seconds, and that would give us some new ideas. Made it so you could understand it in three seconds, and that would give us some new ideas. And to me that's all about I mean, you said it, it's all about the idea of abstraction.
Speaker 1:I read in some object-oriented programming book oh yeah, designing Object-Oriented Software, I guess, is what it's called. It's a book I have on my shelf over there, if I remember correctly. It's a book I have in my shelf over there. Um, if I remember correctly, it's that book and it said flat out in in one section um abstraction is the most important idea in programming and that really struck me, because it's kind of rare to see just such a stark statement like that, and and I'm like, hmm, is it?
Speaker 1:And then I'm like, yeah, I think it is.
Speaker 2:Yeah, yeah, I think it is too and it's. You know, it's a bit of a fight sometimes to find the right abstractions that we want. But it seems to me fundamental to good software design that we're able to hide information from one part to another so that you can change your service that I'm talking to without forcing me to change mine that uses yours, or that's about abstraction and coupling, but it's about drawing those boundaries between things where I care about what's going on beyond the boundary here and I don't care about what's going on the boundary beyond the boundary there, and I think pretty much you want to minimize the degree to which you care what's going on beyond almost any boundary really almost any boundary really.
Speaker 1:something I've been trying to do in the last few years is to take these important concepts like abstraction and distill them down to like a one sentence definition which I found to be quite hard, um, and I tried to do it with abstraction, and it took me a few tries um to get to something I'm I'm happy with. Um, and if if I I can recall what I most recently came up with, it was something like abstraction is the act of hiding irrelevant details or distracting details and showing um what's essential, um, something like that.
Speaker 1:Um and something else I realized is that a lot of people have a different idea of what, just what abstraction is? Um, and I realized the word abstract has two meanings Um. One meaning is not having to do with any particular instance. One meaning is not having to do with any particular instance. So it means like more general and more vague. But then there's here comes another quote, there's that Dijkstra quote. The point of abstraction is not to be vague, but to create a new semantic level at which one can be absolutely precise.
Speaker 2:Yes.
Speaker 1:And I think that's such an important distinction. Yes, it's Go ahead.
Speaker 2:There's something that's kind of hard to nail down in words and I've tried a few times but about abstraction, about wanting to maintain consistent levels of abstraction at different parts in your design. So if I'm, if I'm writing a piece of code that I don't know, reads a file, takes some data out of the file, processes the data and then stores it away in a database, then there's a lot of things going on there and I don't want to just write all of those in one flat script that knows all of those things, because then I've got to know all of those things to be able to understand any one part of it. So I want to break that apart with abstraction. And the other aspect of abstraction that we haven't touched on in the definition so far is it's a form of modeling. We're modeling the boundaries, so we're generating a new model that allows us to express ideas precisely at this level of caring about the detail beyond it, beyond that boundary, so, so, so that's that seems like a useful idea. And then the methods, that kind of com that we compose. We want them to be at some kind of reasonably consistent level of abstraction. We want to get them to be so we can read the story of what, of what this method is doing. So you could imagine a method doing what I just said so read file, read data, process data, store data, and it doesn't say anything about file handling or what's going on in the processing or what's going on in the database, but that would be perfectly understandable, that that's what's going on at that level of abstraction, perfectly understandable that that's what's going on at that level of abstraction. Then you have a function that reads the data and maybe knows about handling, dealing with files and pulling it out and so on.
Speaker 2:And part of my abstraction now, challenge of my abstraction now, is to design how we're going to get the data from one of those places to the other in a way that's useful, you know, in both. So we need some kind of model as part of our abstraction to represent contents of the file, maybe, and maybe that's the model that we change and then pass on to the database, and the database knows how to crack it and get the data out. It needs to store it or whatever. But those then are all different, different, little isolated problems. That's very much part of my design philosophy, at least in the way that I approach software development, but, but I think it is. Abstraction is quite a abstraction, and cohesion I think quite slippery concepts ease. You know there's lots of little nuances. The more that you think about them, the more little wrinkles you can think about it. Oh, I didn't quite nail it with that. It's certainly hard to get a one sentence description of abstraction yeah, yeah, it is really hard.
Speaker 1:Um, and I'm working on a new book, um, that I've put out a couple books in the past and I'm working on a new one might turn out to be vaporware. So, dear listener, forgive me if this never sees the light of day. Um, but I'm calling it the philosophy of programming, um, heavily inspired by david deutsch, um, and, and my intention is to address, address these kind of metacognition ideas. Um, and there are subjective opinions in programming and there are objective truths. You know, it's subjective how well a certain piece of code expresses or makes use of abstraction or whatever. That's objective. But the fact that abstraction exists, like that's an objective idea. That's not a matter of debate. That's not a matter of debate.
Speaker 1:And, and so my question to myself is like how much of these objective truths can we lay out and explain in great detail to give people as firm of an understanding of these concepts as possible? It's okay, the exact structure of your code, maybe that's debatable. It's certainly debatable, but let's at least start with a common understanding, because what I've found in teaching is a lot of people they're not even ready to have these conversations about certain things because they don't have the foundational building blocks to be able to speak the same language and it's like whoa, okay, like we need to go to the very beginning. But, like I said before, it's hard to recommend many books about this stuff because not a lot of books exist. And again, that's why I was so glad to see modern software engineering, because it's like yeah this touches on, like all those important ideas, um, um, yeah, so those are, um.
Speaker 1:I wish everybody knew those ideas everybody should buy my book, then I think that's what we say exactly um and that's probably a good segue into into wrapping up, um, dave, is there anything you want to share with people so they can know about your books and anything else you're working on?
Speaker 2:So modern software engineering and continuous delivery are available on most good bookstores, amazon and so on. I've got a couple of books on LeanPub that are slightly differently that. We've just released one that's called the Software Developer's Guidebook, which is meant to be effectively a collection of bullet points on a wide range of topics to give people advice on careers, how to be a leader, how to organize teams, how to do test-driven development, how to be a leader, how to organize teams, how to do test driven development, how to design software so that it's easy to change lots of the kind how to do how to build deployment pipelines lots of different things, lots of different topics, and it's meant to be the kind of reference book that you can dip into and find advice on those those different things. That was released a couple of weeks ago and he's currently top of the charts on lean book. So that's nice.
Speaker 2:I'm very proud, very nice. Um, we've recently rebranded the youtube channel. Which used to be called the continuous delivery channel, is now called modern software engineering. Part of that change was to bring in a bunch of other presenters other than me. So, as well as me, we've now got content from Tricia G, emily Baish, steve Smith, kevlin Henney and Kent Beck.
Speaker 1:Oh nice.
Speaker 2:Yeah.
Speaker 1:Yeah. So, dear listener, go check out Dave's books In particular. Again, again, I highly recommend modern software engineering. Um, I think whether you're just getting started or if you've been programming for 20 years, you'll get something out of it. I know I did um and check out dave's youtube channel. And, dave, thanks so much for coming on the show. It it's been a pleasure. Thank you for asking me.