Bryce 0:00

What is an algorithm? What would you say an algorithm is?

Conor 0:03

What is an algorithm? It's uh like a set of operations, a routine procedure for calculating something.

Bryce 0:09

Yeah, so that that's the definition that they that they give here. They give it as uh a way to do things. And and it's an interesting definition here where he defines it as an algorithm is like a set of steps that you can follow with pen and paper.

Conor 0:31

Welcome to ADSP The Podcast, episode 218, recorded on January 22nd, 2025. My name is Connor, and today with my co-host Bryce, we chat about a book that he recently purchased in London called Algorithms.

Bryce 0:53

You've got fake snow behind you, but there's also, I imagine, real snow.

Conor 0:59

It is minus 14 feels like minus 21. Let's get that in Fahrenheit for you, folks. That's minus six Fahrenheit. Yeah, that is like that is rough.

Bryce 1:24

Yeah. Yeah, it's pretty uh pretty cold. Funny thing. So I think Yeah, yeah. So so I I spent a few years living in uh Louisiana when I worked at LSU. And uh I remember one time there was like a frost in L at LSU. Um it was like February or something, and there was, you know, gonna be frost and maybe like, you know, some ice. And they shut down the university for like two days because there was a concern that there could be black ice on some of the stairs, and that uh the LSU students wouldn't know how to handle that, and so it was just best just to shut down the whole place. And um, I don't know if they actually got 11 inches, but uh Louisiana has been hit with like a ton of snow. They have like, I think like eight inches in New Orleans, and New Orleans folks, New Orleans has eight inches of snow. Like, what is this weird world we live in right now? Oh man. But uh I I just I I'm thinking about my old boss Hartmant, who uh he really, you know, one of the main reasons he moved to Louisiana was the weather. Like he doesn't want to be cold, he likes that it's nice and warm there. Maybe it gets a little too warm, but I'm just think I'm just thinking like the last uh day, like, God, that poor guy, like I, you know, I don't even know if he owns anything other than shorts anymore. And uh and it's gotta be brutal there.

Conor 2:52

Thoughts go out to those in Louisiana. And anywhere warm that is not I mean it's pretty cold for Toronto, but uh it's not atypical that once or twice a year it drops down this low.

Bryce 3:09

So you said it it is it is negative eleven here Celsius, and it feels like uh well it says it feels like that, but I think it feels it says it feels like negative fourteen. I think it feels closer to negative twenty.

Conor 3:24

Yeah, I mean I haven't gone outside yet today, but uh minimize it.

Bryce 3:28

I'm hoping I won't have to.

Conor 3:31

Yeah. Yeah, in the last couple days, uh trying to stay indoors is uh is a good play. Just go to the gym. Don't uh don't run outside these days. Uh yeah.

Bryce 3:40

I don't even have to go far to the gym. It's just uh I have the the BoFlex masters. That's right.

Conor 3:48

Looks looks like uh Transformers robot uh or whatever you call those things.

Bryce 3:54

It's really good when I'm not lazy. So how have you been?

Conor 3:59

I've been good. How about you? 2025, 2025.

Bryce 4:02

New year. Uh yeah, I'm good. I've been busy at work. I've actually, well, you're gonna you're gonna be shocked. The listener's gonna be shocked. I have not one, but two programming books that are like here that I've been reading this morning.

Conor 4:19

One of them is just called I wanna say that's programming pearls, but I can't actually read it because it's white text on beige.

Bryce 4:26

Um it's uh this one is just called algorithms. This one I te I I tw texted you about this like I got this in November when I was on the way back from Poland.

Conor 4:38

It's like a month or two ago.

Bryce 4:39

Yeah. But then this is just um elements of programming. And yeah. It has a very yellow cover and then it has silver text on the yellow cover. I'm not sure that I agree with that, like, cover design. But uh but yeah, I've been reading them and uh some interesting things. Some interesting things.

Conor 5:01

Um share share with us the uh interesting things that have been accumulated from these two books. Link in the show notes, folks. Perfect. We don't have affiliate links. If you click on them, you're I mean, EOP is for free, so Yeah.

Bryce 5:16

Not if you want this copy. This ironically, speaking of Hart McKaiser, this this book was given to me by my my mentor, my guru, Hart McKaiser, uh for my must have been 20th birthday. It was the first birthday that I was there in Louisiana, and he just came up to my desk and he gave it to me and he said, here. Happy birthday. And that is how I acquired my hard copy of Elements of Programming. Wasn't wrapped or anything, it was just here. You you need this.

Conor 5:50

So this is that copy? You made it sound like you picked up both of these books.

Bryce 5:54

No, no, no, this is that copy. I don't I I think that it is very hard to get this copy um of uh of EOP. Um I do not think that this has been in print for some long period of time. Yeah, I think that this has not been in print for a while.

Conor 6:12

Alright, so one book recently acquired, another book anciently acquired.

Bryce 6:16

Yes, I think this is also a first edition. Yes, it does appear to be a first edition. What are my many randomly valuable uh programming books? So I opened up the other book to the exact page that um that I wanted. So this this book is called Algorithms. I got it at, I went to this we we uh we had like an eight-hour layover um in London. And so my girlfriend Ramona um not liking to just spend time in airports waiting, she's like, let's go into the city and do something.

Conor 6:48

And so we went into this little Was this Heathrow or uh Yeah, this was Heathrow.

Bryce 6:53

Um uh we went into this little museum uh in uh I think it was Hyde Park. Let me go to uh let me go find London here on Google Maps. Uh yeah, it was Hyde Park, and there's this little museum called the Serpentine Gallery. Um and they had this exhibit on um artificial intelligence, of course. Um they had this music that was composed by various neural nets. Um it was just like a small exhibit. It took like, you know, 15-20 minutes to go through it. Um but then they had a a bookshop, and I had both Ramona and my mother with me. Um and so of course we had to spend as much time in the museum bookshop because they both love a good museum bookshop. And I saw this book, um, Algorithms, and I thought, hey, I have a we have a podcast on algorithms. Uh maybe I should pick this up. And it had on the cover, I don't know if you can see, the algorithm for GCD. Um and so I thought, okay, maybe this is maybe this is interesting. Um but it it I I I figured picking this up in a you know, not at a you know tech conference or something, I figured that this book was going to be something that was meant for more of a mass audience. And that is in fact the case. I think it's it's meant to explain to uh uh you know people who maybe don't have as much of a CS background or who aren't programmers what algorithms are. Um I think it's very structured towards our current age. The chapters are chapter one is what is an algorithm, chapter two is graphs, then chapter three is searching, chapter four is sorting, chapter five is page rank, and chapter six is deep learning. So very, very topical. Um and it's interesting because the first chapter like handles like talks about the idea that we're now in the algorithmic age, that we're past the computer age. Computers are not taken for granted, and it's all about the algorithm, and it talks about this idea that that people are starting to treat the algorithms like they're a higher power. You know, you talk about like, oh well, the algorithm decided this, like it's some magical thing. Um and I get- I feel like that does kind of reflect how some folks view technology these days. Um but it's interesting because it's both approachable, I think, to you know, anyone, but also like dives pretty deep into some pretty advanced topics. Um and I certainly think you need some background in math to be able to read this book and not be lost. But uh in the chapter one on what is an algorithm, I came across two interesting things. The first, and this is a quiz for you, I guess, Connor. Do you know the origin of the word algorithm?

Conor 10:00

Uh vaguely, yeah. It's I will I'm not gonna know the exact origin, but it's from the Middle East. If I had to guess, we're gonna go and say Persian.

Bryce 10:07

You are correct. It comes from the name of Muhammad Ib Ibn Musa Al-Kawazim.

Conor 10:15

Alashima, do you know the name of the guy that algorithm was named after?

Bryce 10:20

It's a car, yeah, it's Persian guy. It's a carbsmi or something like that.

Conor 10:24

She said the same thing I said. Yeah, he's a Persian guy. Uh so she didn't know the name either. Which means, let's go. And my uh fiance is like a history buff. She listened to, I want to say, all of that history podcast, correct?

Bryce 10:38

Yeah, but that doesn't make you a history buff.

Conor 10:41

It does.

Bryce 10:41

Al took informal courses.

Conor 10:43

Alright, we'll find you another history question later. Get ready.

Bryce 10:46

Uh, just his name is hard to say.

Conor 10:48

His name is hard to say. She says she does actually know it.

Bryce 10:50

Oh, it is. I just tried to pronounce it and I'm sure I did it very poorly, so I'm not gonna try it again. But, anyways, this this dude, Al Kawasimi. Al Kawazimi, we're gonna go with that. This dude uh was uh circa like 780 to 850, Persian scholar, uh worked in fields of mathematics, astronomy, geography. Um the term algebra comes from the Arabic title of his most influential work, and the translation of this title is pretty great, which is the comp the compendous book on calculation by completion and balancing. And I just think that's a great, a great book name, and I hope it's as good in the original or Arabic as it is in the translation. Um and uh his name was Latinized to algorithmus. Uh and then that in combination with the Greek word for number, which is arithmos, uh sort of evolved into the term algorithm. But algorithm originally just meant like doing decimal arithmetic. And it's only in like the last 200 years or so that it's come to mean uh what the second thing that I found interesting about this chapter is it's come to mean what an algorithm is. And the the second interesting thing is what is an algorithm? What would you say an algorithm is?

Conor 12:27

What is an algorithm? It's uh like a set of operations, a routine procedure for calculating something.

Bryce 12:33

Yeah, so that that's the definition that they that they give here. They give it as um uh uh a way to do things. Uh and and it's an interesting definition here where he defines it as an algorithm is like a set of steps that you can follow with pen and paper. And they repeat that uh a couple times. Um and he sort of makes the claim, the author of this book somewhere here, um, that an algorithm is something that uh you can carry out ourselves. Um I'm not certain what exactly he means by that or why that's significant, because I I I suppose he means something that like you can do in paper or you can do in like pseudocode. Um but he almost says it in a way that it's like it's something that that humans could do manually and just that computers can do it faster.

Conor 13:31

I think it makes it it it's like a layman's olive branch, right? Because if you think of the first algorithms we learned, at least the ones that I consider, it's long division and long multiplication. When you're in I don't know, we'll s we'll say grade three, but uh times may have changed for those that are born after the millennia, the turn of the millennia. Uh they might teach it sooner because that's what happens in uh education systems. So I guess it depends on where in the world you are. But I distinctly remember learning Yeah, long form multiplication and long form division. And it's just like a set of operations. You multiply the first digit in the second number with everything in the first, and then you you know add a zero for the second line. It's just a set of instructions. And I remember too being in math club for the like one min month that existed at DP Todd Secondary in Prince George. Shout out to Prince George. I remember at one point we learned the algorithm for um hand calculating square roots. And I thought that was very cool because up until that point, up until that 30-minute lunch session or whatever, I thought that was like a incalculable thing by hand. Because they didn't teach it in school. Like you just, oh, you go to your calculator and you hit the square root button. Um but there was there is actually like a set of steps where you converge into the you know square root of some number. And I thought that was fantastic, that this thing that was impenetrable and only a button on a calculator, I could now do by hand. And in your head if you um so desired as a party trick at some point. So, I mean, uh I do think it's kind of uh do I agree it's something you can do by hand? Not really, but I think I I looked up this series. It is the MIT Press Essential Knowledge series, and there's a bunch of other books with the topics like deep learning, data science, and machine learning, revised and updated edition, cryptography. So I think this is kind of the academic version of your airport books. You know the Malcolm Glabwell, bite-size, catchy titles, usually half of them are self-improvement, you know, atomic habits, stuff like that. This is like your science slash academic versions of those kinds of books that you're gonna find, which it's unsurprising that they describe algorithms in this kind of fashion, right? They're not targeting people that work at NVIDIA. They're they're targeting people that might also, in the same uh, you know, fell swoop, pick up atomic habits. And uh there's gotta be a little bridge from that walk over from atomic habits to uh algorithms.

Bryce 16:07

I can I I agree. I th I think that that is the the the essence of the uh of the uh uh the the target of this book. Um but so then a little later in the chapter, he he says a couple things that I resonate with. He says the only way to truly understand an algorithm is to perform it by hand. We must be able to execute the algorithm in the same way the computer would execute a program that implements it. And I find this interesting because when I'm implementing an algorithm or coming up with an algorithm, um I usually will first write it out um by hand, and I will first work through some problem by hand. Usually like, you know, in i uh like a smaller problem than what I would actually, you know, solve on a computer. Um like uh I was recently um the the summed area table uh that uh algorithm that we talked about uh on previous episodes, or I recently wrote uh a softmax implementation. And in both cases, I did what I always do, which is I I picked some like trivial small problem size and I sort of worked out on paper and also in Excel. I find myself increasingly using Excel to do this. I used to just do it in paper and notepad, but Excel um makes it easier to work with for these sorts of like uh multidimensional problems, where you know I wrote out my test problems and then I sort of imagined here's the series of steps that I would need to take to get to the solution. Or in the case of something like Softmax, I I can look up online, like it's not like I'm creating some new thing. I I have you know the code, I I I see this as the code of like what this does, but I want to understand like how does it actually work and to understand that so that I so I can explain it to other people, right? Because if I want to use it in a talk, then I need to be able to explain why we are doing each step of this thing. And so by going through and me manually doing each step and being able to look at each step, um, I can sort of develop the intuition for how does this thing work. Um I imagine that this is how most people go about understanding something. Uh uh is that you either work through it uh yourself manually, or maybe you you write some code um and print it out at each stage, but you look at like how does this work for some small problem? Um and that's sort of to me is like what he's talking about of being able to execute by hand. Is that is that what you do when you start playing around with some new problem?

Conor 18:59

I don't it depends on the complexity of the problem. And I'm also I'm also have the difficulty of just gonna wait for my lovely fiance. Finishes in the microwave and slamming doors. She needs to eat breakfast though, folks. Maybe I'll just leave all that in there. I love you, sweetie. I love you. Get that croissant.

Bryce 19:20

Um I hope I hope we're not microwaving a croissant, though. That seems like that's some sort of French food crime.

Conor 19:26

Uh I mean it's not a fancy croissant, and it is indeed being microwave.

Bryce 19:32

That seems like it's some sort of French food crime.

Conor 19:35

I don't know. Our listeners in France let us know.

Bryce 19:39

Quebec. France.

Conor 19:41

Qu the Quebec? Yeah. Um let us know if we violated some kind of um Yeah. Well, you can hear it, it you can hear it beeping. We might as well just wait because she's about to open the uh fun fun fun fun fact.

Bryce 19:54

Fun fact. My dad was visiting us last week and we were talking about family history on my dad's side. My dad was adopted, um, but he mentioned something that I think I'd known but uh uh uh hadn't remembered, which is that his biological father is uh French Canadian, making me one-fourth French Canadian, which uh I would say explains a lot about my my personality, my general love for Montreal, um just many my haughtiness.

Conor 20:26

Um not your love for Montreal bagels, though. You are first and foremost a New York bagel.

Bryce 20:34

No, I'm kind of a convert. Uh uh Really? Yeah, I don't know if I I posted to social media, but we did this uh Montreal bagel making class uh when we were up there uh before we went to Morocco. Um and it was a lot of fun. And they were very, very good bagels. I ate like four of them.

Conor 20:50

Alright, well you heard it here first, folks. Definitely not going to upset anyone on the internet. Former lover of New York bagels, Bryce Adelstein Lelbeck, has now converted and does think that Montreal bagels are superior.

Bryce 21:03

I mean, Montreal in general, just for food, is, you know, the best. Like, that's why you go to Montreal for the food.

Conor 21:09

It is true they do have um Felix and Norton cookies. I'll leave it at that, folks. Pink truck by the waterfront. Next time you're there.

Bryce 21:18

We went to this restaurant, Candide, and they had this preparation of spinach that was the best spinach I've ever had. I don't know what they did to these spinach leaves, but I could have literally eaten nothing but that spinach for the rest of my life.

Conor 21:32

Wow. And you can be like Popeye.

Bryce 21:36

Okay, all right, so back to algorithms.

Conor 21:38

Anyways, back to algorithms. The question was do I jot things out by pen and paper? And uh I was saying that I suffer from the problem of having been so far removed from the first time that I had to understand or visualize a problem. My first thought was that uh I definitely did not. Need to go and implement std reverse to be convinced that swapping the last and first elements uh one at a time and incrementing and decrementing two pointers until they converge yeah works. Like that seems simple enough that I automatically would be like, yeah, that's how it works. But is that actually how I felt? Because I would also say the same thing about one of my favorite algorithms, Floodfill. But I remember the very first time I I didn't learn a floodfill, I just organically implemented it in a qualifier for an ICPC uh collegiate programming contest when I was at SFU, there was free pizza, and it had this sort of graph or matrix problem where you had to go from point A to point B. And I didn't know how to do it, but I just thought, ah, I guess if you recursively start at your starting point and then add the number one, and then if you have some cue where you just basically, you know, recursively search the four adjacent uh elements from each square, and you keep a cache to ones you've already visited, um, that should work. I wasn't convinced that it would work because it's a recursive solution and kind of hard to wrap your head around the first time you implement or think about it, but then I implemented it, it worked, and I was like, ah, my idea worked. So there's an example of something I wasn't sure worked, but once again, I did not do this by hand and paper. I just implemented the algorithm. Then you you look at the, you know, you just output the matrix of numbers at the end of the day that is a zero for where you started and incrementing numbers for the flood fill path. Uh my point being is that sometimes paper and pen helps, but a lot of the times I just implement it on the computer, but I I feel like that's I don't know, like a 21st century equivalent of pen and paper. Um but yeah, I I don't know.

Bryce 23:53

I do even today like start off on pen and paper more than you might think. And I I think I've done that since the beginning of my career. Sometimes it'll be like on notepad or in Excel, like I said, but I think I very often will start off um before I even write any code, I'll start off just thinking through it. Um and I think part of that is that I've I've found that if I start coding before I think through it, um uh I'm more likely to lead myself down the wrong path and then get like tunnel vision. But oh but uh g give me a little more background on this floodfill. What what is the flood fill problem?

Conor 24:34

I mean it's got its own Wikipedia page. I'm sure if uh we will link it in the show notes, but as soon as you see the animation.

Bryce 24:42

Yeah, I'm I'm looking right now.

Conor 24:44

It's basically just a recursive exploration of a 2D array or matrix where you have a starting position. This actually animation is not great. And I believe they call it flood fill because it's the algorithm when you're in uh Windows Paint or some equivalent paint program and you hit that fill button and you need to fill a square, it does this algorithm when basically saturating a space, and I guess in in that example you're just changing one pixel color to another pixel color. And it goes in basically you start at an origin point, and then you have either four or eight directions, depending on how you're allowed to traverse. If you're allowed to traverse diagonally, uh it's eight. If you're only allowed to traverse orthogonally, it's up, down, left, right. And you start at uh it's usually you have a secondary matrix, which is the size of your original, and you set everything to uh either zero or negative one or something like that. Then you assign your origin point, the value zero, and then you put in a queue all the four orthogonal places that you can visit, and then you uh once you're done putting those in the queue, you're in some kind of while loop, you pop off the queue, the first element, and then you assign that one plus the previous value. So along with the coordinates of the position that you've stored in the queue, you also need to assign the previous value that you were coming from. And so then if you're starting at zero, the next four spots will all get the value one, and you just do that recursively. And you have to make sure, either with a cache or by just checking uh when I'm now at uh uh the second point and I'm checking the four orthogonal uh elements that I'm gonna visit, you have to make sure that they haven't already been visited. And in certain algorithms, you actually do need to check, and in if if so, if if you found a shorter path, uh you might need to assign a minimum value. In the classic case, you don't need to do that because you know that the way that the algorithm works, you're always gonna end up uh visiting elements in the shortest way possible. And then you just do that to completion.

Bryce 26:48

So the the the cache is to avoid having to uh to do it redundantly.

Conor 26:53

Yeah, but it the way that I would do it, like I said, if you set everything to negative one, you can always just check, make sure that I'm only exploring elements that currently their value is negative one. And then as soon as it gets assigned uh a value that's positive, um, or zero in the case of your origin, you know that you'll never visit it again. Be sure to check these show notes either in your podcast app or at adspthepodcast.com for links to anything we mentioned in today's episode, as well as a link to a GitHub discussion where you can leave thoughts, comments, and questions. Thanks for listening. We hope you enjoyed and have a great day.

Bryce 27:23

Low quality, high quality, that is the tagline of our podcast.

Conor 27:28

That's not the tagline. Our tagline is chaos with sprinkles of information.