4: Functional Programming - You're probably already doing it

People throw around the term Functional Programming but it's not always clear what they mean. In this episode, Wolf explains what goes into FP, and together we build a clearer picture that reveals you might already be doing it.

Show notes and things to think about:

1. Functional programing isn't academic.  It isn't overwhelming.  It isn't impossible to use.  It isn't inapplicable to ordinary problems like the ones you're solving right now.
2. You can use functional techniques in almost any modern programming language.  In fact, you probably already are.
3. Main pillars of FP:
   1. Pure functions (no side-effects)
   2. Functions are first-class objects (you can pass them as arguments, you can return them as results, you can store them in lists or any other data-structure)
   3. Data is immutable by default
   4. FP languages often provide powerful pattern matching syntax (didn't mention this much in the episode other than briefly noting Python's new match statement)
   5. A couple of things not mentioned: in FP, your code is more about what you want, not about how to get it.  That stack of functions for the sales data example looks declarative, not imperative.
   6. A couple of other things not mentioned: recursion and lazy evaluation.  Not exclusive to FP, but very often available in functional languages.
4. Papers and explanations about monads might be unreadable, but you're already using them and you already know how they work.
5. Using FP techniques appropriately can make your code easier to test, harder to break, and possibly even prettier to look at.
6. There are places in your code right now that you can make better right now with FP.  Do it!

Links:

• We mentioned a ton of languages.  Most of them have easy to find home pages so I'm not going to list out all the links; but there are a couple of obscure ones
  • There's nothing for the original Lisp, the closest these days is probably https://common-lisp.net.
  • ML can be found at https://sml-family.org but the more modern and popular variant, OCaml, can be found at http://ocaml.org.  Microsoft's take on this is F#, open-sourced at https://fsharp.org.

Hosts:
Jim McQuillan can be reached at jam@RuntimeArguments.fm
Wolf can be reached at wolf@RuntimeArguments.fm

Follow us on Mastodon: @RuntimeArguments@hachyderm.io

If you have feedback for us, please send it to feedback@RuntimeArguments.fm

Theme music:

Dawn by nuer self, from the album Digital Sky

Transcript

Jim: 0:03

Welcome to another episode of Runtime Arguments. I'm Jim McQuillen and my partner here is Wolf. Hello. Thanks, Wolf. We have got an awesome episode for you today. We're going to be talking about functional programming. This is something that I've been kind of confused about for a while. And uh Wolf, this is one where Wolf is doing the research for, and he's got some great stuff to talk about. I'm really excited to get into it. We're gonna get going right now. You ready, Wolf?

Wolf: 0:37

I'm ready. Um let me start with a little bit about Jim and me. Uh we're both, there's lots of different words for it. Software engineers, programmers, coders, whatever. That's what we do. We write code all day. And we have languages that we use primarily. Jim's a Perl guy and some JavaScript and some Swift. I'm a Python guy and some Rust, and I've done C and PHP in the past. Um the kind of programming that we do might be called procedurals or maybe object-oriented, or, you know, there's uh a couple different names for it. But there are languages that you see sometimes uh and people talk about. Um I'm going to talk a little bit about them. Um, and these languages are labeled functional. And when you try to talk to somebody who knows about them, they start saying things that are complicated and mathematical and seem like theorems, and and it's just a huge gatekeeping thing. Like I I feel like I'm a good programmer and and that I'm, you know, reasonably intelligent. Uh, and they say things that are completely incomprehensible to me. I feel like I can't know or understand what functional programming is. And I wanted to know. So I spent some time f I spent some time finding out what it's all about. Uh it's not just for academics or mathematicians. Uh the truth about functional programming is you probably already understand the most complicated, hardest parts of it, and you're probably already using those things in the modern languages that you are using today. Um we're gonna strip away all that jargon and show you what functional programming actually is, why it's a good idea for many situations, and how you can use it better. So let me start with um where this all came about. In 1958, um the grandmother of functional languages, Lisp, um, everything's a list, and everything is a function. Uh and from there, things developed. In the 1970s was a language called ML that introduced a thing called type inference. In 1987, um Erlang came along uh that was uh designed and built by Ericsson and ran phone systems that had 99 and some huge number of 9% uptime. Uh in 1990, Haskell. Um what if everything was functional, is Haskell's thought. Uh in the 2000s, we got languages more for human beings, like Scala, Clojure, F sharp, F sharp's Microsoft's thing. Um you have probably seen these things around. Uh the other languages I just mentioned, you probably see less. And there is a modern descendant of Erlang, and that's Elixir. Um, Elixir runs on the Erlang virtual machine. Um so uh these are some of the languages that are functional.

Jim: 4:16

So back up a second. You you're telling me Erlang was running phone systems? I I I thought it was just academic.

Wolf: 4:26

No. Erlang turns out, I mean, it was written by a company um specifically for this job and specifically to be robust. Um, you know, uh languages that are designed by companies um for their own hardware, uh, we see that all the time. Like Apple and Swift, for instance. But uh WhatsApp handled 900 million users with just 50 engineers using Erlang. It Erlang's a real world language. Um let me talk about some core concepts. Um functional, because the languages you you're using right now, they're not called functional languages. Most modern languages are uh multi-paradigm. Uh they're hybrids. They have functional things. So what makes code functional? Um a couple things, some of which you probably do use, and some of which you probably don't use. One thing is pure functions. A pure function has no side effects. You put in a number or something, whatever, and the answer comes out. If you put in the same set of parameters, the same answer comes out. It doesn't depend on anything else, like it doesn't check the weather. If your function adds two numbers together and you give it two and two, it produces four every single time. That's a pure function. Um there's downsides to pure functions. If you don't have side effects, you uh you can't write to a database, you can't uh delete a file, you can't update some global variable. Anyway, uh a second factor of functional code is immutability, especially immutability by default. That means your variables or your data structures or whatever don't change. Um this is very unlike uh a lot of code that we write. Um some languages have actually adopted this, and it's a a really strange shift when you move to it. But like Rust. In Rust, variables aren't variable. They are immutable by default unless you say something. Um the third thing is functions are first class. You can pass a function as a variable, a function can return another function as a result. Um these three big things um make up what is functional programming.

Jim: 7:21

So I I I've been hearing for a long time that functional programs don't have side effects in your functions. That's uh I'm I'm writing programs all the time that have to change data. They update databases. I'm writing business applications. Read data out of a database, modify it, and and update it. Uh how does that work in this paradigm?

Wolf: 7:46

There are two different um paths along which data gets changed or feels like it's changed. Um of those paths is that a lot of these languages that are labeled functional actually are divided into two sets of functionality. Um side is pure, and the other side is the thing that reaches out to databases and whatnot. Um and that's often the way you will write these programs if you're using uh a more modern language like Python or Rust or whatever. Um the other path is something that you see all the time. Um you take in a data structure and you give back a new data structure that looks a lot like the old one, but with your changes. And it feels like you've changed the data, but really it's a brand new thing. Just like in Git, when you add a new commit, yeah, the code is different now, uh, but you didn't change any of the old commits, you just added a new one on top. Um so that's how a lot of these functional languages work. And modern multi-paradigm languages don't have to worry about being purely functional. They've got data mutation built in, they know how to modify things. Um let me talk a little bit about functional programming syntax that you're already using. Um I'm gonna bring up a couple different languages, and this is why I started off by talking about uh the languages that Jim and I write write code in. Um I write a lot of Python, and one of the things that is used a lot in Python uh is comprehensions, especially list comprehensions. There there are different kinds of comprehensions, but a list comprehension is the most popular. And a list comprehension, I'm gonna say it in the most basic instance of a list comprehension, takes a list and gives you a new list. The action in a list comprehension changes, produces a new and different value for each value in the input list, and the output list is a list of those result values. In JavaScript, um there's a ton of array methods like dot map and dot filter and dot reduce. All of those are functional programming. JavaScript promise change chains? Yeah. Um I I don't want to leave Jim out. Modern Perl has a bunch of functional programming stuff in it. Um Perl uh gets uh I don't know, are they called packages, Jim? Yeah. Um so Pearl has list colon colon util, which gives you reduce any all first. Um higher there's uh a book called Higher Order Pearl uh that talks about functional programming. Uh l uh Pearl has closures. Um Pearl from the start had functional powers.

Jim: 11:30

So when Larry Wall designed Pearl, he included functional concepts?

Wolf: 11:36

Yeah. Larry Wall uh is a linguist. He took the best ideas from everywhere. Pearl's motto, uh there's more than one way to do it, included functional waves from day one. You've been doing functional programming since your very first map block. Uh I used map today. Um I have been exploring Rust, and um, you know, when I first came into Rust, I was incredibly dissatisfied because it was missing all kinds of things that I thought it needed. It doesn't have null or none, it doesn't have exceptions, um, I didn't see, but it turns out Rust has option and Swift has optional, and C sharp has nullable types. And um this is where it gets fun. Uh all of those things that I just named are almost um one of the gosh, it's it's so complicated, it's practically a joke among uh the people who do functional programming. Um those things are almost monads. Um but you already understand those things. Uh option is just a box that either has something or not. Um that's a little bit different than uh a pointer that might be null or a list that might be empty. Um it's just a little bit different. Um, but it is different.

Jim: 13:19

Wait, so monads are just option types?

Wolf: 13:24

Uh monads are the pattern behind options and results and promises and even lists. But here's the secret: you don't need to understand monads or monad theory or any of this complicated gatekeeping crap. Uh you already know how to use these things perfectly. I mean, if you're a Rust programmer, you know how to use Option. If you're a Pearl programmer, you know, you know what I'm saying.

Jim: 13:51

Sure, sure. Um if you really want to melt your brain, uh take a look at the monads page on Wikipedia. It's scary to look at. Uh I'm still feeling the effects of what I saw there. Uh and it leads to other pages that are equally scary. Uh, I I it's I don't understand why there's that much about monads that it's that hard to understand when in reality uh Wolf just explained they're they're they're I mean optionals are monads and other things are monads, but I don't know, still scary to me.

Wolf: 14:35

They're kind of like Schrdinger's cat. There's a box, and either there's something in it, or maybe there's nothing in it. Or maybe there's something totally different in it. Um don't open the box. Anyway. Don't open the box. So functional programming um has been a foundation of uh languages that we've been using in the past, and people who design programming languages have seen that it does good things, and so it's getting adopted as people write brand new or grow existing languages. For instance, uh Java 8 streams. Um Java basically went functional in 2014. User streams, filter active, map the emails, collect the list. Uh Python has uh more and more functional programming features. Uh you can see the things in iter tools, func tools, the brand new match statement, um type hints, making result types possible. Um Rust is uh incredible. Uh it it made immutability the default. You have to ask permission to mutate. Result type makes error handling explicit and unavoidable. Um it proves FP concepts work in systems programming.

Jim: 16:05

So basically, every programming language is becoming functional.

Wolf: 16:10

Yeah. Modern languages um are multi-paradigm. Uh they let you do uh basically whatever approach solves the problem you are facing. And that's what these languages are for in the first place. They're to solve problems. Um so providing more and more tools to solve problems, that sounds like the right answer to me. Uh like every interesting kind of technique that you use in programming, um functional programming is good at certain kinds of problems, um like data pipelines, transforming CSVs, uh processing logs, um, things like that. Um concurrent code. Uh if you have immutable data, then concurrency is easy. You don't need locks, um, financial calculations, uh, because um functional code is really easy to test, it's very auditable, there's no surprises. Um, web requests, that's why promises work the way they do. Um there's a lot of real-world wins using functional techniques. Netflix processes billions of events with functional streams. Banks love functional programming because bugs cost millions of dollars. Discord switched to Rust for the functional guarantees. Um but you don't need uh to use Lisp to get these powers. Uh you can use functional techniques in your own code right now. Um here's a couple things you can do. Um parse. Don't validate. Validate email once, create an email type, trust it everywhere. Instead of throwing exceptions, returning success or failure, uh a Python example, uh return a tuple of success and uh value or an error. Uh make invalid states impossible to express. Don't use strings for status, use separate types. Um you can't accidentally ship an unpaid order if they're different types. Um pipeline your logic, chain operations together instead of nesting them, and then each function in the chain does just one thing.

Jim: 18:44

All right, I'm I'm still confused. Um I I I don't quite understand it. Can you walk us through an example of of just what you're talking about?

Wolf: 18:56

Absolutely. Um I'll I'll give you the classic example um having to do with um regional sales data. Um so let me tell you the before and the after. Um so here's what I used to write, and and I'll bet you've seen this before, maybe even written it before. Um I'd start with an empty dictionary for the results, I'd loop through all the sales records inside that loop, check if the sale is completed. If it's not, skip it, you know, continue, whatever. If it is, check if we've seen this region before. If not, initialize that region in our dictionary to zero, then add that sales amount to the region total, and I probably need error handling in the case that the amount field is missing. By the time I'm done, I've got nested loops, multiple if statements, mutation everywhere, and good luck unit testing this without mocking. The entire database.

Jim: 20:02

That sounds like every reporting function I've ever written.

Wolf: 20:06

Alright. Let me tell you a better way. Here's the functional approach. Take the sales data, filter to just completed sales. Group by region. Sum the amounts for each group. Done. Each little thing that I said there is one function. Each of them takes input from the function before and passes it, passes the their output, their result, to the function that comes after. It would be one expression. It's easy to test. It's easy to prove. It's ridiculous.

Jim: 21:12

Is it actually shorter and cleaner when you do that?

Wolf: 21:15

It is. It's about half the lines. But more importantly, in the functional version, each step it either works or it doesn't. In the imperative version, I might forget to initialize a region or accidentally double count or mutate the wrong variable. The functional version makes those bugs impossible. To test the imperative version, I need to set up the entire environment, track mutations, and verify intermediate states. To test the functional version, first I test the filter. Given sales, does it return only one com only completed ones? Test the grouping. Given sales, does it group by region correctly? Test the sum. Given amounts, does it add correctly? Each test is trivial because each function does one thing.

SPEAKER_02: 22:06

So it's cleaner and it's more testable.

Wolf: 22:09

Exactly. And here's the kicker. If tomorrow they say we also need to exclude refunds, where do I change the code? In the functional version, I'd add one more filter step. In the imperative version, I'm adding another if statement somewhere in those nested loops and hoping to God I put it in the right place.

Jim: 22:45

I think that would be kind of crazy. But when I write SQL, I sort of write it in a functional style, I think. I start off with what is our basic data. So there's my select, right? I I select a bunch of data out of a table. And then they they want to filter it. So I mean I have a where clause, but sometimes I need to basically wrap that uh select statement in another select. So my first one is a sub-select, a subquery. So in that outer query, I'm filtering or grouping or doing something, and then often I'll wrap it again with another uh select. So so I've got like two subqueries, and that's kind of doing what you just said. When I need to make a change, there's a certain place in that chain of uh queries uh that I make my change. So I I guess I'm doing some functional programming in SQL. Does does that make sense?

Wolf: 23:44

It uh it kind of feels like you are. Like my SQL is not nearly as good as yours. In mine, instead of making a more complicated um query, I'd probably make temp tables and have little queries that moved data from one table to the next. Um, but I think uh you doing it in the query is is exactly the same as me doing temp tables.

Jim: 24:15

Uh I think, I believe that by me doing it all with in nested queries, uh, it allows the query optimizer to do its thing. And it might flatten some of those nested queries. It does all kinds of magic. I I use Postgres and it's got a pretty good query optimizer, and I think my solution would be faster than yours.

Wolf: 24:36

Oh, I'm sure it would be. I definitely trust your SQL more than I trust my SQL. I mean, I'm okay. I can solve the problem, but uh you you've been at it a lot longer than me. That's my feeling. Um let me tell you uh where you can start, what you should do to understand this stuff better. You don't need to learn Haskell, you don't need to understand category theory, you don't need to read academic papers. Here's what you should do: use map, filter, reduce, uh, instead of loops, when it's cleaner. Um I'm absolutely not telling you to make your code less understandable, because to me, clear code is one of the highest goals. Uh but if you can use one of these functions instead of a loop, and it is the right thing, you should. Um if you're using Rust, uh you know about result. If you're not using Rust, see if you can use Rust's result pattern in your language. Um take a class that you use, um I don't know what language you're using, but it probably has something that is a class or is like a class. Take a class like that and make it immutable. Um for instance, um, if you're working with geometry, um why are points mutable? Why should I be able to change the X in a point? Um why not, when I move a point, just return a new point? Points are probably small anyway, that seems easy to me. Um try writing some pure functions for your business logic.

SPEAKER_02: 26:38

Um Yeah? Okay, what if I do want to go deeper?

Wolf: 26:46

Well, if if you want to uh l use a language that actually has functional features, uh even if you're not trying to write a a real program, you just want to play around, uh give Rust a try or Elixir. Um they're both pretty pretty friendly. Um or just keep writing Python, JavaScript, Java, Perl, but in a functional style. Um I think the takeaway here is that whatever you're doing now, almost certainly, you're already functional. Every time you write a list comprehension in Python, every time you use option in Rust, every time you chain promises in JavaScript, you're doing functional programming. Uh the question isn't whether to learn functional programming, you're already using it. The question is, what if you did it on purpose? Um, your homework. Find one ugly function full of mutations and exceptions. Try making it pure. Return results instead of throwing. Make the data immutable. I guarantee it'll be easier to test and harder to break.

SPEAKER_02: 28:08

So that's it. I'm already a functional programmer. Yes, you are. Now go be a better one. Challenge accepted.

Jim: 28:18

All right, that was an excellent uh little uh tutorial on uh functional programming. Uh, as always, you can contact us at feedback at runtimearguments.fm. Uh more info is in the show notes, including uh additional ways to contact us. And as always, thank you so much for listening.

Wolf: 28:40

Hi, everybody.