Cloud-Native Patterns with Cornelia Davis

Developing cloud-based applications requires unique patterns and practices that make them suitable for modern cloud platforms. Host Tim Berglund catches up with Cornelia Davis, author of Cloud-Native Patterns and VP of Technology at Pivotal, on what cloud-native patterns are, the example code she created, her latest book, and how she wrote the book for the customers she interacts with on a daily basis. 

EPISODE LINKS

• Get 40% off Cloud Native Patterns with the code podcon19
• Join the Confluent Community Slack
• Fully managed Apache Kafka as a service! Try free.

SEASON 2
Hosted by Tim Berglund, Adi Polak and Viktor Gamov
Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed
Music by Coastal Kites 
Artwork by Phil Vo 

•  🎧 Subscribe to Confluent Developer wherever you listen to podcasts. 
• ▶️ Subscribe on YouTube, and hit the 🔔 to catch new episodes.
• 👍 If you enjoyed this, please leave us a rating. 
• 🎧 Confluent also has a podcast for tech leaders: "Life Is But A Stream" hosted by our friend, Joseph Morais.

Transcript

Copy Transcript Copied

SPEAKER_01 0:00

Cornelia Davis is the VP of technology at Pivotal and a longtime industry veteran of the cloud and open source worlds. She joins me on the show to talk about her seminal new book, Cloud Native Patterns. I had a great time talking to Cornelia. You can check it out right now on today's episode of Streaming Audio, a podcast about Kafka, Confluent, and the cloud. And I'm delighted to be joined in the studio today, the virtual studio today, by Cornelia Davis. Cornelia, welcome to the show.

SPEAKER_00 0:40

Thank you, Tim. It's so good to be here.

SPEAKER_01 0:43

Cornelia, you are uh VP of technology at Pivotal. Um, I think it's fair to say, uh longtime open source and cloud uh industry person and author of a book, a book called Cloud Native Patterns.

SPEAKER_00 1:00

Yep, that's right.

SPEAKER_01 1:02

So those facts are true.

SPEAKER_00 1:03

Those facts are true. So yeah, I am I have been in the industry as a technologist. I'm a computer scientist, um, still still doing tech as a computer scientist. I will admit that in my day job, I don't get to cut code too too often anymore. Um, but I have been cutting code for the book, and that is true. I did publish a book earlier this year, um beginning of June, uh, with Manning as my publisher. It's called Cloud Native Patterns, and it's a book that I really wrote for my customers, the people that I've been working with in my capacity at Pivotal, which is at Pivotal as the VP of Technology, I'm in the enviable position of working kind of in emerging technology spaces and helping our customers and partners and even our own internal engineering kind of grok this emerging technology and start to build value from that. And so building cloud native applications, you know, building applications in this new cloud native way is certainly something that's been emerging over the course of the last, you know, five plus years. And so I wrote the book uh to reflect on all of the things that I've been learning along with my partners and colleagues and customers.

SPEAKER_01 2:17

Awesome. Uh and I appreciate your use of the word still when it comes to doing tech. Uh for those of us who have been in the business for a few decades, there is a sense of of, you know, even like you know, you're VP of technology at Pivotal, right? This is not uh a life that has fallen into ruin. This is a great thing. But we always talk, we always say, you know, I still am hanging on, I get to write code, and it's this feeling of they haven't stopped me yet. And I 100% get that feeling.

SPEAKER_00 2:48

Yep. Oh, no question about it. I am I have every intention of doing this until I'm well into my 80s. Um you can't rip this out of my hands.

SPEAKER_01 2:58

You may have to keep writing books to do it. So my experience is I have to I have to write demos for presentations or tutorials online. I mean, uh that they have code in them.

SPEAKER_00 3:06

That's how I get it. Yep, exactly. They're not gonna stop me. Exactly.

SPEAKER_01 3:11

Yeah, so uh the the idea of cloud nativeness as a fairly recent phenomenon, I think you're you're dead on there. And sometimes I like to reflect on the fact that um, you know, I remember well I remember when S3, AWS S3 was released. That was the beginning. Um there had been managed uh hosting and things like that before. Uh there's Rackspace predated that, I'm pretty sure. So that idea was sort of floating out there, but then S3 happened, and it's like this, oh, that's interesting, this data storage service, quote unquote, in the cloud. And then EC2 came out, and we all like believed in managed hosting all of a sudden. It wasn't the first one, but like, oh, okay, this is a thing. And we told these stories about how, well, you know, during the holidays, your e-commerce site can scale up because you'll add uh cloud servers, and then you don't have to pay for them, so you'll just take them away, and you know, you won't pay for that infrastructure when you're not needing it. And that was the pitch. And I don't know about you, but the application architectures I was using at the time, yeah, they didn't do that, right? That was not a thing at all. Um right?

SPEAKER_00 4:19

Yeah, totally. I mean, that was I'm sitting here grinning from ear to ear, chuckling about how that was definitely very much a fallacy of like, okay, well, we'll just like use these other servers. But if it takes you three months to deploy something, and then you know, it's it just so so that is I like to say that's cloud, and I sometimes refer to cloud as the where you're computing, but cloud native is how you're computing. And what you just described was until we change the how, we can't even really take full advantage of the the changes in where.

SPEAKER_01 4:56

Yeah, like the the economic case for it in terms of resource utilization and you know, actual cost of compute and storage. We should be able to amortize that better and get that that you know, that that's all true, but not without uh building our applications in a different way, which I think is what your book is all about. So it was it was not, and they they weren't lying, you know, when we were talking about that 12, 13 years ago, it was more of like a promissory note, it was a vision, you know, and we're we're sort of now getting there uh some years later.

SPEAKER_00 5:29

Yeah, no question. And what's interesting is that um I sometimes talk about virtualization, because virtualization plays an important part in this whole cloud and even cloud native thing, is that VMware, you know, they were the ones who brought virtualization to the industry. But that what Amazon did though, even though it didn't take us all the way to cloud native, what they did was they they laid a really important foundation, which was they added the as a service. So even though our applications weren't built to necessarily take advantage of this dynamicism that you get when you have the as a service part, um the the it's a chicken and an egg. The as a service is what allowed us to start thinking of, well, gosh, if we can do virtual machines as a service, and we it doesn't take us six months to rack and stack and hardware, you know, procure rack stack hardware, get operating systems installed. If I had a model where I could get these things more quickly, then boy, I I could I need to change the way I write software so that I can take advantage of this ability to get things more quickly. So even though it wasn't all the way there, it was a huge enabler to where we are today.

SPEAKER_01 6:49

Indeed. Um and we're using, well, your book has uh the words cloud native in the title, and I I know you have a fairly precise definition of that. I think, like in marketing copy, cloud native means uh a computer program I want you to like. It's definitely good, whatever it means, but you're a heck of a lot more precise than that. So tell me what you mean by cloud native.

SPEAKER_00 7:15

Yeah.

SPEAKER_01 7:16

We're sort of getting it.

SPEAKER_00 7:17

So I'm I'm gonna constrain my definition of cloud native a little bit more tightly, and I'm gonna talk about cloud native software. So there's also cloud native processes, cloud native, you know, ways of working. So I'm sure most of your listeners are probably familiar with 12factor.net. And those aren't only architectural patterns for software. There's actually some practices in there, like the way that you structure your repositories and and and and those types of things. And another word that we might use for some of those process elements would be, and organizational structures would be DevOps and so on. And so I actually find that defining cloud native in the broader sense is a little trickier. Um, can be done, and the Cloud Native Computing Foundation has their definition and so on. But given that what I I'm gonna constrain the problem a little bit and say, I'm gonna talk about and define what I mean by cloud native software. So the software that is gonna be able to take advantage of this elasticity up in the cloud, for example. And there's really two elements to it. And what I always say is cloud native software is that which is highly distributed, extraordinarily distributed, far, far more than it was in the past. It's not just three tiers and a couple of nodes, but we're orders of magnitude more distributed than we were in the past. And it's experiencing constant change. And in fact, the subtitle of my book is designing change-tolerant software. So it's so important that we create these design patterns so that we can be adaptable to change, both to change that's happening because we release software more frequently, all the time. We're releasing many times a day as opposed to once every 18 months. But also that the infrastructure that this software is running on is experiencing constant change. And so we need to be adaptable to that. So highly distributed, constantly changing.

SPEAKER_01 9:16

Right. And so we're changing the definition of the program or programs that comprise our system. We're we're actually deploying new versions of them. Plus things come and go. It would be a you know, things things things will go down in the cloud. Um, you'll you'll intentionally scale out or scale back, all that stuff happens. And you're saying cloud native is software designed to is designed to optimize uh for both of those states of affairs. Exactly. I love that definition. Uh that is precise and um what's the word I'm looking for? It's objective, right? Like you can tell uh if those things are there. Your book is um, I think, targeted at uh developers and architects, which are certainly two of my favorite kinds of people. Uh so that's uh seems like a good thing to me. But in chapter two, you get into production concerns like right up front.

SPEAKER_00 10:12

Um why is that because honestly, in order to leverage these patterns, and sometimes, by the way, leveraging some of the cloud native patterns takes a little bit of effort. Um, and certainly for people like me who have the gray hair to show for it, um, we have to relearn things. We have to unlearn patterns that we've been employing sometimes for decades and re-relearn new ones. And we really need to understand why. We need to understand um what are the things that are motivating that? Why should I not use sticky sessions? Because my software is already using sticky sessions. You're expecting me to change something. You know, I'm not always building that new applications. So why do I have to change something that's working today? And oftentimes that comes back to these operational concerns. And it goes back to this notion of you know what, software has always got to be up and running, even if the infrastructure is changing out from under us. I spent a number of years at Documentum, Documentum content management software that was built originally 30 years ago, layered on top, layered architecture, layered on top of a database. If the database went down, we threw up our hands and said, not our fault. And you know what? That was okay. Nobody was fine. Yeah, nobody would hold us responsible if the database went down. Um, they would say, sure, it's not your fault. But today, if a an Amazon region goes down, um, and in fact, the first four words in my book are it's not Amazon's fault. And I talk about and I I launched the whole book with the story of this Amazon outage in September of 2015, where they um they had an outage that lasted five, six hours, depending on how you count, and all sorts of companies were down. IMDB was down, Nest was down. And famously, we all know that Netflix runs on Amazon. And when Ask, Amazon, when Netflix was asked, Netflix shrugged their shoulders and said, Yeah, we experienced a brief availability blip. I'm quoting that. That's what they have in a blog post. A brief availability blip. And so there's a perfect exemplar of they, and by the way, it wasn't necessarily just that the developers had done some of the right patterns. They had all sorts of operational characteristics that they had exercised so that the developers knew what patterns made a difference and they were able to weather that storm because of that. So operations is absolutely essential. Developers and application architects have to have that empathy to know how to best apply these patterns.

SPEAKER_01 13:13

And I think one of the one of the fallacies about cloud nativeness is that it removes operational concerns from the world. Um it changes operational concerns, and I think it's fair to say reduces the investment that uh you have to make in addressing operational concerns, you know, because you you really have outsourced uh a lot of that to the cloud provider, but you're still operating cloud services and you're you're operating your program in the cloud.

SPEAKER_00 13:43

Yep, absolutely. And I like to talk about it. And you you just made a great point, which is that there's still operations happening. It's just that we've partitioned things differently than we used to. Um, in the traditional enterprise, we used to have application development, and then we had INO, infrastructure and operations. And so the infrastructure and the teams that were operating, not only the infrastructure, but most of the time operating the applications as well, were in this infrastructure and ops team. Now, in the repartitioning, what we've done is we've changed it so that the cloud providers, and by the way, in cloud providers, I would include somebody who is managing an on-prem cloud or an on-prem platform instantiation, is that they are doing operations for their software. Their software happens to be AWS or Azure or GCP or Pivotal Cloud Foundry on-prem. That's their product, and they're doing operations for that. Then the application teams leveraging those services, they're doing development of their applications, but they're also taking on the operations of their applications. So it's it's there's no longer a single operations team that does infrastructure and application operations. We've actually partitioned things in a much more sensible way now.

SPEAKER_01 15:14

So you have uh smaller teams that are a little bit more directly attuned to the operational needs of their application and their attendant cloud services and everything. It kind of it sounds like in a roundabout way we are describing DevOps professionals, and that's really the context that gave rise to that whole discipline, right?

SPEAKER_00 15:34

Yeah, absolutely, absolutely. And in fact, I just uh just spent some time at the DevOps Enterprise Summit recently. Um and uh that particular conference is very, very much focused on the people and process side as well. Um, there is a little bit of a technological um theme running through that. That's usually where I play because I love the technology part. But yeah, it it requires a change in, like I said, it's change in partitioning. And so we've partitioned our stacks differently. So we've got a platform now, and then we've got apps on top of it. But you'll notice in that description that we're also repartitioning our organizational structures.

SPEAKER_01 16:14

Uh yes, we most certainly are. And you just said uh something that got my attention. You said there's a platform. Tell us a well, I would ask you for a definition of what you mean by platform and tell us where that fits into all this.

SPEAKER_00 16:28

So I think that there's a number of elements uh that I think are important in what I mean by platform. It goes back to what I said earlier in that there is some type of a service provider where some set of customers can go and leverage some services. You know, it's the as a service part. It's that analogy that I made from VMware was providing infrastructure, you know, virtualized infrastructure, but it Amazon did kind of the next major thing in that they added the at-a-service on top of that. So it is about having a contract that some set of individuals can go and leverage some capability through a very formalized contract. And by the way, by contract, I don't just mean API. As a technologist, we immediately think API. Oh, that means that I'm using the AWS API or CLI or the GCP CLI or the Azure CLI, and I'm requesting a machine, or I'm requesting a Kubernetes cluster, or I'm requesting an instance of a relational database. But it's also in the way that those teams, the contract also is in the way that those teams work together. So for example, um, if I need uh some database capability, I don't call up AWS and say, hey, I have this new fangled, weird, bespoke, you know, open source database that I found. I'd like you to provide me that as a service. There isn't a phone number I can call to make that request. And if there was, they would say, here's our catalog. You know, here's the set of offerings that we have, here's my product. So part of that is the con that that contract. It's knowing that there's a catalog, this is what I get to use, and then of course I use APIs to request it. And then the other part of platform, of course, is that the platform provides things that I need to bring some digital product to my consumer that I don't want to have to build myself, or I don't want to have to operate myself. So a database is a great example of I don't want to have to operate that myself. But also things that I might not want to build myself is that there's an important pattern in these cloud native systems, which is a reconciliation loop, which is to say, let me compare actual state to desired state and bring them in alignment. The most well-known reconciliation loop, I would say, in the industry right now is that which Kubernetes brings in terms of container orchestration.

SPEAKER_01 19:06

Exactly what I was thinking. You've got the configuration and then you make sure it stays true, even though like you said, the way the software is deployed is constantly changing. A node might go away. Hey, it doesn't matter. Make go make it true.

SPEAKER_00 19:19

Exactly. And so that's something that before the days of Kubernetes, if I'm not using a platform that offers Kubernetes, then I have to build that reconciliation loop myself. And so that's another thing that platform does is it provides services that I can leverage, like database services, but it also implements some of the very cloud native patterns that I talk about in the book. So I talk about a whole host of patterns in the book, but I'm constantly emphasizing that while as a developer, you need to understand all of them, and you need to understand when to apply them and how to apply them and all that stuff. You don't necessarily need to build them yourself. You can either get them from a platform, you can get them from a framework like the Spring framework, but you have to understand them to be able to leverage all those things properly.

SPEAKER_01 20:12

Yes, you do. And you have to not just understand them, but understand um, and this is a point you make, uh when to apply them. Like in what context are they applicable? Um these aren't these aren't just Legos that you can stick on anywhere. These are things that solve particular problems, and knowing that uh you know set of conditions in which the pattern applies is almost more important than knowing how to implement the pattern, because you might have a framework that's doing that for you.

SPEAKER_00 20:42

Exactly. And just as importantly, downstream ramifications. So let me give you a concrete example on that one. So one of the really simple patterns, um, and it's so simple, I think, that oftentimes people don't even really talk about it that much, but it's so important is the retry pattern. And so that the analogy that I like to make is that Tim, when you and I are browsing the web, when we get to a web page and we click on a hyperlink, sometimes you know the little icon spins and it doesn't render the page. And we we don't just give up and go home. We hit the stop button or we hit the refresh button, right?

unknown 21:25

Yep.

SPEAKER_00 21:26

So that's a retry. And so when it comes to these distributed systems, we know that there's you know the network always being up as a complete fallacy. So we know that there's sometimes network outages or you know, something else, even if it's just a split second, a packet can be lost, whatever the case may be. Um, when we make a request from one service to another service in this network of distributed systems or distributed services, we might not hear back. And so it's really simple. You you do need to understand that, okay, yeah, in some contexts, I'm gonna do a retry. Right. Now, here it's really important to understand the context before we even get down to the downstream ramifications. So, Tim, when you click on, when you're browsing Amazon.com and you click on, you've done a search and you've got a list of things, and you click on one of those things and it doesn't render. What do you do?

SPEAKER_01 22:20

Well, uh my actual behavior is I will probably have opened that in a new tab. And my way of refreshing is command L enter. You know, I go back into the address and hit enter again. So that's going to issue that get a second time. Yeah. It's like refreshing. That's just my own weird funny, I didn't even think of that until you asked me the question.

SPEAKER_00 22:42

Oh, yeah. That is relatively unique.

SPEAKER_01 22:45

Yeah, yeah. But that's what happens. And so I am I am resubmitting the get.

SPEAKER_00 22:51

Exactly. You're doing a retry. Now, when you hit the final button to purchase the product, to to close, you've put in your credit card information and you say submit order and nothing happens. What do you do?

SPEAKER_01 23:08

Okay. Now that is a little scarier. Um what do I do? I will probably in that case, I'll probably go back to my cart somehow and try to check out again.

SPEAKER_00 23:21

Exactly. So you go back to your cart, and if your cart's empty Well then I yeah, then I'm like, oh, okay, it it worked.

SPEAKER_01 23:28

I didn't get the D page, and I'll go. There'll be an asynchronous notification later, an email. I'll look for that if I think about it, you know.

SPEAKER_00 23:35

Exactly. So you have an algorithm that you use to decide whether the retry is appropriate.

SPEAKER_01 23:42

Yes.

SPEAKER_00 23:43

So you don't just hit retry.

SPEAKER_01 23:45

I don't. I I actually do have a little process I go through. No, I thank you for helping me explore that. I didn't know.

SPEAKER_00 23:52

Exactly. Exactly. And so so in order to apply a retry and to do it in a way that you're not going to break some systems, is now remember now this is the computer. So the one service is deciding whether it's going to retry a request, is that you need to know things like is this request this request that I'm making itempotent? So if I do it a second time, is it going to yield the same result as if the first one went through and I just didn't know it?

SPEAKER_01 24:23

Right.

SPEAKER_00 24:25

And so that's that's you have to understand that context before you can apply the retry.

SPEAKER_01 24:32

Right. And ordering is it's certainly possible to build a website that that makes order requests itempotent, but please, please sell me this thing doesn't feel itempotent by default, right? You can carelessly retry that and and get uh you know two 15 count packs of kind bars when you only wanted one and you know you don't want that.

SPEAKER_00 24:55

Exactly. And we've all been on those websites where there's a little little bit of text that sits next to the submit button that says, please do not hit this a second time.

SPEAKER_01 25:04

Fills me with confidence and like I want you to store my credit card number and we should hang out.

SPEAKER_00 25:12

Yeah, yep, exactly, exactly. Okay, so that's like understanding the context. Now let's talk about downstream ramifications. So if I am doing retries, one of the things that can happen is that I, as a client, I being a piece of software, I'm a piece of client software, and I've done a request and I don't hear anything back. There's a timeout, and I'm gonna go ahead and do the retry. Now that service that I'm retrying might have dozens or hundreds of different clients and thousands of different instances of those clients. And so if we have something like a momentary network outage, like something that's on the order of five seconds or 10 seconds or a minute, when we issue those retries, if we do it quite naively, then what can happen when the network comes back is that we have a huge number of retries that have been essentially queued up. So now what had been, I like to think of it as kind of a traffic scenario, is that I'm on a four-lane highway, I've got just the right amount of traffic on that highway to sustain the load. We're all going 60 miles an hour. There's an accident, it closes two lanes. When those lanes reopen, there's just the volume has continued down that road. People didn't stop trying to merge onto the freeway. So now the highway is completely overloaded. And you run the risk of doing that with retries. And you can end up with a retry storm, which then overloads this the system that you're trying to retry, the service that you're trying to retry. Critically important that you understand as a developer not only when is it okay for me to do retries, but then how do I best do retries? How am I gonna be what I call a kind client when it comes to retries?

SPEAKER_01 27:19

There you go. And that's that's um probably why you were talking about framework before. Um this is a this is a developer trap, right? I always speak of this as um you know, we should look to the advice of the great software architect, Admiral Akbar, um, who warned us that it's a trap. Uh something like retry sounds trivial, right? It's a loop and maybe a count and um a delay or something, and I can do that, you know. But there really are and it's not look, we you know, it's been eight minutes or something like that. We've been talking about it and we're we've explained it. It's not that difficult of a concept, but uh that tricks you into thinking as a developer, oh, let me just code that retry, I'll do that, it's no problem. Um but but there's all this subtlety that just in a few minutes of exploring, I don't know, maybe the simplest pattern in the book, um, you've shown a few ways you can get tripped up in that. Yep. Um and that number one tells you, I think reinforces your point that you have to understand the downstream ramifications, you have to understand the context that makes the pattern applicable. And maybe you shouldn't write the code for it. If there's some kind of solution that someone else has done, you know, because retry a thing is to some degree uh the same problem. And you don't, you know, your solution to that is probably not better than X framework's solution to that.

SPEAKER_00 28:47

Yeah, exactly. And so for example, uh the Spring Framework has um a part of it called Spring Retry, which does exactly that. And so in the book, I actually uh go through an example where I first implement very naive retries and I do a simulation. This was the part of the book that took me the longest to get the simulation just right, um, is that I have an example that runs through the entire book, and I simulated a network outage. Um, and I implemented it with the super naive retries that I implemented myself. Just a loop that says, until I hear back, keep trying.

SPEAKER_01 29:24

Nice. No uh no give up, no panic, no uh nothing, nothing.

SPEAKER_00 29:28

Just just keep trying, just keep trying. Um and then I I took I simulated a like two or three minute network outage, and then I brought it back. And um the short of it is, and then I sat there and watched and waited for the system to recover. And um I think maybe in that case it never recovered or it took half an hour to recover. Then I implemented spring retry, and I started to implement some of the basics, some of the defaults, like okay, you're not gonna retry more than three times. After three times, you stop. Um, so that it it you stop putting more cars on the freeway. You put traffic light there. Those are exponential, yeah, exponential back offs, those types of things. Um and it simulated the same network outage for the same amount of time, and it recovered in, I don't know, a few minutes as opposed to in half an hour. So it changed dramatically. Now there's an another important part. What we're talking about here when we talk about retries, I actually talk about three fundamental kind of parts of the mental model for cloud native software. It's cloud native services, so sometimes we refer to those as microservices, although I hate the micro part. The size isn't really what's important. Um, cloud native interactions and cloud native data. And what we've been talking about here is a cloud native interaction. And we just like we need redundancy of our services, we need redundancy of interactions. That's what the retry is. Now there's two endpoints to that interaction. There's the initiating endpoint, which is what we've been talking about, and I talked about being a kind client. But if I'm on the receiving end of that interaction, of that request, I can't simply depend. I can't naively say, well, okay, I'm just going to assume that all of my clients know how to be kind clients. I also need to protect myself. And that's where another pretty well-known cloud native pattern comes in, which is circuit breakers. So circuit breakers are the case where we say, you know what? If all hell breaks loose and I am overwhelmed with responses, I'm not going to let myself melt down. And so you put the circuit breaker on the service side of that interaction. And then I actually took the simulation to that third step and said, okay, now when I add circuit breakers and I and I took a look at that combination, I can get it down to where the network comes back and it almost instantaneously recovers itself. The system's healthy.

SPEAKER_01 32:12

Really? With circuit breakers and uh thoughtful spring retries, it's it's right. Bounces right back. That is awesome. Yep. Um now in both of those cases, there are requests that will simply fail, and there will be error indications that return to the requesting client, right? Yeah. So that's important. And in the first one, um that well, that same thing is gonna I mean the first one, the the naive one that took forever to to come back up or didn't come back up. Um you still get that. It's just that the system isn't now making decisions about uh sort of on principle about which requests need to be denied. There are just gonna be things that fail because thread pools will fill up, and you know, basically you're you're DOSing some component of that system with the naive retries, and and it will fail in an unpredictable way, unpredictable way, and its manner of recovery will be a thing that you work out over the next three hours in a highly stressed way and write the write the um uh outage report blog post about the next day.

SPEAKER_00 33:18

And it's funny you should say that because three hours might not be enough. That um that outage that I talk about at the beginning of the book, what was behind that was actually a um it was a denial of services. It was an inadvertent denial of service. It was a change. They had deployed a new version of RDS. And that new version of RDS, when when everything was fine, when this request volume was fine, it was still running within parameters. They had a momentary network outage. When it came back, it got overloaded. RDS melted down, and then everything else melted down.

unknown 33:52

Uh-huh.

SPEAKER_01 33:53

Okay. So it was a DOS own goal kind of. Yep. Um changing gears, you talk about event-driven microservices. And I I share your antipathy for the term. I I I've kind of that part of me, I think, has died inside. I used to feel I used to feel a part of me dying inside every time I said microservices, just because it's uh it's you know so so ubiquitous in writing and on conference stages, and it just feels like this overhyped technology trend. I mean, in reality, it's an amazingly good idea for just about every reason, with a few notable downsides to it. But I'm with you. I don't like saying the word. I think the little piece of me I felt dying every time I said it is completely dead now, so I don't feel it anymore. But very good. You have a chapter, this is chapter four in your book on event-driven microservices. Uh, and it's subtitled, It's not just request response. Let's just say, as a Kafka guy, that got my attention. Tell me what that's all about.

SPEAKER_00 34:57

Yep. Yep. So a lot of these patterns, and in fact, just the ones that we've been chatting about, the main retry pattern and then kind retries and circuit breakers and all of that stuff, um, already reflects our bias. And by the way, uh, I think that this bias comes from all the way back in the way that most of us are taught computer science when we first start. Most of us are taught computer science. Our first languages are imperative languages. So they are languages where we learn about variables and we learn about changing the value of those variables and updating the values. The first four-line program that I wrote was in BASIC, where I set X to a value. I printed the value of X, I incremented it by one, and then I did a go-to back to an earlier line number, and it just counted on the screen.

SPEAKER_01 35:50

And you never looked back.

SPEAKER_00 35:51

Yep. It literally, it is that much of a story. I wrote that program and I was like, okay, that was really cool. And I have been in computing ever since. Um so yes, it was that, it was a life-changing moment for me. Um, but that's what we learned was we learned control loops and those types of things. And so what that is, is reflecting kind of, we've been trained to think front to back, linear, you know, like from the start to the end. And that's what request response is. And so when we talk about microservices in general or we talk about any of these architectures, we almost always already come in with this, like, we don't even question this idea that we're making requests and getting back responses. And um when I when I started writing the book, I was already, and I I come from that background, although I will say that I I went back to graduate school for a while and I became a functional programmer. Um and uh so I've and I've all long wanted to get back to functional programming in my career after I left school, went back to industry, became kind of an imperative programmer again. Uh I'm so delighted now because this is something I riff on a lot these days is that these highly distributed, complex, you know, constantly changing systems require that we do things in more of a functional style. It's the only way we can achieve resilience.

SPEAKER_01 37:19

Because if just I mean just a thought on that. It's imperative, imperative programming is fine, right? There's all kinds of things it's good for. It's just that that kind of synchronous method call, hey, I need a thing, uh, will you go make it and give it back to me? Okay, thank you. You know, that that sort of mindset works fine when the the call is literally implemented by a hardware stack in the processor and machine instructions optimized for making that work very quickly. And like if those don't work, if if call and return instructions don't work, you have much bigger problems to worry about.

SPEAKER_00 37:56

Yep.

SPEAKER_01 37:56

Um, and that's nothing like a distributed system. Okay, the reliability of the network and whether that other thing is there and what it's called and all that, you know, all these, and you have patterns to deal with these things in the synchronous distributed world. Um it's it's not a call instruction anymore. And it's not just that it takes longer, it's that it's a radically different kind of thing and its failure characteristics are absolutely nothing like a call instruction. And so that you know, the the relative faithfulness of the imperative paradigm in a non-distributed system ends up betraying us. And like you said, you know, we now, if we if we had all these functional tools, if that's how we grew up, that's how you thought, oh boy, it'd be so much easier. So you were you were going there.

SPEAKER_00 38:39

Yeah, absolutely. And in fact, a lot of the patterns, quite frankly, a lot of the patterns in my book are these these are the like the stopgap things. These are the things that we're using to take care of these failure scenarios that didn't exist when we were just doing machine instructions on a single processor and a single thread.

SPEAKER_01 38:57

Yeah, it was a good life. It really was a good life. And sometimes I want to go back to it, but not how either of us earns a living and not how the world works anymore.

SPEAKER_00 39:06

Yeah. And so so I guess to some extent, um, because I had this background in you know functional thinking and also just realistically seeing how the industry is starting to wake up to these additional challenges and potentially new and novel ways of dealing with the challenges of distributed systems. Of course, things like, you know, we've been doing messaging for a long time, um, and some of those patterns were in place, but I had a natural interest there and had been starting to work in some of those areas myself. And so in the second part of the book, the first part is kind of the, you know, sets the context, like we talked about earlier. Why do we care about operations? What's the value of platform, those types of things? And then chapters four through 12 are where the patterns are, and that's where you'll find code and things like that. Um, I said right from the beginning when I was working with my editor, before I even jump into any more of the other patterns, I want to challenge the readers right from the get-go to say, hey, I want to challenge your bias toward request response. And I'm not going to take you all the way there. And I don't go all the way to event sourcing, where we talk about immutability and the event, you know, log being the single source of truth and some of those patterns. I do that later in the book. I don't go all the way there because I think I thought that that would be a little bit too far too fast. But I started with this notion of, hey, I want to challenge your thinking and I want to think about turning request response on its head. And I think of request response as kind of a, like I said, a linear, like I want to control every step. I'm going to go through this kind of linearly in time. A request comes into, let's say, the Netflix home page. And in order to render that page, I now that I'm making the request need to go down and get more information from this subsystem and this subsystem, and this subsystem is actually going to make further downstream requests. And so it fans out to these hundred request response things going down through the hierarchy, and then it comes back and we can render the page. And what I like to think of is let's turn that on its head and say, instead, let's think of what were these downstream kind of leaves of this call, this request response hierarchy, and let's start at the leaves and push back toward the root at the point that events happen. So that you can now start thinking of it from this perspective of, okay, well, these things are going to be happening. Somebody makes a change, they star a movie, or they um they start following somebody else. Those things can actually proactively propagate through the system so that when the request does come from, and by the way, this doesn't mean that you never do request response. There's still places for request response. Making a request from a rich uh web browser, you know, from JavaScript in a web browser to some back end system still makes all the sense in the world. Even if it's asynchronous, it's still a request.

SPEAKER_01 42:23

Right. Even if there's asynchronous things in the front end and asynchronous things in the back end, my experience of pretty much every app and website I'm going to use is synchronous. Yep. I I do things and I wait.

unknown 42:36

Yeah.

SPEAKER_00 42:36

Yep, exactly. Exactly.

SPEAKER_01 42:38

That's fine, but everything else, not not so much.

SPEAKER_00 42:40

Yep. And so now that request comes in from the website, it hits one one node, and that node already is has the information, has the context that it needs to generate that response without making a whole bunch of downstream requests because you did some of those requests in the opposite direction ahead of time.

SPEAKER_01 43:00

There you go. Another thing, um there's so much more in this book I want to talk about. We're we're getting close to time, and so I'm gonna have to just deal with the fact that we can't. Um but you said some things there that got me thinking, you know, there's request response and then there's the um you know the event-driven thing. I guess another thing that goes on with in a request response system is that the requests are ephemeral, right? Maybe you log them, but uh you you log something, certainly you log something about them, probably a lot of things about them, but uh what went out in the request and what came back, it's it's gone. It's like speaking words as opposed to writing. You know, the words you say them and they're just in the air for a moment and then they they go away. Um in your last chapter, it seems like you come to that, the cloud native data chapter, you uh maybe in an indirect way address that problem. So tell us what you're going for there.

SPEAKER_00 43:57

Yeah, so the way that you just described the Events is exactly the way we've been doing messaging for the last several decades. And in fact, if you look at those messaging protocols that were in things like JMS, they had you had things like cues and topics. And there were semantics around a queue, for example. As soon as somebody consumed that message, any one of the subscribers consumed it, poof, it was gone. That was part of the protocol. And what ended up happening, I'm going to go all the way back and geek out all the way back to this imperative programming thing. What you just did in doing that type of a message passing protocol is you just did the equivalent of a variable assignment. You assigned the variable, but now the assignment is done, and that machine instruction that created that, you know, reg that caused that register to get updated, it's gone. It's not in a call stack anywhere. It doesn't exist. We don't know anymore what caused that ver that that register to have that value. That's what messaging was. And you know, Tim, know this far better than I do, being at Confluent, is that what we've done and the protocol. Now, sure, Kafka can be used just in a simple messaging protocol like that. Absolutely, people do it. And people do it. I'm I'm certain of that. But Kafka can also be used in a different protocol, in a protocol where you don't assume that the message is ephemeral, where you assume that the message itself is the valuable piece of data. That is there's an everything else is just, even as we just talked about a moment ago, everything else is just a downstream consequence of that. But the actual piece of data is the fact that that event happened. So we don't throw away those events, we save them, and that's where I make the distinction. And you can please correct me if I'm using the wrong terms. But event driven is what I talk about in chapter four. And I don't talk about persisting those messages, those events, or anything like that. I simply talk about instead of flowing from root to leaf, flow from leaf to root. But then in chapter 12, I come back and say, look, if we actually preserve this these events as an event log in something like Kafka, then that becomes the source of truth. And that changes us from being a more imperative style, mutable state type of a system into being a much more powerful system in terms of not only distributed system, but future-proofing your ability to do those things. And so that's where I come back to it in chapter 12 and say, okay, now we learned all these patterns, which by the way are valuable even in these event-driven systems. Networks still go out, you still have to maybe deal with retries, even if you're going leaf to root. Um, but now you have this new way of thinking about it. You understand these patterns, and now you can go back and kind of close the the circle on that.

SPEAKER_01 47:18

Absolutely. And that um fully developed uh data pattern uh that you talk about that that you were just talking about now, and that's that's in chapter 12. Um, we still call that event driven. Um and I think the reason we get away with that is because there's when when when confluent is speaking, you know, uh there's this universal assumption of Kafka. And so, you know, yeah, things are logged. It's just that's a given. We're all about that. And and I think at the without that as a given, um, that is a very useful distinction for you to make. And I'm just I'm I'm just really thinking, kind of thinking out loud here. I think the reason we don't make that distinction is just because there's always a log, you know, it's coffee.

SPEAKER_00 48:02

There is always always a log. And you know, interestingly, if you take a look at the code in chapter four, I don't even have a message queue in there. I'm actually doing the events, I'm propagating the events through my little sample application using HTTP request response. I'm just going leaf to root instead of root to leaf. And I'm doing fire and forget. So there's there's no message queue, there's no log at all. I introduce that later on.

SPEAKER_01 48:27

I love it. All right.

SPEAKER_00 48:28

Yeah. Yeah. And I should say um also that by the way, in chapter 12, the coverage that I give on this topic is so basic and introductory. And probably I think in the last chapter, the last paragraph of the book, I say, if you want to really learn this stuff, go read Martin Clettman's book.

SPEAKER_01 48:50

And that's so true about it. Turns out everything I believe about stream processing. Sometimes I'll think I got the idea from somebody else, and then I'll oh no, he got it from Martin also. So that's where pretty much all the good ideas come from. But hey, you know what? What if we had a world without a Martin Klappman? Uh that would be no good at all.

SPEAKER_00 49:14

No, definitely not.

SPEAKER_01 49:16

No. Um we were talking before we started recording. We were trying to remember how we met. Um what what was that?

SPEAKER_00 49:25

Yeah, so so um see, you won't know this part of the story because actually uh I feel like I've known you for a long time. And it's because I spent several weeks with you. You don't remember this because it wasn't synchronous. Oh I spent several weeks with you. I learned git from you on Safari Books Online. Really? Cannot remember who you did that video series with, remind me.

SPEAKER_01 49:54

Matthew McCullough.

SPEAKER_00 49:56

Matthew McCullough, exactly. So, of course, I had done you know basic git push and git pull and all that stuff, but I really learned git from you. So thank you very much.

SPEAKER_01 50:06

My pleasure, my pleasure. That was uh uh fun. That was uh did a couple of those. That was actually my first O'Reilly training video, was Git with Matthew. And um that was uh I still that was a long time ago. It's been a while since I have earned my living by teaching people git, but when somebody runs into a problem, it I feel so good because I'm still the one that can uh pull them out of a jam, you know, even if we have to break out uh reflog and uh go for the go for the throat. Yeah. We can do it.

SPEAKER_00 50:38

Yep. And you know, the funny thing is that I I have a son, a 24-year-old son, who just started his career a few months ago as a software engineer. Happy mama. Isn't it great? Yes, it is. It's so awesome. And I get the inevitable git questions. Hey mom, I've got a git problem. Or, oh my god, I was in merge hell today.

SPEAKER_01 50:59

That's so funny. Because I I have a son about that age who works as a web developer and he uses Git, and he never asks me any Git questions. I wonder if he's either he's just real good at it or he's got somebody else to. I I'm kind of I'd I'd love that. It'd be very endearing if Zach asked me a Git question.

SPEAKER_00 51:14

It's pretty fun.

SPEAKER_01 51:16

I need to talk to him about that.

SPEAKER_00 51:17

Yeah.

SPEAKER_01 51:18

My guest today has been Cornelia Davis. Cornelia, thanks for being a part of Streaming Audio.

SPEAKER_00 51:23

Tim, it has been such an honor and a delight to be here. Thank you so much.

SPEAKER_01 51:27

And there you have it. Before I go, I want to tell you that we have a pretty cool new offer to help you get started with Confluent Cloud without you having to pay for anything. If you're a new user and you go through the regular sign-up process and start using Confluent Cloud, your first $50 of usage per month are free. This will last for the first three months after you sign up. So that's $50 per month of serverless Kafka for three months at no cost to you. So go to the sign-up link in the show notes. I don't want to read you the URL, and sign up now. I think the only thing I could really do more is write your code for you. And I think we can both agree that's too much to ask. So check it out and hey, let us know how you like it. Anyway, as always, I hope this podcast was helpful to you. If you want to discuss it or ask a question, you can reach out to us on Twitter at Confluent Inc. or reach out to me at TLberglund. That's T L B E R G L U N D. Or you can hit us up in Community Slack. There's a sign-up link for that in the show notes as well. And while you're at it, please subscribe to our YouTube channel and to this podcast wherever fine podcasts are sold. And if you subscribe through iTunes, be sure to leave us a review there. That helps other people discover the podcast, which is a good thing. Thanks a lot for your support, and we'll see you next time.