In this episode, I caught up with Ben Bridts, who is an AWS Community Hero and consultant at Cloudar in Belgium.
We talked about many topics, including common mistakes companies make in AWS; the problems with AWS Organizations; pitfalls with platform teams; and some success stories from his work as a consultant.
Links from the episode:
For more stories about real-world use of serverless technologies, please follow us on Twitter as @RealWorldSls and subscribe to this podcast.
Want to step up your AWS game and learn how to build production-ready serverless applications? Check out my upcoming workshops and I will teach you everything I know.
Opening theme song:
Cheery Monday by Kevin MacLeod
Link: https://incompetech.filmmusic.io/song/3495-cheery-monday
License: http://creativecommons.org/licenses/by/4.0
In this episode, I caught up with Ben Bridts, who is an AWS Community Hero and consultant at Cloudar in Belgium.
We talked about many topics, including common mistakes companies make in AWS; the problems with AWS Organizations; pitfalls with platform teams; and some success stories from his work as a consultant.
Links from the episode:
For more stories about real-world use of serverless technologies, please follow us on Twitter as @RealWorldSls and subscribe to this podcast.
Want to step up your AWS game and learn how to build production-ready serverless applications? Check out my upcoming workshops and I will teach you everything I know.
Opening theme song:
Cheery Monday by Kevin MacLeod
Link: https://incompetech.filmmusic.io/song/3495-cheery-monday
License: http://creativecommons.org/licenses/by/4.0
Yan Cui: 00:12
Hi, welcome back to another episode of Real World Serverless, a podcast where I speak with real-world practitioners and get their stories from the trenches. Today, I'm joined by Ben Bridts. Hey, man.
Ben Bridts: 00:23
Hey, thanks for having me.
Yan Cui: 00:25
Yeah, so we finally met face to face at the AWS Community Day in the Netherlands a couple of weeks back. And it was really good to finally see you in person. And also, we had a really good chat about some of the best practices around the AWS organisation, some of the things that AWS is not doing quite enough in that space, but also went to your talk about common mistakes you've seen people make, or rather mistakes that you've made yourself that you wish you knew.
Ben Bridts: 00:52
That’s what I’d like to talk about, like, I've made a lot of mistakes. I've also seen some mistakes like, a lot of the mistakes that are interesting to talk about are the ones that are like the logical mistakes to make, like those like little traps that are out there. And I've definitely like, like falling into some of those. So that was like a very fun talk to like compile those and share those.
Yan Cui: 01:15
Yeah, those definitely strike a chord with me as well. I've made similar mistakes in the past myself. I guess before we get into that, let me just maybe ask you to introduce yourself and talk about what do you do. Because I know you work for Cloudar in Belgium, and you guys are a really big consultancy, focusing entirely on AWS. So maybe, you know, give us paint us picture of what do you do?
Ben Bridts: 01:40
Yes, so I work at a company called Cloudar. We're a premier consulting partner which means that like we do everything that has to do with AWS, but also only AWS in our case, that's not true for every consulting partner. But that's like, the way we approach it is that we want to be like the like light bulb that goes on. If you think about I need some help with something in AWS and I'm in Belgium around there like we have some international clients as well. Like we want to, like be able to help there. So that goes from doing like very hands off stuff. Sometimes it's just doing reselling and billing and cost allocation to like we also have like a full managed service business very well, like, take everything that's on the AWS level out of customers hands and like manage that for them. And in between that whole spectrum, we do consulting where we will either do short-term projects, just like getting people started, have like, do migrations, or we will have like long, long-running projects where we have someone at a customer on-site and when we're working remotely with them for for like years sometimes.
Yan Cui: 02:53
Yeah, I guess the sort of traditional consulting business, everyone's kind of familiar with that. But I think what's really interesting from what we've discussed at the AWS Community Day was this, almost like a managed AWS account as a service that you provide to your customers, which I think it's a really interesting business, I guess a proposition and also something that I actually, I just think a lot of customers will be interested in because they want to use AWS, they want to use different services, they want to get all the benefits from AWS, but they don't really know how to scale that AWS environment, how to secure them how to manage the environment, and oftentimes you find the internal teams, essentially doing what you offer to your customers. But they're coming up with bespoke implementations, which I think oftentimes creates too much friction, and is not necessarily following industry best practices. Can you maybe talk about what does this managed AWS account as a service that you guys do?
Ben Bridts: 03:54
Yeah, so we, like we, we call it our landing zone, it's also for us built on the open source in this landing zone solution. I think it's technically open source. I don't think because it's actually on GitHub, where the idea is that, like, there's certain things that you want to do in every account. And so we're managing accounts for our customers, we're managing the workloads for our customers in those accounts. But we also want to make sure that like even like, we want to be flexible in what we allow customers to run. So we're not going to say like, this is like, we offer you like this packet of like you can have a web server with something in front of that. And you can put your like your file in that and that's it. We basically say you can use whichever AWS service that you want. But we're going to put like those guardrails and those monitoring inside of that, so we will make sure that like things are deployed with CloudFormation that we're looking at either like security groups that are open to the world or the S3 buckets that should be configured differently. Like we will have our tools on top of, or like, maybe next to your application, where we will do that, like central management of these are things we know are not a good idea to do in your AWS account. And so we can like, hopefully, prevent it from getting that far, because like, this is part of our managed services. So like, we will do the setup for you as well, if you want, like we have some customers that do want to write their own CloudFormation or TerraForm or CDK, and that will be deployed for them after like doing a review of it. But like, we also just offer it like, we will do everything for you. It's just like you tell us what you need. And we will write the CloudFormation and deploy it for you. So like we have basically different approaches of making sure that what actually gets deployed is it's not something that will blow up in your face basically.
Yan Cui: 05:52
Okay, yes, I think that's, that's quite, that's a quite interesting approach and quite interesting service. And again, I think a lot of customers that I've seen would have been quite happy to use something like that to get them started, at least, until they get to a certain level of maturity and being able to do a lot of that themselves.
Ben Bridts: 06:09
It is a problem that like everybody runs into at some point because like the general best practice within AWS is to use multiple accounts. And that is great. And like, it's, it's less work than you think it is, like the step from going to one to two or three accounts, and then going to like 100 accounts like that scaling, once you have like that first three accounts in place, is not terrible. But there's also all the tools that are like, we're not going to say all of those. There's a lot of different tools that like help with that. So it's hard to know which one do you want to use. And there's like, within the AWS ecosystem, you have Control Tower and Organisations and none of it is perfect. It's all like it has like all the tests edges everywhere you need to know like, it's, for example, just as an example, a Control Tower will not deploy GuardDuty. GuardDuty is something that you basically want everywhere in your accounts. So like knowing that and like having a way to then deploy GuardDuty on top of your Control Tower configuration. Like those things are just hard. And I don't think that our solution is necessarily like the end all of it like I wouldn't like if I would start again, I wouldn't build it on top of that landing zone solution. Because like, that was like the best thing out there the moment that we started. But you need something you need like something in place to know like, I'm going to have something like in every account and like enable a CloudTrail to have config for CloudTrail. Like all those things. And it's there's not a great solution out there yet.
Yan Cui: 07:49
Yeah, I kind of agree with that. Personally, I do like to use org-formation. I think for me, that's probably still the best one out there and solves some really interesting problems that are quite difficult to do with a landing zone, which is focusing on one account. But when you got cross-account references and things like that, where stack set is useful for sort of just taking a template and deploying the resources to different accounts, but not so much for oh, I want to have your so one audit account with S3 bucket for all my CloudTrails, and then have all my other accounts be able to just say every time I create a new account, that's not the audit account, just enable the CloudTrail and ship my logs to this one S3 S3 bucket in this audit account, that sort of thing that I think I have to do almost every single time I create a new organisation. And that kind of thing requires cross-account references, it becomes quite hard to do without something like org-formation.
Ben Bridts: 08:48
I agree with that 100%. Like the recommendation that I usually give is, if you have already something I probably would use that because you've invested in that. But if you're like starting from scratch, if you know what you want, org-formation is probably the best solution. But like it requires knowing what are the things that you want to put in there. If you have no idea what you want, but you want to have something, then Control Tower is like a decent way to start. Like it has a lot of things that it doesn't do or like I wish it would do differently. And like you still end up running your own customizations software on top of it, which is like more management for you. But it's a place to get started at least. Whereas org-formation really requires a little bit more of you making decisions where you can start with Control Tower just like click a click in the console, click on like four buttons and you have some.
Yan Cui: 09:48
Yeah, with org-formation you have to learn its own syntax and all that as well. There's some really clever things you can do which again, you know, when it's clever, it means that there's a bit of a learning curve.
Ben Bridts: 09:59
I think it's absolutely worth the investment. But it's a balance you have to choose there sometimes it's not something you want to immediately start with.
Yan Cui: 10:07
Yeah, totally agree. And, and yeah, talking about things that org-formation you wish that they would do differently. One thing I've come across, sorry, not org-formation, organisations, I've come across several times, that is, you have got these organisation events which are basically, you know, when you enable all of these events in, in CloudTrail that you can then capture and all of that, but then the, you know, it enables the sort of centralising of all of your CloudTrail logs in the, in the one central accounts, but it charges you based on the number of events that gets raised. And that got really expensive really quickly, even when I've got like bare bones empty environments. Whereas, you know, when I just do what I normally do of enabling CloudTrail, but then shipped the logs to S3, in this one audit account, I almost pay nothing for that. Because S3 is so much cheaper and you don't get charged extra for every single event. So there are some things like that which keep money into small things that you just think, Okay, why would they do this, it just doesn't make any sense.
Ben Bridts: 11:15
I think that's especially true with serverless, where there are certain services we didn't really ask like where the unit cost works much better if you work with servers. Like you have some things where it's like small like config, you pay per change, which means if you're working with serverless application, you're going to be paying for every deployment. If you're working with servers, and you're just not replacing that, which is like pushing config, or like pushing your code into that server, then you don't pay anything. And I think the same is true for DevOps Guru, where you like pay per monitored resource. And then like, Lambda gets more expensive, because there's a lot more of those. So that's, I like I feel that pain where like the, I want my billing model to be similar to Lambda for everything where I like pay like a very, very tiny amount. Like I don't get punished for creating more of something. Like I'm, I'm willing to pay more for using more of something. But like, for the creation, having to pay for a thing that exists. Even if you don't use it is always a painful thing.
Yan Cui: 12:27
Yeah, I remember back in the day, DataDog was charging, I think $5 per resource. And when you've got a Lambda, and you can easily have hundreds of those, so it just becomes crazy expensive. I think, I think was it last year, they changed the pricing model for Lambda. So that is no longer based on number of resources you monitor, so that it just doesn't just become crazy expensive when someone uses a lot of Lambda functions. So I guess talking about sort of Lambda and serverless, you know, you guys do all AWS, which obviously means quite a lot of serverless. But also, sometimes containers, sometimes EC2 as well. Do you have some I guess some sort of high-level guidelines in terms of when would you recommend serverless to a customer? And when would you say you know, let's just do containers, or let's do EC2 instead?
Ben Bridts: 13:17
I'm a big proponent of serverless. Like in general, and managed services specifically. So I like to try to do as much serverless as possible. But it also really depends on where you're coming from. I'm pretty sure you have like this kind of conversations before because I listened to some of those episodes of your podcast. So the you can't go or you can't always go from we're running, like on a virtual machine in a data centre, to now we're doing everything with Lambda functions and DynamoDB, not because like that's too big of a leap, but because the timelines just don't always work, like there you will need to like rewrite everything at some point for that, like you can't just take well, there are ways to do it. And like, it's usually not the best idea to take like your production application, like this very important to you then like, put a wrapper around it and like put it in a Lambda function, like it works and there’s tools for that. And if it's something that's I'm sure there are people that are very successful with that, and I like those. But I don't think that's like the end of it. And it's probably not what you want to do with an application that's going to be working on keeping improving. So then it becomes a question about how much development effort do we want to put in this specific application to make it serverless and what are like the wins there. And in a lot of cases, the first wins are not in your compute. Because if you're like if it's a somewhat modern application, and in that I mean like the last 10 or 15 years, your the most part of your computer is going to be stateless. So whenever you run that like, it's easier to run a Lambda than to run in containers than to run on a server. But if something goes wrong with that, that's not your biggest pain point, like your biggest pain point is going to be your database, your storage, those things. So like moving things to S3, and making that part serverless, like the part that like has your data to put that on S3 or like have something that has to be there, I think is usually much more important than whether you end up running it like on Elastic Beanstalk or on a container, or in a Lambda function. Even though like I would want everyone to like get the benefits of the Lambda function where they only pay for the actual requests that they get. Sometimes that's like too much of a rewrite to do that immediately. One thing that we do see with our clients is that you don't have to do it all at once. Like you can move like you can do with like a longer-term thing, where we say like, Okay, we need to like get out of our data centre like our contract is ending there, we lift and shift these services like we read that from the database, read that from storage. And then once we've once we're in the cloud, we put API Gateway in front, and then we look at what are like our critical paths, like do we need like is this something that is going to get a lot of requests, that's going to be like, are very burstable requests, let's pull it out of there, put a Lambda function in there, and then run that, like, basically, like, intercept that entrypoint. The other thing where we see that, like, the more serverless approach is very successful, is our clients did a lot of data processing. Because their workloads are, almost by definition, very batch centric. And those are usually like they like are like even more stateless than like your typical website. Or, like, know already about like having external data, and moving those to either Step Functions with either Lambda functions, or Fargate containers underneath, depending on what what data they're going to be using, and how long they're going to be running is something that like, I feel like works very well.
Yan Cui: 17:22
Yeah, I definitely agree with that, especially when most projects, you know, have got deadlines you gotta worry about, you can't just do the, the absolute best thing, you have to do the thing you can achieve within a reasonable amount of time. But I guess this, this strategy of trying to, you know, push someone towards, say DynamoDB, or S3, I think S3 is fairly easy to integrate with. But I also find that a lot of customers struggle with DynamoDB, especially when they've always worked with relational databases. DynamoDB the fact that it's really restrictive is something that I really love, personally, the fact that you can't do anything that just won't scale. But I think a lot of customers find that really difficult. Have you found some, have you found some sort of successful pattern for introducing something like DynamoDB to customers who's just never used something like that before?
Ben Bridts: 18:15
Yeah, like, I agree with that, like, and especially if you're working with frameworks, if you're used to working with like an ORM layer, and then you have to plug in DynamoDB, where you not only have to care about defining your data, like entities in your code, but now you also have to, like, really define them in your database. That becomes really hard. The places where I think it's, I think that, well, let me rephrase differently. I think like, the same thing applies here with like, digging into like little pieces, there are like certain places, where DynamoDB is like a very good fit for almost everyone. Sometimes that's just like on the operational side of things where if like, oh, we have to do some, like user management for like our internal users, like, that's very, like straightforward. It's obviously couldn't be like exactly that user. Sometimes things like session management, depending on like, how latency sensitive you are, can be also like a good fit there. But yeah, I agree it's hard. Like I don't think there's like one way to go to DynamoDB especially if you're coming from like the like if if you're not running everything serverless like if you're like still using some and even their containers, or especially EC2. If you're already like thinking with like those long-running processes in mind, then DynamoDB like a very big jump. I think once you like inside like Lambda that function as a service ecosystem where like everything is very ephemeral and very short-lived. And you actually run into problems if you want to talk to a relational database because now you’ll need that RDS proxy in front of it, or like you're doing a lot need a lot of memory tweaking or like the Aurora HTTP APIs, at that point you start feeling the pain of using a relational database. And I think that's like the place where like, you go into look at, can we use something else? Like, where's the data coming from? Does that need to be in a relational database? Or can this be in DynamoDB. And I think that also ties into like the second place where it's usually an easier thing is people will move to GraphQL. I see there that like DynamoDB is like a much, like easier way to get started with them. Or like, much more popular because like, you have the direct integration with AppSync and DynamoDB, where you can just say, like, oh, this is my endpoint, this is my database and like, figure it out. You do end up like in a single table design, or multi-table design that way, which I know like, Alex DeBrie will not be a fan of. But like, it is one way to like, get familiar with it at least.
Yan Cui: 21:01
Yeah, I think Alex is a fan of using multi-tables approach when it comes to AppSync. I think he's got a pretty good blog post that I can link in the show notes, where he talks about the trade-offs. And when I spoke with him before he was he was saying how he felt with AppSync and GraphQL. It makes more sense to use multiple tables rather than the single table design. I think one of the things that you also get forced into doing more if you use a single table design with AppSync is having to write more logic in VTL, which drives people crazy. People just can't stand it. Yeah, it's not it's not it's not everyone's cup of tea, but I think you get used to it. At least I have.
Ben Bridts: 21:47
I think it also makes sense because like with GraphQL, you're mostly working like your data access is going to be or like the data pattern is going to mostly be defined client side. So you will be harder to model it as a single table anyway, because like, it's very hard, much harder to like predict those those accesses.
Yan Cui: 22:07
Okay, so switching back to kind of what we started off with, this talk you did about mistakes that you've made yourself that you wish you had known better back in the day. Are there some other common mistakes that you've seen customers make when they make this transition to the cloud or to serverless?
Ben Bridts: 22:24
Yes, one thing that I've been like thinking about a lot lately, and that ties a little bit on that common mistake thing is that it's very hard to be like a platform or cloud enablement, or like, whatever you want to call it like that, like Cloud Centre of Excellence team. Because like almost every mistake that I made, in the end lead to you have like some piece of code or infrastructure, where the ownership is not clearly defined. So like my general recommendation for if you're getting like started with the cloud, and you're starting like a Cloud Centre of Excellence, or like a Cloud Enablement team, where you can like put your cloud experts in, is to make sure that if you build something, or if you're going to be like defining best practices, that you make sure that you actually allow people to learn the best practices, and like have them own the thing that they're deploying, instead of saying, oh, we'll do everything for you. Like, it's a good thing to like, make things easier. But you can't hide the things like you can't say, like, we will set up your S3 buckets with all the best practices. And like, you just need to call it like, talk to the API. And that's it. Because at some point, people will have like their own special requests that they need, like, I need different kinds of S3 buckets, or there will be a new best practice. And they have to update all the S3 buckets of all the teams. And that is hard, because you might think that you know what you're doing with that S3 buckets, but you don't. There will always be some edge case where you think everything is private, but… you think everyone is using bucket policy, for example, there's still one team that's using ACLs. And you want to disable ACLs like company wide, you flip the switch, and suddenly the application is broken. And I think that is like the main thing I've learned, like making those mistakes is that like the place where they're most painful is you tried to do something for someone else. And then you break their thing. And it usually starts with you you have like first held them and like they're fast and like everything goes great. And then a year later you break their thing. So it's like not always like immediately obvious that like you're taking things away from them is leading to that. That place of now we have to do everything for them or they like don't own it anymore, like very muddy. So I think that's like my main my main takeaway of like things to be careful about is like you don't want to do things behind people's back. Or like to give like a very like clear example. Like there are some tools like CDK that allow you to plug in. We see it gets called aspects with like our tools, similar things where it can basically can say every time I see an S3 bucket being deployed, change this setting, right, like, make sure it's encrypted, make sure public access block is enabled. And like doing that thing of doing a check of like making sure that it follows best practices. I'm a big fan of that, like, make sure that like everything is deployed the right way. But you can't do it by hiding it, you have to just like fail the pipeline, show people like this is something that is not following best practices, please change your configuration, because they own the configuration. Instead of going like oh, like I could easily like I know what the best thing is, I'll just like set all the properties the way I think they should be. And then they deploy it, and it ends up working for them and like being like following the best practices. But then the moment that something is not working, like you need cross account access, they will not know that encryption is enabled. And that will have an effect on like your IAM policies and things like that. So I think like the, you have to empower people, basically, you don't, you can't take, you have to make things easier for them. But you can't do it by like, creating a mess, like they don't don't know what they're doing. We have to teach people who are new to Cloud, like how you should do it and how you can do it. Instead of saying like, you don't know this, I'll just do it for you.
Yan Cui: 26:41
Yeah, I think when it comes to defining or designing abstractions, sometimes it’s easy to go be too crazy and think, you know, we tried to reduce the amount of code users have to write as much as possible, which is not the whole point. It's not just about reducing the number of lines you have to write. But also, you want to make sure that things that are easy to do are still easy to do with your abstraction. But things that are more advanced, you can still enable those behaviours, those those decisions, rather than just something that, okay, now we are restricting the amount of things you could do with this new abstractions. So when you do need to step outside of the boundaries that we have said, you can't do that yourself anymore, which I think I've seen that a lot of times, especially with a lot of the sort of homemade abstractions you've seen in law companies that are probably not that experienced at, you know, building abstractions and tools and frameworks. And you see these kinds of mistakes a lot, I guess want to maybe just pull back what you said earlier about in terms of the responsibility of where those platform engineering team should be. Because one of the things I've seen a lot recently this meme that DevOps is dead, long live DevOps. And now it's all about the platform team, even though, you know, you and I know, DevOps is a collection of practices and principles. It’s not, you know, it's not specifically about someone's title, the fact that you've got DevOps engineers is kind of silly to me. And one of the things I've seen a lot is exactly what is described in terms of platform team coming in, doing everything for you, no education, no integration, just use this thing that we've cooked up because we know what's best. And then the actual application team ended up having to just getting getting a lot of friction, sort of getting what they need to do done. And when they need to do something different from what the platform team had envisioned, it becomes almost impossible. And you end up having lost the sort of friction between different different parts of the business because of that.
Ben Bridts: 28:38
It's frustrating for everyone, because like the platform team is going to like we built something, nobody's using it. And like the application teams are like, we want to use it but it doesn't do what we want to do or like it's like, it takes too long to change something or like there are like multiple different things that can go wrong. Versus like I like there's a lot of value in having platform teams, and like SRE and DevOps and all those things, like, it's super important. You just have to make sure that you're working together and like doing the right things.
Yan Cui: 29:13
And I guess instead of just talking about the things that customers do badly, are there some examples of things that the customers are doing that you think, oh, wow, that's really cool, that I wish more people will be doing that?
Ben Bridts: 29:25
Yes. Like, I think there are like a lot of places where they actually get platform teams or like however you want to call it like get it right. Like there's a lot of value to be created by like managing or like looking at things across all the accountants across the whole estate. And then it can like be simple things as like, like help the security team like, have this like, everything has an API. This is great. Like we can do the same thing that we did like now you can like use Docker and like all that kind of more operational stuff on your local machine. And like, be very easy to like build things and like have access to everything, we can do the same thing in the Cloud. But we can like now it's just an API instead of something that you run locally. But like so just like a lot of API driven things that the platform team can do, like, look at all the states, like I've seen that platform teams look at cost. I've seen them focus on tooling, it’s something that's also very useful where you can like, again, they have very clearly defined owners like we're there, we will need, for example, a Jenkins server, we're going to run Jenkins as our CI/CD. Someone has to own that. Someone has to build that. Someone has to make sure it's available. Like, right examples for that like, make sure you can integrate all the things as libraries for like the common things that your, that your organisation is going to be doing. So I think that there's like there's like a lot of value to be created and a lot of value being created. One thing that also like very, like, usually recommend, I know, like some companies like LEGO are doing it's like looking at what are best practices and like doing things like well-architected reviews, or something similar to that, like we're going to look at things we're building, as an organisation. And we're going to make sure that we have thought about like the set of questions. And that doesn't have to be like an enforcement thing where you say, like, our team is going to like, like, look at all those questions. And like, you're not going to production before you say yes to all of them. It can be like, like, I like to call it cloud enablement, it can be that like, we're going to, like, look at your application, those questions is going to be a conversation about like, where are the business priorities? Where are like, the things we still need to work on? And make sure that like, we're on the same page of like, what are things that will enable us to like, not just go to the cloud, but take advantage of all things that are available in the clouds. And I think that's like, where serverless also comes up very frequently when you look at, OK, like we have all these like questions and like all these problems and things we have to solve. And like a lot of those can go away if you don't have to manage the servers yourself. Or if you don't have to manage the scaling yourself. Like there's a lot of there's a lot of tricky things about making things highly available, that are solved easily by using API Gateway and Lambda.
Yan Cui: 32:27
Yeah. And I remember talking to Sheen about this recently as well. And I think he he talks about, you know, building this community of practices within companies like LEGO so that what they're doing in terms of the architects, they're not gatekeepers as as opposed to just being you know, people that you can pull into your project and get help from as opposed to someone who have to sign off on everything that you do, which I think creates a very different dynamic in terms of the relationship between people like himself and the application teams themselves, which I think makes it much more harmonious, the environment for people to adopt best practices because they want to, and also, I guess like you said earlier, it's not about creating tools and forcing people to use it, but rather, it's about education. It's about making it easy to do the right thing, but also help people understand why that why you do it. Why you need to do it in the first place?
Ben Bridts: 33:23
I really liked that phrasing and like I want to go back to what you said earlier. Because like that, gatekeeping is like the exact reason that you started calling things DevOps, like the whole thing of like, just like I told you over the wall. And operations teams started saying, No, we're not doing that. We just have to make sure that we're not getting that same situation again with clouds, where it's like, oh, I talked to the cloud team. And now the cloud team is saying, No, you're not doing that. Everything everything old is new again. I guess but like, I think that's like this. It's the same way. We're just like doing it at a higher production level.
Yan Cui: 33:57
Yeah, you can say the same thing about Agile as well that agile was initially about, you know, a very simple manifesto, a few guidelines and principles, and then the things that scrum came and then this again, we back to that rigid structure of every two weeks you have a sprint, and then every sprint you've got various ceremonies. Again, we're going back to following your plan as opposed to being actually agile and reacting to circumstances and contexts and actual needs of the project which again is another thing that just drives me nuts.
Ben Bridts: 34:33
I think a lot of those things are just very human. Like we all start out with the best intentions. But once something goes wrong once we're like it's like we need a rule about this. It’s like a very human thing to have. And like you need some rules. But like you also need like some some freedom to like, build things within within those rules.
Yan Cui: 34:54
Yeah, I'm not I'm not calling for anarchy within a project that needs to be some oversight, some governance, but yeah, it’s just I just find that this whole Agile has been so turned around into the thing that you're supposed to be, to not be.
Ben Bridts: 35:11
Like, that's something that like, by sometimes, like I also like, give some trainings now and then, like the serverless monolith is something that I mentioned in that same way, or like we want to like do things serverlessly and like we want to use it. Let's just start like using microservices and like, these are microservices and this like how that works and why we do that. And like this whole very nice in theory, but I do also like want to explain it and like it's not just about speeding things up in the parts of codes. And so like you can think about them separately is actually so you can like deploy them separately, scaled them separately. Because otherwise, you end up with like that distributed monolith where you don't like you make it you it's not necessarily more complex, it just like more externally complex, but like you're still doing everything at the same time, still running it through that same problems like the same ownership thing like that I basically started with comes up again in that way like we're all deploying things at the same time. So now we have to wait on each other is something that's not solved if you just like make your microservices like all interconnected.
Yan Cui: 36:17
Yep, that's why you have things like distributed monolith instead of actual microservices, right? Before we go, I do want to just ask about, you know, right now everyone is kind of laying off and from what I can see, the industry is having a bit of a downturn. So how are you guys doing business-wise and are you guys still looking to hire people right now?
Ben Bridts: 36:40
We're, well, it's not my business, so I can't like speak for everything. I just worked there. I feel like we're still in the right paths, like focusing on that that AWS stuff and also like we're we haven't grown like super fast. Like we're 50 People now. I hope it'll still keep growing and I like I know, like as a business in general are being a little bit cautious. Like maybe we do see like the the downturn, but like I personally also think that cloud is not going away. Like there will be more cloud work will still be like, looking for people to join us. It's probably not going to be like a like super fast like we like hire everyone as fast as possible. Like in this like, for me, I feel like in our industry, we're still pretty small, in the sense that like, I know you're probably a bit of another one by basically you could like, if you really wanted to, you could know everyone would ask clouds in Belgium and Netherlands. And that's not going to be a list of like 10,000 people. There's gonna be a very small list. And I want more people to do cloud things and I think there will be need for more people to do cloud things. So while there might be like I'm like, I'm definitely aware of like things like economically I can, like salaries going up and all the prices of everything going up and inflation, I think, like, I don't want to be like super optimistic, but I also think that we will need more people at some point, and that is going to be sooner rather than later. Like, I don't think cloud is going to be the business that suddenly goes away. And I feel like we are still like. Everyone was working in clouds has work like it's not like we have like extra people like sitting around doing nothing.
Yan Cui: 38:36
I'm hopeful that the economic downturn and now there's more focus on companies actually being profitable. And this sounds like more and more companies are paying attention to the operational expenses that should hopefully help propel cloud and serverless especially when you consider how much people are spending on redundant CPU cycles that are sitting there doing nothing 24/7 and looking at serverless for a lot of applications would have made a lot of sense from the operational cost point of view but also from the amount of people you need to get the same amount of output and productivity as well. So I’m actually hopeful that this is a good opportunity for cloud, but also for serverless to make a bigger impact in terms of the industry.
Ben Bridts: 39:25
It's definitely something if I was starting a startup, I would be running the company completely serverless not just for the whole operational thing. But also for like if my like my idea is not the right one after the pivot. Then like I haven't invested in like a whole bunch of servers that I don't need anymore. I can just like play with it and start paying for the other thing that I'm doing now.
Yan Cui: 39:49
Yeah, this whole capital expenditure versus operational expenditure was a really big discussion early on in the in the sort of cloud thing. I remember Werner talks about it all the time, but I think a lot of people just kind of forgot about it once they're in the clouds. Because when you're taking about, you know, building applications and having to run a bunch of EC2 clusters, or having to run the RDS and relational databases on the on uptime basis again, those are uptime cost versus just paying based on usage, which again, something that we are kind of going back to where we were before, but just in a slightly different way.
Ben Bridts: 40:26
Yeah. Although I do think it's important to keep that in mind like that fact that like things are not like set in stone the things for spending because the like the best way to get everything out of clouds is to experiment. Like you can solve every problem like 10 different ways. So figure out like which way is the best one for you. Sometimes we'll take it to find things or even just like like the typical example that I like to give here's like all those AI services. Like sometimes you want to do image recognition, and like the AI services and like recognition that is available is perfect for you. And you can just try it and it works and gives you all the things you want. Sometimes it just doesn't give you what you want, because you have a very specific use case. And then you have to like build your own model and like start doing like more complicated or like more expensive run, running things. If you don't like experiment and try out that like first service even if that like will cost you like a few dollars for a few days. Then you will never know which one of the two you need and you may start to get expensive or difficult to maintain thing because that can everything instead of looking isn't like a very specific thing that maybe fits 90% of our use case.
Yan Cui: 41:41
Yeah, yeah, totally agree. And unfortunately, I guess that's also something that's really difficult to do. Because finding the time to do the experimentations and as you learn those different services is not easy to do. It’s time-consuming.
Ben Bridts: 41:55
Yeah, yeah. It's not a perfect world obviously, but at least you don't keep paying for it after you stop your experiment.
Yan Cui: 42:02
That's also one of the nice things about all serverless components as well that, you know, when you're doing experiment, most of the time you don't have to pay for anything because Lambda functions are based on the usage.
Ben Bridts: 42:12
Yeah, it's sometimes actual like it costs you more to like calculate how much it will cost that will actually cost.
Yan Cui: 42:19
Yeah, you say that, I actually I had someone told me about they had this meeting of I think eight people for about an hour talking about how they can optimise the cost for his Lambda function. And when I heard how much their function actually cost per month, it was like a couple of dollars so they can do all they want but it's gonna save them pennies and it's gonna take them years so even just pay back the time they spent talking about how to save, cut down on cost. So I think that's when it comes to Lambda you really have to just find the right place to optimise before you decide to even think about optimizations.
Ben Bridts: 42:57
Yeah, exactly. And that's like that's also my point Lambda, like the fact that you're gonna split it to like all different functions. Like makes it very interesting that way because maybe there's like a function that gets called 100 times as much as the other ones. And then that's like a good candidate look at like, what's like specifically we can do for that function. And then for the other functions, maybe you want to do something like this more generic to save costs, like how like do we like switch everything to Graviton and save money that way, for example, like that something that we're that engineering time for, like how much are we going to save how we're going to do it? Like you can sit like all Lambda functions at once, and then it might be worth it. If you're doing that like calculate for just one Lambda function, then it's probably like a step like not worth the time. Because you're saving pennies, like spending, even if you're spending like 15 minutes on deployment to switch to Graviton it's gonna take a while before you get that money back.
Yan Cui: 43:55
Yeah, yeah. All right. Again, thank you so much for taking the time to talk to us today. And I guess, are you going to re:Invent this year?
Ben Bridts: 44:03
I am going to re:Invent this year. Yeah.
Yan Cui: 44:06
Okay, cool. Yeah. So enjoy. I think quite a few people I know are going to re:Invent for the first time. I think Luc is gonna be there as well. I just spoke with him recently on this podcast. But yeah, been great chatting to you today. And I will hopefully see you around these parts soon.
Ben Bridts: 44:22
Yeah, hope to hope to see you soon. We're not that far away. So we might try to see each other more.
Yan Cui: 44:27
Cool. Take it easy, man. Ok. Bye bye.
Yan Cui: 44:29
See you.
Yan Cui: 44:43
So that's it for another episode of Real World Serverless. To access the show notes, please go to realworldserverless.com. If you want to learn how to build production-ready serverless applications, please check out my upcoming courses at productionreadyserverless.com. And I'll see you guys next time.