AWS Serverless Services with Paul Swail from Serverless First
In this episode the team talk about AWS Serverless Services with Paul Swail. Paul is an independent Serverless Consultant helping teams get started with serverless on AWS.
The Value Flywheel Effect Book Review
Paul also reviewed our book The Value Flywheel Effect. The team talk about his views of the book. And the fact that it is essential reading to get higher level buy in to serverless in your organisation.
Serverless Mindset versus Serverless First
We talk about the difference between the terms serverless mindset and serverless first. And how it is essential to be pragmatic in your approach and not insisting that serverless is the default for all your architectural decisions. It is better to use serverless as the architecture or service of choice. And then fall back to non serverless or less serverless services for certain parts of the architecture.
Wardley Mapping your Tech Stack
Wardley mapping and mapping your tech stack became really useful. Because you can see the individual components that can be moved to serverless. Instead of insisting the whole thing must move.
The origin of Serverless
Ben Kehoe came up with the term 'the serverless spectrum'. And it captures the notion of falling back. You can see if you have fallen back to things on one side of the spectrum.
Leading with Context
There's a fundamental thing with developers. Generally, they identify with the technologies they've use frequently. And they get really attached. And fanboy/girlism kicks in as well. There is a natural tendency to do that. Before your rational mind kicks in and says hold on. What is this? And what are the drawbacks of this? Or what sort of context would this not work in?
Context can be vague
I split it up into the application context of the actual system. What are the features or operational, latency and performance concerns you need the system to have. And the actual organisational context. The first application context is usually known. People know that they are building a specific app. It has these features and these requirements.
But the second organisational context is what developers do you have available to you? What skill sets do they have? Are they developers without ops skills so we need to hire infrastructure folks. Or do we have infrastructure folks, and they might not have much to so because we're using serverless? These things need to be considered when you're adopting technologies for a new project.
Serverless Maturity
We have reached maturity in the serverless ecosystem. Developer advocates are starting to codify and articulate their patterns. We have CDK patterns and things that are Google-able. Teams can reach out to see how to do it in an API gateway, lambda or dynamo. And see videos, tutorials or documentation of workshops that are maturing and established.
AWS effectively becomes a platform team, if you follow the serverless first approach. And they're going to keep investing in making it easier and adding more features and capabilities.
How will Serverless evolve?
Tools that optimise for small teams to get stuff done are essential. Like operational stuff or standardised CICD pipelines and testing mechanisms around that. There's a lot of good work being done on the friction that serverless had over traditional app development. Because it's a big overhead f
Hi folks! Welcome to the next edition of Serverless Craic. We have a special guest today! I am Dave Anderson, Technical Fellow at Bazaarvoice. And Author/Contributor at The Serverless Edge. guests
Mark McCann:
Mark McCann, Architect at Globalisation Partners. And Author/Contributor at The Serverless Edge.
Dave Anderson:
We have Mr. Paul Swail with us today. Paul, welcome to the show. Do you want to introduce yourself?
Paul Swail:
Thanks Dave and Mark for having me today. I'm Paul Swail. And I'm an independent Serverless Consultant working with and helping teams get started with serverless on AWS.
Dave Anderson:
We've done a few things together at different Meetups. So it's great to have you on. Before we get on to that, I want to say thanks for reviewing the book. You're one of our early reviewers. And you were quite kind. Our new book is The Value Flywheel Effect. What did you make of it from your perspective and your experience?
Paul Swail:
I really enjoyed it. I come from the individual contributor level. Like Engineers and Senior Architects who take a bottom up approach as to why Serverless is good. And the architectural patterns that make it great. It's harder to get buy in at a higher level. And sell the higher level of benefits like 'total cost of ownership' from the top down. Such as why serverless is worth adopting in an organisation. It's a longer term sell. You need to invest in learning it upfront. But the benefits are there in the long term. I liked the angle you have in the book. It's something you have to put in place at the start. And you reap the benefits over time.
Dave Anderson:
It's funny to hear you describe it. Because we would often talk about the pincer movement. The engineers get excited about it. But you have to bring the C Level in as well. You need both.
Mark McCann:
Creating that environment for success has led to us having hard lessons learned. When you're trying to make these changes without the right environment. When you do the bottom and get all engineers excited by technology x or technology y. And the environments not there. It gets stifled. And the engineers get frustrated and either leave or abandon what they're trying to do. So like you said, connecting those dots and bringing the top down and the bottom up is we've achieved in the book.
Dave Anderson:
One of the big concepts in the book is serverless first. When we started meeting up, I remember us thinking about serverless first and reading about it. We started to wonder what the domain name was for serverless first. We looked it up and realised that a guy form Belfast
Paul Swail:
I can't really remember where I first saw the had it! phrase serverless first. I should Google it and see what the first chronological result is. I had a notes page with the serverless mindset and then serverless first mindset. They are similar. But serverless mindset is the whole benefit of using serverless. In terms of total cost of ownership and why you can do things with smaller teams. And using serverless for your architecture by default for everything. Serverless first is a slightly more pragmatic adaptation of that. Where you recognise there may be use cases, like brownfield integrations or systems with special operational requirements. Say low latency, like financial transactions. Something which severless wouldn't quite build. So it's saying we'll use serverless by default as our as our architecture or services of choice. And then fall back to alternatives. Whether that's EC2. And then non serverless or less serverless services for certain parts of the architecture. Where that makes sense.
Mark McCann:
We find out that as well. We were gung ho serverless. And people thought that sounds a bit mad. So we changed to serverless first and not serverless only. There's plenty of compelling fallback options if this doesn't meet your needs. It resonates when there is an established solution, context, skills, experience and a pathway to production. It definitely opened a lot of doors for us. internally. People were accommodating or willing to embrace serverless. Because we weren't coming from a purist point of view anymore. We were pragmatic and more understanding of the context that the teams had. It brought the teams on the journey with us when we talked about serverless first. Rather than saying, you must go serverless.
Paul Swail:
There is a misconception. Somebody posted a
blog post recently:
"Serverless versus microservices". In other words serverless is something where you have to go all in. But they're different concepts. It's not one or the other. You could have an existing microservices based app. And then decide to use serverless. Individual services within that individual micro services could be serverless based. It's about showing ways to gradually introduce it into a huge microservices based system. And just introducing it piece by piece.
Mark McCann:
When we were facing similar challenges, Wardley mapping and mapping your tech stack became really useful. Because you can see the individual components and say:"Okay, let's take that component and move it to serverless.". Instead of saying the whole thing must move.
Dave Anderson:
I remember in 2016/17 thinking that serverless is now mature enough to start using properly. We had mobile first and then API first. So it was a great. And it's ready for us to start building this way. We are not going to move everything. Don't bother with the lower level stuff if you don't need to. Can you remember when it first started? Was it Ben Kehoe who started talking about it?
Mark McCann:
I'm not sure who came up with the original term. Ben came up with the serverless spectrum. We used that like a diagram. And that inspired the whole fall back. You have fallen back to things on that side of the spectrum. But at least you're on the spectrum. I'm not sure of the lineage of of these terms.
Dave Anderson:
Paul, your newsletter is brilliant with your daily writing. I don't know where you get the energy to keep writing so much.
Paul Swail:
I had a break over the summer. So I need to get back to it!
Dave Anderson:
It's a really impressive library. For anyone that wants to look that up. it's ServerlessFirst.com. There are brilliant writings and notes. One idea you wrote about is'leading with context'. And the tools and practices that are not a good fit. That's something that we've been talking about forever. Especially when you are working with teams. A team will
say:
'x is interesting'! And everyone does x automatically. But you need to bring context. I think it's a fascinating topic.
Paul Swail:
There's a fundamental thing with developers. Generally, they identify with the technologies they've use frequently. And they get really attached. And fanboy/girlism kicks in as well. There is a natural tendency to do that. Before your rational mind kicks in and says hold on. What is this? And what are the drawbacks of this? Or what sort of context would this not work in? Context is very vague. I split it up into the application context of the actual system. What are the features or operational, latency and performance concerns you need the system to have. And then there's the actual organisational context. The first application context is usually known. People know that they are building a specific app. It has these features and these requirements. But the second organisational context is what developers do you have available to you? What skill sets do they have? Are they developers without ops skills so we need to hire infrastructure folks. Or do we have infrastructure folks, and they might not have much to so because we're using serverless? And that may cause friction. These things need to be considered when you're adopting technologies for a new project.
Mark McCann:
The article is great. It's available on the Serverless First email newsletter and on LinkedIn as well. We've all got tool bias. It's like we are all going CDK! Or we're all going to the serverless framework. And then you ask what's your pathway to production? How do you provide value to your customers? Oh, you've got to set up in this way. We're more aware of leading with context. And making sure you're not impacting on time to value when you introduce new tools or techniques. And you need to know the socio technical context. It's not just the technical, it's socio health as well. And the whole organisational hierarchy. How does stuff get done in this company, organisation or team? You got need to be pragmatic. Your article really resonates with what we see. You have to understand the context, before you start insisting on using a particular too. We see that a lot. When you're more experienced, you think I
Paul Swail:
It's a message for anybody who advocates for serverless as well. There's a lot of focus on the architectural benefits and lower technical stuff, which it's all know this or I've got this. But a junior developer who's just important. It assumes that you have a lot of knowledge or in house AWS expertise already. Or there's a lot of assumptions that this is the way you should do it. Even though there's a wide range of services available from AWS. There are many nuances and decisions on the best way to use it. come in the door knows more than you ever will. You have to be open that. I heard a really good podcast recently from Joe Anderson with Adam Elmore. And I've heard Joe talking about lots of things. I think my worldview is very similar to his. He is CTO of an insurance startup in the United States. They usually use App sync. But he said something that really resonated. It was around their software engineering guidelines: we optimise for long term maintainability. And their hiring policy is for junior developers. And to get them trained up rather than looking for seniors. So that informs any architectural decisions that they make. They probably shun complex services or things which require higher operational burden. Because they don't want to hire infrastructure specialists. I like that mindset of focusing on smaller teams. Junior Developers can come in, learn quickly without a lot of mental overload.
Dave Anderson:
That's so important. There are two parts in the book that are interesting. The second phase of the value flywheel is challenge. Once you have an approach, create the environment that can challenge the thinking. And then fourth phase is long term value. That's what you should be always shooting for. We talk about using well architected as a way to framework that. It's the worst thing in the world, when a senior engineer builds something really complicated because they are smart. However, the beauty of a good system is simplicity. It is worth going the extra bit to get to simplicity. And don't overcomplicate things just for the sake of it. I don't you know if it's an ego thing, but sometimes people build things are way too complicated.
Paul Swail:
I've seen that in recent stuff as well. I can't help you if you want to go this complex.
Mark McCann:
The Team Topologies guys, Matthew Skelton and Manuel Pais coined the term 'organising for cognitive load'. You build the solutions to minimisie the cognitive load for your teams. That's exactly the same language we use. The best solutions fit in your head. Don't over engineer it. And only build what you need. We want to make sure we are lowering the cognitive burden for teams. One of the saying we use is: "Is it Google-able?'. Can you google these technologies? Or is it something that you've custom built? And only you know?
Paul Swail:
Even if you're the architect, imagine not being here for two months. How will the team fare? Have I designed a system that's simple enough if they hire a new developer in the meantime. When I'm not there to get them up to speed. That's a bit extreme. When a lead architect, or the initial developer creates a complex architecture, it's tied to them. If they're away, or just not there, the burning question is how does this work? How are we passing messages through all these different systems?
Dave Anderson:
It can happen when you go back to a piece of work as well and you can't remember how it works. Because you designed it six months ago.
Mark McCann:
I love the maturity of serverless or the ecosystem. And the fact that developer advocates are starting to codify and articulate their patterns in serverless land. We have CDK patterns and things that are Google-able. Teams can reach out to see how to do it in an API gateway, lambda or dynamo. There's maturity in those resources. They can reach out and see videos, tutorials or documentation of workshops that are maturing and established. So we can feel good about the way we've architected the system. We're set up for long term value. And we're not going to build something, move on and leave the team to crumble. AWS effectively becomes a platform team, if you follow the serverless first approach. And they're going to keep investing in making it easier and adding more features and capabilities. There's lots to be said for architecting in that way. We have integrity and we're not setting you up for failure. We are not the superstar that comes in, builds a solution and then sods off!
Paul Swail:
I am a consultant so I'm not a long term employee. So I want to get teams up to speed quickly. I do support them but not as an embedded engineer on a long term basis. That makes me extra aware. My goal is to get it built. But also make it understandable so that I don't become long term. I effectively roll them up and roll them off so they become self sufficient as soon as possible. That helps me optimise decisions around architecture that the team can can take on board themselves and run with.
Dave Anderson:
It's a bit like the saying of 'teach a person to fish'. Your goal is how do I get out of here as quickly as possible!
Paul Swail:
In a nice way! Because people could do the opposite. You have a working system until then leave. And then they are left wonderinf how do we fix bugs or add features to this?
Mark McCann:
You don't want to be a hard dependency on for a team. We've seen too many those type of consultancies. They come in they become a dependency you rely on. We want to enable and empower the teams to be successful. And then go off and have your own adventure.
Dave Anderson:
How do you see Serverless evolving? You have been involved in serverless for a long time. Where do you see things going next?
Paul Swail:
One reason for being attracted to it, in the first place, was as a developer you are working close to the product. Full stack developers build a front end and back end. But you can have a team of one back end and one front end developer. And they do everything. Tools that optimise for small teams to get stuff done. Like operational stuff or standardised CICD pipelines and testing mechanisms around that. There's a lot of good work being done on the friction that serverless had over traditional app development. And having to deploy to the cloud locally and making that fast. There are tools doing that already. I think that will continue to get better for the individual developer experience. And Cirrus stack and SSD are another who are making good abstractions. And make it less boilerplate. They are making good decisions around IDM. So creating stuff with would otherwise have complex AWS nuances there. It's an important service. Because it's a big overhead for for developers to learn without AWS
Dave Anderson:
I was speaking with Emily Shea recently. She certifications.
has a great phrase:
'Serverless is an operating system or operating models for the cloud layer'. So what do teams need to put on top of that? It could be la micro service architecture or an event driven architecture or whatever. That's a really interesting space. There's definitely a lot of optimizations and tightening up of those abstractions.
Mark McCann:
You're bang on about the developer experience. It's matured a lot but there's still a bit to go. It's in the conversation now. And they have plugged a lot of gaps that we had earler. So it's great to see that evolving. And by connecting disparate components together. the holistic developer experience is coming to the fore.
Dave Anderson:
Cheers for the conversation paul. I really enjoyed it. Wht's the best way for folk to reach out to you?
Paul Swail:
My website and newsletter is on Thanks very much. Great to chat! Don't forget to have a look at ServerlessFirst.com. Follow me on Twitter @PaulSwail. I have just launched a new testing audit service. So if you've teams that are having problems with bugs and your serverless out in production, and you want better tests, let me know! ServerlessFirst.com.Thanks for the conversation.