Few companies say they don't want good engineering. But they don't define what good looks like. And secondly,engineers are expected to write lots of code. For me, engineering excellence is not writing loads of code. It's about getting into code reliability, feedback loops and how well engineering is functioning. And how your team is performing and meeting bigger goals.
I love to say 'slow is smooth and smooth is fast'. So how do we set ourselves up for that? At the beginning, I like to go slowly to look at our process.
To define engineering excellence, we use Dan Pink's motivation factors of autonomy, mastery, and purpose. We translate those motivation factors to technical excellence, autonomy, and customer focus.
Well architected is better defined than engineering excellence. But I find that a lot of people haven't heard the term well architected and they think you've just made it up!
If you want to move fast, you've got to be empowered and know what good architecture looks like. When you have guidance or patterns that are implemented in a well architected fashion, that sets you up for rapid delivery, moving with speed, and in an engineering excellence way. It's fast flow aimed to facilitate you moving fast, getting things out quicker and being more competitive.
Teams are rapidly delivering product value multiple times a day. And they have a trail of evidence that they're going the right way because of these practices. They have dashboards, logs, alarms, alerts and the run books and the playbooks. And the key business indicators (KPIs) at their fingertips. So any stakeholder can challenge them on what they are doing, how they are operating or what they are deploying.
If you arrive in an area that isn't well architected, you've got to spend three months trying to understand what went on before or trying to understand the decision making. Or you've got 12 months of technical debt that should have been taken on board before you moved there. That very difficult from a personal perspective.
There's a whole bunch of advantages. One is a problem prevention culture. But I don't think it exists in a lot of organisations. It hasn't really cut through. They're still on the feature, factory, server mindset. Deliver, deliver deliver. Feature, feature feature. And then they have a tipping point where everything grinds to a halt for months or maybe years. And it may never recover.
But if you have a continuous improvement mindset and you are investing in fraud prevention, well architected, engineering excellence and continuous learning. And you are enabling and applying this to your teams, you get ahead of problems before they become problems.
From a problem prevention perspective, do you want to spend all your time babysitting a non well architected workload and dealing with all of that? You want your teams moving, adding value and moving on to the next thing. That's not possible if your build lacks quality, is always down or if you're having to constantly upgrade.
With well architected and engineering excellence, we can build things that run and increase in value. And there's less work over the long term, while your teams move on to what the business needs. I don't think a lot of orgs think in that way. By the time they do get thinking that way, they're already experiencing a lot of problems
Hi folks. Welcome to the next edition of Serverless Craic with Dave Anderson, Author and Contributor at The Serverless Edge and Technical Fellow at Bazaarvoice.
Mark McCann:
Mark McCann, Architect at Globalisation Partners and Author and Contributor at The Serverless Edge.
Michael O'Reilly:
Michael O'Reilly, Architect at Globalisation Partners and Contributor at The Serverless Edge.
Dave Anderson:
You almost forgot who you were there, Mike?
Michael O'Reilly:
I had to think about that for a second. Marks on his holidays there in the background.
Dave Anderson:
We were chatting earlier in the week about two things that I think are quite interesting: engineering excellence and well architected. They are two phrases we kick about a lot. And they're heavily featured in the book, The Value Flywheel Effect coming out soon. So do you think engineering excellence and well architected are well understood in the wider industry?
Michael O'Reilly:
That's a good question.
Dave Anderson:
It depends.
Michael O'Reilly:
I think it depends. The term engineering excellence is something we would use quite a lot. And it emcompasses a lot of broader topics. Well architected is emerging. People are becoming more aware in the organisations we work with. Because we are obviously big fans! Engineering excellence, enabling constraints and engineering performance in general are thought about by comapnies. But on an individual engineering level, I am not sure if it's well understood.
Mark McCann:
It's something that needs to mature and in the industry. The 'Accelerate' book and the work that Jez Humble and others have done with continuous delivery has helped. And Dave Hardy and others have helped to bring forward a culture of seeking engineering excellence. But it hasn't embedded across all organisations yet. You see it in pockets. Some say they have the four key metrics or we are focused on deployment frequency. But i's not just that. There' a lot more to it. I think it's improved. Teams are starting to understand the value of having a fast feedback loop. And baking engineering disciplines and guardrails into their pipelines and getting faster feedback and quality issues. But again, I don't think it's done in an holistic manner. Or a joined up manner yet.
Dave Anderson:
I think having two phrases or two terms for these things is important. And that there's a good definition for them. Let's dig into them, one by one. So let me start with engineering excellence. It's a funny one. I first heard that term from people in the States. I know that Amazon use it. But two things are interesting for me. Few companies will say they don't want good engineering. And we are happy with bad engineering. But they don't define what good looks like. And then the second thing is that engineers are expected to write lots of code. For me, engineering excellence is not writing loads of code. It's about getting into code reliability, feednack loops and how well engineering is functioning. And how your team is performing and meeting bigger goals. That's what we find super interesting. When we are doing this, we tie to a business goal. And that make the difference. As opposed to thinking, my complex is really low, aren't we really great! Lift it up a level.
Mark McCann:
Also they can start to articulate this when they're having prioritiation conversations with customers or business partners. It's not just feature, feature, feature! Or delivery, delivery, delivery. They're able to have a well articulated conversation about value. And investing in better testing or pathway to production. Or reducing code liabilities, doing a well architected review or a threat model. They're starting to have a language that can explain the value of these things to people that aren't on the team. So the stakeholders can see the benefit. In other words if we do these things, we can prevent problems down the line. We can help the flywheel turn faster in the short or medium term if we invest in engineering excellence. The work involved is described in 'Accelerate' and in our own book 'The Value Flywheel Effect'. They can help you make a case for investing in these things. So hopefully it is easier now than it has been in the past to get buy in and backing to invest in engineering excellence. So it's not just feature feature feature anymore.
Michael O'Reilly:
The'Accelerate' book and DORA metrics are for technical leadership . Teams want to do stuff fast and get stuff out. They are keen to solve problems or address business or customer need. Technical leadership is there to make sure it's done well. And down the line, how are we actually performing? We're delivering fast. And we're delivering with quality. But how many errors are coming back with each of these releases? What was the team's approach to solution that? Are we are we moving fast enough? I love to say 'slow is smooth and smooth is fast'. So how do we set ourselves up for that? At the beginning, I like to go slowly to look at our process. How are we going to test that this is successful in production? And how do we limit the the errors? What do we know and what do we not know? Also, what do we need to do up front? Leaders have got to be prepared to go slow and put those things in place to enable teams to operate more effectively. We talk about problem prevention being baked into the long term value section of the value flywheel. You either pay upfront. And do discovery and put rigour into pathways to production. Or else you are going to pay for it in the long term! You have to decide what's right. Obviously, we prefer to do things in the right way. With engineering excellence; engineering maturity and experience come into play. You have to control flow and observe how people are appraching things. So it's a really important topic for Tech Leadership in the industry.
Dave Anderson:
With enabling constraints, you're making decisions for the engineering teams. To define engineering excellence, we use Dan Pink's motivation factors of autonomy, mastery, and purpose. We translate those motivation factors to technical excellence, autonomy, and customer focus. It is a nice way to phrase it to explain to people what it means. The DORA metrics are also excellent. Herie is a question for you. Can you name the fifth DORA metric? The fifth one was added last year and it is reliability.
Michael O'Reilly:
We talk about engineering excellence, which is your flow and feedback cycles. And how you think your are performing. That bleeds into well architected dimensions so we are starting to see an overlap.
Dave Anderson:
We talked about Engineering Excelence. So that what about well architected? I think well architected is better defined than engineeing excellence. But I find that a lot of people haven't heard the term well architected and they think you've just made it up! It's actually quite specific.
Michael O'Reilly:
I'm starting to see lots more talks, And I'm starting to see it more in the community. People are starting to know what what what we mean, when we're talking about well architected. Certainly within the organisations we're in. This is where a need for discipline comes in. You have to assess the best way to go about something. Have you considered the various dimensions? What are we good at and what can we improve upon? And what would we like to change? Well architected is a vehicle for continuous improvement. It's evaluating where we're at and what we could do better.
Mark McCann:
We haven't created this ourselves. There's a body of evidence, supporting material, videos, tutorials, workshops and labs to support well architected. People can google well architected, whether they're talking about AWS, Azure or Google. And they're gong to get a fairly consistent response. They self serve and self learn. They can don't need to go to anyone to learn about well architected. There are courses and certificates also. It's starting to permeate through. Maybe we're in an echo chamber or little bubble, but it's definitely resonating a lot more than it ever has. Teams are loving the fact that we're starting to focus on these things, because they've always wanted to do them. And maybe they didn't have the capacity, the space or the language to fight to deliver well architected.
Dave Anderson:
Well architected is actually defined by a
Michael O'Reilly:
There's something that clicks with well separate body. I remember years ago, when you talked to architects, they said that good architecture is a non functional architected. When you run well architected with squads for the requirement. It's performability, scalability, extendibility and every stupid word ending in 'ility'. Everyone had a different list. And if you talked to somebody a month later, they had a different list again. So you never knew what someone was going to say. It became folklore and black magic. Now the fact is that it is well documented, understood and you can rhyme them off as SCORPS makes it easier. first time, it's interesting what sorts of reactions you get. There's always stuff that comes up: maybe if we had time, maybe if we had been thinking about that or we should definitely do this. But ultimately a team that wants to move fast does not have time to stop or pencil in a two week architectural review. Or bounce something off someone three weeks down the line to make a decision. If you want to move fast, you've got to be empowered and know what good architecture looks like. When you have guidance or patterns that are implemented in a well architected fashion, that sets you up for rapid delivery, moving with speed, and in an engineering excellence way. I's fast flow aimed to facilitate you moving fast, getting things out quicker and being more competitive.
Mark McCann:
With both of these approaches you are able to give all your stakeholders competence. Not only are you going faster, you're going faster in a good way with the data, metrics and evidence to back it up. So teams are rapidly delivering product value multiple times a day. And they have a trail of evidence that they're going the right way because of these practices. They have dashboards, logs, alarms, alerts and the run books and the playbooks. And the key business indicators (KPIs) at their fingertips. So any stakeholder can challenge them on what they are doing, how they are operating or what they are deploying. They have information radiators that teams can look at so they know what's happening
Dave Anderson:
That's really important. Back in the day we aimed to make good engineers who said the engineering was good because they were good! I have talked to a lot of C Suites and Board members. And when you say well architected or engineering excellence they want that. And you can rattle off the salienet points and direct them to the dashboard That's hugely powerful to convince people that this stuff's important.
Michael O'Reilly:
There's also another dimension to this. I've experienced squads that work with well architected, and they are fully bought in. They work through all the processes. And I have worked with squads that aren't bought in. When you compare the two, they're very different. At an organisational level, you need a set of commoditized standards. So tat teams can move from one area to another, and move to and from an engagement with a certain level of proficiency, qualiy, engineering excellence. or well architected. These sorts of things facilitate that. If you arrive in an area that isn't well architected, you've got to spend three months trying to understand what went on before or trying to understand the decision making. Or you've got 12 months of technical debt that should have been taken on board before you moved there. That very difficult from a personal perspective. These patterns enable fluidity to move one team to a different workload. And they'll be effective quickly, as opposed to them going to an area where they have to absorb a lot of cognitive burden.
Dave Anderson:
There's a whole bunch of advantages. One is a problem prevention culture, which is an interesting concept. Do companies undertand that they need a problem prevention culture in your engineering and product areas?
Mark McCann:
I don't think it exists in a lot of organisations, It hasn't really cut through. They're still on the feature, factory, server mindset. Deliver, deliver deliver. Feature, feature feature. And then they have a tipping point where everything grinds to a halt for months or maybe years. And it may never recover. But if you have a continuous improvement mindset and you are investing in fraud prevention, well architected, engineering excellence and continuous learning. And you are enabling and applying this to your teams, you get ahead of problems before they become problems. You're investing in performance, security, resilience, high availability, every day aand every week. And those things compound on each other. It's like compound interest! You're getting better returns as you go. I think that problem prevention culture is critical. But one of the big challenges is that silent excellence can go unrecognised or unrewarded. So you need to be careful that you have enough people who understand what good looks like, and reward it and recognise it appropriately. You don't want to fall into a superhero 'save the day' mindset! If you invested in problem prevention culture, there's no need for superheroes to come in and save the day!
Dave Anderson:
Another superhero is Mickey who stayed up all night. So he's a legend. Well you should have put more prep into the design review on Monday, and you wolud not have had to do that!
Michael O'Reilly:
From a problem prevention perspective, do you want to spend all your time babysitting a non well architected workload and dealing with all of that? You want your teams moving, adding value and moving on to the next thing. That's not possible if your build lacks quality, is always down or if you're having to constantly upgrade. This is why service is such a big concept. With well architected and engineering excellence, we can build things that run and increase in value. And there's less work over the long term, while your teams move on to what the business needs. I don't think a lot of orgs think in that way. By the time they do get thinking that way, they're already experiencing a lot of problems.
Mark McCann:
We talk about creating space for innovation. If you have a problem prevention culture, you're removing a lot of busy work. So your teams are truly able to focus on things that are innovative and help your business succeed. Because they're not fixing disasters, managing outages or chasing their tail trying to keep things alive.
Dave Anderson:
That's quite hard to explain to senior people. If you want more innovation, you need to do more boring architecture. Because one feeds the other.
Mark McCann:
Do you want all your best people firefighting? Or do you want them helping you grow in the marketplace?
Dave Anderson:
Or have your best people do architecture so everybody can innovate? That's the craic! Follow us on Twitter@ServerlessEdge and at TheServerlessEdge.com for our blog. And subsribe to Serverless Craic on YouTube. Thanks very much.