Contract Testing with Lewis Prescott

It's no surprise that this episode's guest Lewis Prescott has a website called "pactman". Pact is a popular contract testing tool, and Lewis is indeed a man who can tell quite a bit about Pact and contract testing. In this conversation, you'll learn more about contract testing, its benefits, misconceptions, and where to start. 

Find Lewis on:
- LinkedIn: https://www.linkedin.com/in/lewis-prescott-12409a44/
- Twitter: https://twitter.com/WuigPrescott
- Lewis' website: https://www.pactman.co.uk/

Mentions and resources:

• Spotify Testing Honeycomb model: https://engineering.atspotify.com/2018/01/testing-of-microservices/
• Pactflow's Bi-Directional Contract Testing: https://pactflow.io/bi-directional-contract-testing/
• Lewis' How To Start API Contract Testing Podcast: https://www.pactman.co.uk/contract-testing-podcast
• Youtube video that was Lewis' first intro to contract testing: https://www.youtube.com/watch?v=nQ0UGY2-YYI

Follow Quality Bits host Lina Zubyte on:
- Twitter: https://twitter.com/buggylina
- LinkedIn: https://www.linkedin.com/in/linazubyte/
- Website: https://qualitybits.tech/

Follow Quality Bits on your favorite listening platform and Twitter: https://twitter.com/qualitybitstech to stay updated with future content.

If you like this podcast and would like to support its making, feel free to buy me a coffee: https://www.buymeacoffee.com/linazubyte

Thank you for listening! 

Transcript

Lina Zubyte (00:06):
Hi everyone. Welcome to Quality Bits - a podcast about building high-quality products and teams. I'm your host Lina Zubyte. In this episode, my guest is Lewis Prescott. As Lewis is extremely passionate about contract testing, I could not miss a chance to talk about it with him. We discuss what contract tests are, their benefits, some misconceptions that we have about them as well. I do feel a contract testing is a concept that not every team knows about and they could really benefit when it comes to getting better feedback and doing better testing practices in building high-quality products. Enjoy this conversation.

(01:04):
Hi Lewis! Welcome to Quality Bits.

Lewis Prescott (01:07):
Hi. Thank you so much for having me.

Lina Zubyte (01:10):
It's so nice to talk to you again. We met at a conference years ago and I'm so excited to learn what you're up to. So for myself and for the ones that are listening, could you shortly introduce yourself?

Lewis Prescott (01:25):
Yeah, so I'm Lewis Prescott. I am currently QA lead at Cera Care, which is a healthcare startup here in the UK. I previously worked for companies such as Cancer Research UK and Asos. Been around the space for a while, I talk a lot about contract testing and end-to-end testing and things like that.

Lina Zubyte (01:48):
Exciting. And who are you apart from your work?

Lewis Prescott (01:52):
I'm bit of a sports junkie, I guess. I've run a marathon before. I've cycled to Paris from London before. That's kind of all a bit on hold at the moment because I have a soon to be two year old daughter, so she takes off a lot of my time. But I have a triathlon in the summer, so I'm kind of working towards that at the moment.

Lina Zubyte (02:17):
Wow, I did not know that. Super exciting. Could you tell a little bit more about your current role? What are you doing there?

Lewis Prescott (02:28):
Yeah, so I'm a QA lead. I joined like two years ago now and the startup space is kind of hectic, obviously building things quickly and getting to market as soon as possible. So yeah, when I came in things were kind of really chaotic, so I've put some processes in place, built out by team and we do technology in the home, so health technology in the home. So what we're trying to do is kind of revolutionize that industry. So the kind of social care industry in the UK is really behind the times in terms of technology and also it's really hard to find people to work in this industry. So we are trying to basically optimize it through digitizing the whole process. So yeah, I work in the, the QA team for that and yeah, we're, we're doing some really cool things in that space.

Lina Zubyte (03:23):
What does your regular day look like?

Lewis Prescott (03:27):
Yeah, so day-to-day I manage my team and I also work hands-on on a team as well. So do a bit of programming, whether that be test automation or working with the developers on some tests and then working on strategy. So looking ahead at what we're doing in the future. We're doing some cool things around becoming cloud agnostic at the moment and then working with my team to try and build embed quality within the team at the same time as delivering features very quickly at the same time as refactoring legacy code. So yeah, there's lots of interesting challenges that we deal with on a day-to-day basis, let alone the things that happen with working in an industry, which is... People have been in it a long time, the processes are embedded for a long time and lots of it's on paper. So getting them to, to move away from that is also something that we deal with a lot.

Lina Zubyte (04:31):
That's, uh, so common in Germany. So in Germany, if you want to cancel anything, you should often send a paper letter. And even once I was talking to my insurance and I had to upload a note and I couldn't because I did not have my login credentials. And trying to get it, they sent it to the wrong address in paper. And I contacted them and they said, yeah, like if you want to change an address, you should send us a letter that you want to change your address. And I was like, oh, come on. But you know, it's me, you know, we're already talking. It's like human to human, but there are so many of these cumbersome processes which you're exactly talking about in these like more conservative that were traditional domains which involve, I don't know, social services or registration or residence things. So lots of bureaucracy. Really interesting field.

(05:33):
So you mentioned that you would pair also with developers that you're trying to embed this quality and also release fast. There is this sometimes misunderstanding/misconception of quality that, you know, it slows people down. I hope we have enough of conversations to say that actually it can speed things up as well. So how does your testing process look like? So if we look at some companies, they just have, you know, E2E suite and that is what, you know, gives them feedback. What kind of tests provide you feedback?

Lewis Prescott (06:09):
So yeah, we don't have end-to-end tests. It's something we are working on at the moment because we're doing a lot of work to reduce our reliance on third party software. So we're building a lot more in-house, so we're doing more end-to-end testing around that cause we have more control. So currently what we have is we have lots of integration tests. So we basically base our testing model on the Spotify honeycomb model. So we really focus our efforts on that integration layer. So we have .NET core APIs which are hosted in AWS currently. And so yeah, we, we basically do the, the full integration of that API. And then we have micro front ends, which we have UI tests on and they mock out all of the integration points. So we have everything in isolation. So we're doing microservices basically model it like that, which comes with its challenges, but we can move quickly with that model.

Lina Zubyte (07:14):
The testing type that I associate you the most with is contract testing. So before I ask you also how it plays in your current setup, how would you describe contract testing? Because a lot of QAs, and not only when they talk about testing, they imagine usually the UI tests and joint tests as something that runs through the whole system. They may have heard about unit tests, but contract tests is not always a known term. What is it?

Lewis Prescott (07:47):
Yeah, so it's really difficult to explain what it is just using words without having diagrams. But an easy way to to picture it is it's like your integration tests where you are sending a request and getting a response. It's exactly like that, but you're flipping it upside down. So instead of you as the API making the request, the actual application that is using the API is making the request without having access to that request, they mock it out. So they send their request and say, I expect this response. And so they have that interaction, they send that interaction to the API and they say, here's what a request looks like from our perspective. And the API provider then replays that request on their side. So there's no actual interaction between the two. It's completely independent and everyone knows that they conform with the central thing that is called a contract. So that's contract testing in a few words.

Lina Zubyte (08:58):
Great. And what made you so excited about contract tests? Why do you like them the most?

Lewis Prescott (09:05):
Yeah, so started in initially another healthcare startup I worked at called Zava. So the CTO there, basically CTO or head engineering, one of the two, Eike. He basically said to me, okay, you're doing great things with the integration tests, but there's this other thing that we can look at. And we're currently building microservices which work across lots of locales, so lots of countries, so maybe we could set up these contract tests to test all these different scenarios. So he sent it to me,, a YouTube video which we can find and link to. Basically I had no idea what was going on, like it was completely over my head. I was like, there's this thing called the broker and you store your contracts and how does that work? And like how can you possibly have your consumer test your UI test interact with this thing?

(10:00):
And like I was so confused it took me until it worked ASOS where we were kind of split up into different teams and different squads where we had pure API teams and pure kind of front end focus teams. That's when it really clicked because I was like, wait a minute. There's this huge communication issue here where we are basically saying to another team, we're gonna make an API change and it's gonna happen on this date. And then everyone's leading up to it, all of the deadlines kind of conflict and this team doesn't have time to work on it so they can't upgrade and all this and we're, we're supporting multiple versions of the API. So that's when it really hit home for me. Okay, wait, there's this thing that I got introduced to before - contract testing - and yeah, my manager at ASOS was really supportive and like, go ahead and do it.

(10:51):
And yeah, it really makes sense when you've got those distributed systems and you want things to scale. And that's where it kind of really made sense to me. And then in my current context, it's like, okay, we've got lots of tests but they're not connected in any way apart from the fact that we have great integration tests which basically hold us all together, but if we make a change then we have to fix them together. So the speed at which we can get that feedback is something that really excites me about the topic basically.

Lina Zubyte (11:24):
And have you already introduced contract tests in your company?

Lewis Prescott (11:28):
So we're in the process of doing it right now. We did a few POCs with the teams that I've been in, where we had a chatbot. So we had basically contracts for the chatbot and then we are rolling out the scheduling solution. So as I was talking about earlier where people are doing scheduling on paper, they're mapping it all out and using a whiteboard to obviously kind of configure things. So obviously we wanna get technology to help with that. So we're building that and that's got so many dependencies and so many factors that contribute to it. So now that Pactflow have introduced the bidirectional, we can now use the existing tests that we have. They're written in playwright and we can use them to create our contracts and then we can use our open API specs to verify them. So we're doing contract testing. Well we're rolling it out at the moment, we're doing it, but we are using the bidirectional method rather than the traditional consumer driven method.

Lina Zubyte (12:32):
And what would be the main differences when you use this method instead of consumer driven? Because this is what I would imagine as well, like the contract tests for me always had this consumer driven contract tests.

Lewis Prescott (12:45):
Correct. Yeah, yeah. So the difference kind of is that you don't have to have one for the other. And the bi-directional, the kind of provider driven if you like, is having a way of verifying your documentation, right? Before we didn't have that ability to say, okay, here's the spec and does it actually work? We've generated the scenarios, generated the documentation, but does it actually do what it says it should do? So that's where the kind of provider side comes in. And that works really well when you've got things like public APIs where you don't have those consumers to be able to generate those contracts for you. And yeah, also those third party kind of interactions where you, you are just one side of the equation. So yeah, the difference basically is that where before you were getting the consumer to drive the contracts, now you can use your open API specs to verify your own contracts and then you can plug in the consumers into that. So it's kind of a dual verification.

Lina Zubyte (13:55):
Awesome. I think that would have helped so much if it existed when I first worked with contract tests because our initial implementation of contract tests meant that we had to have all the teams that have some services in the company on board for it to work. Yeah. So that was some kind of precondition that we had to have because if only one team is using, it defeats the purpose. So if I'm the consumer of some service of another team and I've read those contract tests and they don't integrate it, so what's the point?

Lewis Prescott (14:27):
Yeah, exactly. And that's one of the biggest hurdles, right? So onboarding is the buy-in and that's where the beauty of this bidirectional comes in, where it's like you've got all your mocks, you've got all your existing contracts, let's turn them into something that Pact understands so that you can do that verification process and not rely on those heavyweight kind of integration tests. And yeah, it's kind of reducing the effort, reducing the maintenance, but also making sure that you've got those contracts that live next to the code, which is really important. Really a great benefit. And yeah, selling it to, the kind of consumer side, when I was ASOS, it was the hard part because I was on an API team and so I wanted people to, to provide the consumer contracts. I ended up just doing it for them and saying, here's a contract, do you agree with it?

(15:23):
And it's like, yeah, well that's yeah, exactly how we interact with you. Okay, perfect. So I've put your name on it and now if the stuff changes, it's not my fault. And then the value definitely comes from when everyone's bought into the process. But you can get started like small and then just build and build. And now we can clearly see, especially with like the adoption of microservices being so high, is that each component having its own service means that the need for this is so much more and like it scales so well because you're using mocks, whereas end-to-end tests don't scale as well.

Lina Zubyte (16:03):
I really like the concept of contract tests. It is a bit hard to sell because likely it's hard to grasp it properly for a lot of people or to imagine it as well. But an example is when we had contract tests in one of the teams I worked in, we would get such a quick feedback on any kind of changes we did. So consumer driven contract tests in that case would fail and they would say, Hey, here, we expected a variable with this name, it doesn't exist anymore. And then you're like, oh, oh, we actually never spoke to them about this change. And we just did it and then like we would find it out. And very often the funny thing is when we talk about contract tests, one of the arguments against it is actually one of the benefits of contract tests I feel like. They say, oh yeah, but then like we will need to talk to each other more if you know, like the contract fails and it's like, yeah, but that's the point. Like you are talking like you should be talking even more if you do not have a contract test because contract test is an automated way for you to get feedback. And if you don't have it, you will need to talk even more, but then they're afraid because they all need to talk to each other.

Lewis Prescott (17:15):
Yeah. Completely been in that scenario. For me, the biggest thing about contract testing is not the tool, the biggest thing is the communication. And that's so important. And we as humans make mistakes. We forget to communicate things or there's something which we miscommunicated. I work with people who are based all over Europe, we're all using English to communicate, which is not people's first language. Like there's gonna be those kind of challenges. And so yeah, the communication is the most important part. And I think if, if we can use tools to enable that, then that's all good. And then on your adoption point, I think that's really key is why is it so difficult to grasp and why is it so difficult to understand the concept? And I think that's where we as testers or QAs really need to kind of educate and make sure that we're providing the best solution for the job. Because contract testing isn't the best solution for every situation. It's about having the best tools at our disposal.

Lina Zubyte (18:27):
Right? I like the idea of tools being a support for us to communicate better. They're just helpers. They're not there to solve everything. And for contract testing, the first time I saw it, we had our self-made homemade solution. So we did not use Pact then at all. Like we were just like using Docker images basically. And then we would just integrate your pipeline and we would fetch it from the bucket. Yeah. So there was the whole like self-made solution. You don't need to have like very fancy tools, even though they can help to make it easier. An interesting question that pops into my head right now is, you know, contract tests, it's sometimes not every QA knows about it as well and they may say, oh, e2e tests, you know, is the one that covers everything, right? We need to have it. You also mentioned that you don't have e2e tests like in that traditional way, I guess. So would you imagine products, where would you not need e2e tests?

Lewis Prescott (19:26):
Yeah, it's a really great question. So the kind of simple answer is that you'll always need those integration tests because when you deploy to an environment, you need to check that your configuration is correct, right? Contract testing doesn't tell you whether you've deployed successfully because there's no deployment. But I think now with infrastructure as code, with cloud providers who are giving you something out of the box, like AWS Lamba for example - is a snippet of code run running inside a serviced pod, right? So like you don't need to worry about the deployment process, you are just testing the code. Like messaging. Again, event-driven architecture is another great example is you are sending a message, right? Which is a small snippet which the other person then consumes. And so in those scenarios, I think there potentially is a scenario where you'll get to the point where you don't need those full end-to-end tests. But right now I think definitely not endorsing that and make sure that you check your integrations to make sure you've done your deployment successfully. But yeah, I I definitely think we'll get to that point in the near future.

Lina Zubyte (20:44):
Yeah, there was this controversial article, this is why I mentioned it. They said, oh, we have no end-to-end tests. And they said contract tests are the ones that we do. And you know, it's like, it's not a replacement for that, it's different, but it is true that it often catches some of the problems that e2e tests also may catch. Very often we catch what kind of problems? We think about the boundaries and let's say one service gave something else then we expected. And that is the integration point, which you cannot see if you're mocking the expectations. Having your assumptions. So contract tests could help out here. And I guess there are cases where you may not need the e2e big journey, but if you had testing done right in the lower levels. But that's so rare, that's really rare to see.

Lewis Prescott (21:39):
Exactly. And I think, yeah, as you were saying, some of testers don't know about it or haven't learned and got over that learning curve. And then yeah, often developers aren't really sure like what should live at what level of test. And so yeah, I think they're still learning to do on, on the subject. But I think, yeah, if you are working in a really mature team that has like you were describing, right, spinning up your environment as part of the, the process, then I think you can work towards that. And also if you've got great monitoring in place, if you great, great observability in place and you're not working in a high risk environment, then why not just release it a little bit and see what breaks and then quickly roll it back.

Lina Zubyte (22:32):
Yeah. Actually also, having this balanced testing and let's say having contract tests but not having that many of end-to-end tests... That makes the process much faster.

Lewis Prescott (22:44):
Definitely. Yeah. And that's what you were saying earlier is about the feedback cycle. And that's something that you can't really sell because people are happy with the situations they currently have. They're happy with their current feedback loop. It's until you implement it, you realize, wow, the feedback is so much faster and we can see what's happening across teams so much quicker. That's when you really see the impact and you can't feel that and you can't really sell that to people without them feeling it for themselves.

Lina Zubyte (23:22):
Well, I don't know how we can make the stakeholders or C-level feel it. So if I was a part of engineering team right now and I'd want to start doing contract tests, what kind of tips would you give for someone who just would like to start it in their company but the company does not have any contract tests? Where to start?

Lewis Prescott (23:46):
Yeah, so now I think definitely start with the bidirectional, provider driven. They come with consumer adapters. So if you're using technologies like Cypress, mock service worker or Playwright, then there's adapters available. So I think that's the easiest way to get involved. But then the other way is like if you're using messaging... Messages are a static form of documentation, which you can really easily implement contract testing because it's an asynchronous way of communicating already contract testing fits nicely and into that process. So yeah, I think they would be my two kind of scenarios of where to get started.

Lina Zubyte (24:31):
It's so nice to hear that you also mentioned the synchronous communication because the team I worked in years ago, it wasn't as common, it wasn't common at all to like have contract test for synchronous because nobody could understand how to get the response back, you know? Because it's a synchronous because we expected something to come back immediately. So you call an a p i, you get the response you validate, you know? Yeah. And at that point we had to do a proper research, you know, to find out how to do it. And now hearing that it's already there, it's great.

Lewis Prescott (25:04):
Yeah, exactly. I did a similar thing that like previous company where it wasn't messaging. We had processing on on a database that had multiple jobs. And so the output into the database was actually our response. So as you say, there was no immediate feedback, so we couldn't just get that contract immediately, but there's no reason why you can't have a contract, right? Because I'm sending this request to this data processing pipeline, so that is the contract, right? And then whatever's output at the end is the response that I expect. And so, yeah, we again set up a kind of self-made contract testing solution, which enabled us to pull that information from different parts of the pipeline. So I think there's no reason why you can't do it. And asynchronous is definitely something that I think people are adopting a lot more and realizing that everything doesn't have to be an API.

Lina Zubyte (26:14):
Everything is possible once you put your mind to it. This is the message I hear, right? Even if the harder cases you could find out how to do contract testing. Are there any kind of prerequisites for companies when they want to start off with contract testing?

Lewis Prescott (26:32):
Yes, it's good question. So obviously having some API documentation or documentation for your messaging, that kind of thing is kind of a prerequisite because you need to have that form of communicating because that's what kind of contracts understand. And then, we should talk about a little bit on my podcast, how to get started with a API contract testing. It's having those boundaries of understanding what is communicating with each other. Cuz sometimes that's not well defined and especially when you're moving from kind of a monolithic or you've got a central database or something of that nature, sometimes it's not clear what the boundaries are and who your kind of consumers are. So yeah, starting off that conversation, working out how you want it to interact and who owns each section, and then coming up with making sure that you build in that documentation wherever form that may be, plumbing that into the contract testing solution.

Lina Zubyte (27:42):
It's definitely essential to have good documentation or clear boundaries. So actually when I work as a QA in a team, that's my favorite activity to draw architectural diagrams of services we have because that supports any kind of conversations we may have about testing. So if I say, okay, this is like our front end, this is our backend, this is a queue that we consume, and then we get this data and it arrives there, and then I can draw these rectangles, sometimes different shapes. And then I see boundaries as say, okay, okay, this is our team, this is another team. Okay, we cover this part with unit tests how did we cover this one? Because we are reading this message and then this sparks all those conversations and saying, Hey, this is actually a contract test. And then you can see what's missing and you say, okay, actually with this team we have nothing with this one we have something. And then here we have only unit tests here, we don't have any alerts. And this also not only testing, it's also monitoring practices as you said, right? Having a good kind of feedback loop on also observability on production data as well on what's happening. So having good monitoring can be extremely useful. A good instrumentation can help us learn from the system.

Lewis Prescott (29:02):
Yeah, definitely. Really important point. And yeah, like what you're saying about this is covered by unit testing and putting that into some kind of diagram is really important because your code coverage doesn't tell you that. That just tells you line coverage and your integration tests kind of tell you, okay, these two points we understand are tested, and then you've got your end to end, which come from a more user kind of customer journey perspective. So you see it from that perspective. So there's that kind of middle ground of, okay, here's the user journey and here's the two integration points. There's that middle ground of, okay, wait a minute, all of these things need to communicate each other for that to happen. So what is testing that middle section? And that's what, yeah, I think as you say, if you put that in a diagram, it becomes very clear, okay, there's a gap in our testing here. How do we approach that?

Lina Zubyte (30:04):
Yeah. But not always we have diagrams or... Sometimes even architectures are huge. It's really hard to grasp them. It's it's hard to understand where, what is it doesn't mean it's still not essential.

Lewis Prescott (30:18):
And I think just take it in small chunks because as you mentioned, it's like, okay, here is our boundary for this small section. Okay, build a contract around that, start from there and then build and build and build. And then that's where you'll really hit home with the fast feedback and, and getting the value from it. I think we overcomplicate things at times, don't we? It's like, okay. And that's where you were saying about how end-to-end tests is the silver bullet in that scenario because okay, I don't really understand how everything's working together. I want to test it from a black box because that's what the user's doing. So here's all your tests and then you get to the point where you're maintaining your end-to-end tests for the rest of your life.

Lina Zubyte (31:05):
Wow.

(31:08):
50 years later, a sad QA engineer still sit with that suite that transfer 10 hours every day.

Lewis Prescott (31:17):
Nightly. Goes overnight. And then you wake up in the morning and then you want to go back to bed again

Lina Zubyte (31:23):
And oh no, sometimes it's flakey... and they failed.

(31:28):
Wow. That's also... may be a joke, but there's a saying that there's a bit of truth in every joke.

Lewis Prescott (31:34):
Yeah, exactly. It was, it was an exaggeration of the truth,

Lina Zubyte (31:38):
But it's really true. Like we do tend to over complicate and contract tests funnily sometimes give an impression that why are we reinventing something? We have our e2e test, it's enough, you know, you don't need this, but it's actually simplifying it. Maybe you will need less e2e tests if you have good contract tests and you will get faster feedback. You will not need to wait your 10 hours that your e2e suite runs to get feedback. So to wrap up this conversation, what is the one piece of advice you would give for building high quality products and teams?

Lewis Prescott (32:14):
Well, yeah. So I've been really lucky to work in some great teams and one of the kind of core attributes of that is the cross-functional nature of it. We're all in it together, pulling in the same direction. We're all sharing knowledge all the time. We're all working together, whether that be solving production issue or whether that being pairing on tests together. So that's a really important part. And the other section that I really put focus on is people. Building trust within the team and being open about what your strengths and weaknesses are, because we can't do e everything all the time. And, especially as a tester, we take on a lot of the kind of responsibility for releases, for making sure that stuff's ready to, to go to our customers or our users. And we need to take that pressure off and work together as a team and take the team responsibility for that. So that's the kind of my one piece of advice.

Lina Zubyte (33:24):
Great. It does relate to contracts with maybe human contracts and collaboration.

Lewis Prescott (33:29):
Yeah, I like that. I like that.

Lina Zubyte (33:32):
Thank you so much, Lewis.

Lewis Prescott (33:34):
Thank you so much for having me. It was real pleasure.

Lina Zubyte (33:38):
That's it for today's episode. Thank you so much for listening. Do not forget to check out notes if you would like to learn more about contract testing or contact Lewis, subscribe on your favorite listening platforms. And until the next time, do not forget to keep on caring about and building those high-quality products and teams. See you next time.