In this presentation, we will discuss our firsthand experiences of constructing agile testing methodologies through the utilization of BDD (Behavior Driven Development) and example mapping. Our aim is to minimize gaps and functional errors while enhancing lead time.
Jorge Luis Castro Toribio
Jorge is an agilist, Agility & Digital Transformation Lead Coach, QA Manager, DevOps Program Manager, and global speaker who feels passion for Testing Automation, Agile, DevOps and business agility. He enjoys research and learning about business agility and working with cutting-edge technologies. He has been learning continuously for more than 12 years and is still learning new ways to foster enterprise agility and team greatness. He has worked in several roles (developer, tester, IT program manager, Software engineering in Test manager, QA manager, agile coach) which helps him to see the big picture and operational and team members dynamics happening inside organizations. This experience lets him help teams design, build, and implement digital, DevOps and agile transformation strategies. He encourages focus on: People, Productivity, Continuous improvement, Innovation and having fun to enjoy success in our agile journeys.
Jorge has been a speaker in Agile and DevOps events organized in Peru, Canada, UK, Netherlands, India, Germany, Mexico, Colombia, Nigeria, Panama, Azerbaijan, Australia, US among other countries.
Find us here: www.agileleanireland.org
Please welcome George Castro, who is our guest speaker today. And George is naturalist agility and digital transformation lead coach, a QA manager, there was DevOps programme manager and global speaker who feels passion for testing automation, agile DevOps and business agility. George encourages focus on people productivity, continuous improvement, innovation, and having fun to enjoy success in our on our agile journeys. George has been a speaker in agile and DevOps event organised in Peru, Canada, UK, Netherlands, India, Germany, Mexico, Colombia, Nigeria, Panama, Azerbaijan, Australia, US and few other countries. And George is today dialling in from Lima. And you know, and the topic he will be speaking on is building out a testing and BDD example, mapping ways of working. George, if I miss anything, feel free to add the details that you would like people to know about you. And thanks for, again for spending time with us and sharing your knowledge with us today. Thank you, thank you very much for for for such a kind introduction. You know, thank you very much for that. No, no, I think that's all. So that serves very much for the invitation. I feel really happy to be here with you guys. With all these community. It's awesome, you know, to share knowledge from Ireland, for Ireland, to Peru, and so forth. It's amazing, you know, so thank you very much for that. Capital here. So I'm going to share my screen right now. We are okay. So let me share my screen. I hope it's working with this. Let me know when you can see my screen. Not yet. Yeah, we can see it now. Okay, great. Thank you. Thank you very much for that. Okay. I'm going to share my screen one second. Yeah, I think sorry. Okay. Okay. Okay, thank you very much, everybody. Thank you very much for your time. I hope you enjoy this conversation. Thank you very much for the kind introduction. My name is Jorge Castro. And today I'm going to share with you some experiences about building agile testing and behavioural remedy development or BDD, you know, from the trenches, you know, when I say from the trenches, I mean, from the field, I work as an Agile coach. So I work with agile teams, you know, with extreme programming teams, common themes. So I work with them with developers, with testers, you know, in the journey to build this in these new ways of working. So my experience is from the trenches from the fields. Okay. So they do have this session is taking advantage of NGS of accessible mapping technique, by the three amigos, by the way, those guys, you know, I assume that you, you know, those guys in my screen, those are the three amigos. So yeah, it's kind of funny thing. So, well, let's start. So well, you in this moment, you know, who I am. So because of the interaction. But anyway, this is my contact information, please feel free to add me to your LinkedIn accounts. That will be great. If you can add me to your LinkedIn accounts. And of course, if you find something interesting in this conversation, or maybe you would like to get, get more details about experience, or maybe exchange some advices, or some experiences about agile testing and so forth, you can add me to your to your LinkedIn accounts. And I will be very happy and I will be glad to share with you and keep in touch, you know, in LinkedIn, okay, so please add me to your LinkedIn accounts. Thank you. Okay. Okay, as I said, in this conversation, we are going to share experiences from the trenches about how to be good testing with working BDD with XML mapping, you know, and basically, to we use XML mapping to decrease gaps, unfortunate bugs and improve lead time, you know, as you know, in these agile ways of working, we have this we have this idea of sprints and short times, you know, each day we are we are looking to decrease the lead time, right? And deliver software faster and faster and faster. So, in this scenario, our times are short. So the idea is try to decrease the gaps on the functional blocks as much as we can you know, so I'm going to share with you some techniques that can help you with that. Okay, let's let's continue Okay, real life, right. This conversation is about real life is not about a book. I like I like read books, of course. I admire Lisa Crispin. I admire Janet Gregory as well. But of course, this is about real life situation. So once upon a time, there was a development team, you know, similar maybe similar to Europeans, you know, in your company, maybe similar, maybe, with some tiny problems, right? This development team has have some tiny problems, lack of team collaboration, you know, developers, hate tester, and tester, hey, developers, despite off despite of they, they were working kind of agile team, you know, your real life. Fortune backs a lot of functional back gaps in the functionality to do many manual activities, you know, a lot of a lot of manual things, maturation, manual functional testing, and so forth. And of course, as a result of as a result of all of that, unhappy stakeholders and unhappy customers, you know, so, by the way, those are tiny problems, right? They are not different, they are tiny problems. Yeah. Yeah. So I think just a real life for to Okay, so, to face those problems, you know, to try to sort the sub sort them out. We tried our first experiment, you know, but unfortunately, we failed in the experiments, which what we tried, we try to reinforce refinements, without a structured conversation, we had refinements, you know, as part of our were Sprint's, but without a kind of strategic without kind of structure right? At the beginning, our refinement was the kind of conversation basically, you know, the blue coordinator or the business, telling the stories and, you know, having a conversation about it. So that was a problem, that was something that we learn. Also, we try to reinforce development and testing techniques. Without a common driver, you know, you can basically what we did is we, we delivered a lot of training, but without a clear vision or a clear goal, right or clear OKR, you know, so we did a lot of training, training about the LTC training about EDI trainings. But without a clear vision, a clear goal. That was a mistake in the beginning our first experiments, I'm also score, automate everywhere, right? Automation automation, I Sorry, sorry about that. Let me say this. Okay. We trade also a lot of automation, automation, automation for integration automation testing for, for functional. We did a lot of automation without a strategy. Okay, so we fail. In this first approach, we fail. But of course, we are idealists, and we try to do things better, right? In the second sprint, we try to move forward and do new experiments. Okay. So this is what we try, you know, in our second experiment, we try these are combination between agile testing mindset and practices, plus BDD, Behaviour Driven Development. You know, as part of testing, we tried, first of all, shift left testing. So we try to move the testing to the left, you know, we do need, we start, we start with these with unit testing, and test driven development at the beginning. And also, we tried telling you, we tried to apply Ling mindset and lean practices in our testing cycles, to reduce the waste of our testing, we tried that. And I think also, another important aspect of that is test as a team, right? In our case, we try to say, okay, no matter no matter if you are a pro gardener, or if you're a scrum master, or you are a developer or a tester, we all are responsible of the quality of our product. So with that mindset, everybody in the teams has the responsibility to test you know, to test the product, you know, and take care of the quality that was another approach. And also we try beat behavioural driven development, with a discovery formulation and automation approach, right. As part of discovery, we tried, we did a couple mapping. That is, that is why we are here, right? I'm going to share with you how we apply example mapping in our in our project, about about formulation and about automation. We use gherkin for that. In automation, we use Selenium and cucumber and serenity BDD. And for that, not for the automation part. I'm going to share with you some examples of that. So you can take an idea about how we how we apply this in our testing. Okay, so that was our main approach in our second experiment. And strategy, right? This is something quite important. I know that you know that because you are, some of you are people for working in Agile, right? So you're maybe I assume that you are scrum developers or you are a scrum master. So you or your coaches maybe? So our first approach was, we need to define our strategy and our objectives. So this was our objective our OKR, right? improve quality and customer, the developer experience, that was our goal, you know, for that goal, we have three key results. Number one, reduce functional backs, gaps. Number two, increase the MO review, NPS, you know, the, the NPS of our customers in the reviews, and reduce waste, you know, we had the goal to reduce all the waste from our testing cycles. So as you can see here, our strategy and our key results, were related to our mindset, right? fizzling is testing, testing as a whole quality of the whole team. And of course, applying BDD techniques, you know, this is part of the strategy and, and pm people, this is quite important, right? Because it's quite different when you take a training and do stuff, it different from half ambition, right? And now you go, you know, I think it's sad if it's different if you apply this in this way. Okay. First of all, right quality as Three Amigos team sports, as I said, we try to use this gel testing mindset to make the testing responsibility, the quality inside the team, you know, everybody is responsible for the quality so we work together as a team to do tests to check the quality and so forth. So that is why we apply this idea the quality has three amigos team sports, Product Owner business testers, and coders does a year, it was not it wasn't easy, of course, but at the end, when when you as a team have the same OKR no matter if you are a tester or business or Scrum Master, you can make people working together you know, working as a real team, right. So, so, that was the first approach. Second one was the Teslin automation mindset, right. And this is part of the testing approach, basically, what we did is we use a flow framework for that flow Franco, you know, this is a framework to is a framework to set up your value stream, you know, I am identify some, in this case, we use the metric, the flow efficiency, you know, and the image here you can see the entire development process in one of our things. So, in this process, we define the waste or we define, we didn't define those, those stages, or those times that were are not productive to our cycle, you know, so we define, we define these wastes. And we analyse the situation, we prepare some initiative to remove the waste. And we try to improve our flow efficiency, right? So that was one approach, you can use it right flow framework with flow efficiency metric to apply this link in your testing cycle, right. This is an example of what we did number two, okay, and number three, its behaviour Driver Development with example mapping, right? I know that some of you maybe know what is BDD and some of you maybe don't know, what is BDD, but Behaviour Driven Development is the techniques, you know, to develop software, it's where you first, you first, you create your scenarios, based on your behaviour of your functionality, right, the behaviour of your users using your software, it's this idea, right? You are going to create your scenarios. Working together with the three amigos with business with development with testing, you're going to define the scenarios, the behaviour of use of words, working together, right, that doesn't mean approach. We have Behaviour Driven Development, Behaviour Driven Development, because at the end, those behaviour or those scenarios are going to drive what you are going to quote, you know, so So that's the idea, right? Your code is going to be driven by your behaviour of your application of your business, does it have behavioural driven development? So I'm going to share with you this example of a step by step about how you can apply behavioural driven development, okay. Okay, step number one. Sorry, before that. So, sometimes people ask me a Jorge, but this is a technique right. But where I have to apply this technique, right. In our case, we apply this technique in our refinements, right? More or less, we have one refinement per sprint, in sometimes depending on the number of user stories of functionalities or features that we need to work on. We have to refinements, right. But more or less, you can have one refinement, depending of your maturity and your your complexity of your software, you can have one hour refinement or maybe two hours or one hour and a half, depending of your of your situation, right. But first recommendation, you need to apply this or that is my recommendation, you can apply this technique in your refinements. So, so, see if you right now you're having your refinements in the way that we had at the beginning, in the first experiment, where we had basically conversations, you can use this approach to structure your refinements. Okay. That's the idea. Okay. Number one, in defy the user story, right? Of course, in the, in the left, you have the steps. In the right, you have the real example of, of the technique, putting in practice, right. So, so find account statement is a user story, you know, a simple user story from a bank, you know, because I work with banks right now. So, yeah, so that's it. Yeah. So first steps. In defy that you say the story. Of course, this is more or less the responsibility of the product owner of the business guy in your team, you know, the people who know the business, you know, this is something that she needs to prepare before the session, right? She needs to understand what she needs to pick up the user stories or prepare the user stories or kind of raffle you for the stories. Bring the user story to a session, right? Define the user stories number. Step number one. Okay, step number two, and this is this is in the session, right? In the session, where with the product owner, with the developer with a tester with a DBA with all the people that is part of your team, right? If you if your team, for example, right? In my case, some of my teams has people from cars, developers, has testers has designers, Scrum Masters, also also DBAs you know, you need to both of them in your refinement session or in your example mapping session, right, all of them. So step, step number two is about least that business. Sorry. Sorry, again, one second, please. Yeah, okay, number two, step number two, sorry. Here. Yeah. Okay, step number two with all the guys in the session as I said, first least the business rules right, the simplest criteria, you know, you need to work together okay, you can take the story you need to ask question, understand the user story of the requirements, you know, or the needs wherever, whatever the name that you want to call call them, user story repairman needs or whatever name that you want to use, you know, depending of your your approach. Step number two, you need to talk to where with your team, the whole team together and least the business rules, you know, for example, right? If the user stories find account statements, you need to ask okay, what are the business rules that support this story or that I need to work on or need to apply to, to develop well this user story, okay. So, there are some examples, account statements can be created by any type of accounts, searchable by date range search by account number, the account statements for giving finance, financial year and so forth. So, you need to discuss with your team, what are those business rules, if they are similar to some of the criteria and actually they are pretty the same, you know, but this is part of the exercise you need to work together on that maybe you maybe you are asking to yourself a but this is this is something that the product owner should work before the session. Yes, yeah, that is the ideal case, you know, but as you know, as some of you know, this is not happening usually right. So, that is why you need to take advantage of the session, you know, and in the sessions right to discover these subjectivity or, or these business rules, or step number two, okay, at least, that is my recommendation. If you have a mature protocol, that will be awesome, you know, but this is a kind of a step by step process. Okay, step number three, with this business rules, you need to discuss and agree on right examples, you know, similar to college, right? I don't know in UK, but for example, in my case, when I was in college, when the teacher said okay, or the professor sorry, said okay, this is a kind of a exercise or this is a example of these soft words or example of these concepts. My first question is a professor, can you ask, Can you give me an example of that, right, in real life, okay. You can apply this functionality in a bank in the customer service approach, or when you have some claims about your product, whatever, right? A real example of that. So, either same situation you need, we need to ask together for examples. Okay? That is why this technique has this name right example mapping, you know, so you need to ask per each rule, you need to ask with your team, you know, okay, what are the example of these rule account statements can be ordered by any type of account? Okay? Give me examples like what about if I am a client, I am a user, and I need to follow these rules, what are the examples? Okay. One example can be user one, you know, a generic user user one source account statement, but by saving accounts, for example, this was the kind of example right of the situation. Let's take another imagine that business rule is search by account number, okay, that is the business rule. Okay, what are the examples? User user one search for a particular account statements? For a particular one, you know, a specific account number. Another example can be user one does a search where the account number doesn't exist? For example? What is the behaviour of the application in that example? In that situation? You know, so those are the kinds of questions you need to ask with your team. Okay. Maybe at this time, you're asking me okay, but when, when I'm going to be okay, about the number of examples, you know, basically, basically, Team these are kind of agreement between product owners, testers, and developers, when you when you feel that we the example that you have, we are okay with that, to start development? That's great. Let's do it. Please, please don't think too much. Just take the samples. And please, Coach, you know, it depends on your team agreement depends on how do you feel that you understand the entire user story, right? That's my recommendation, at least, there is no there is no kind of GPT chats, were kind of robots that are going to be the answer for you know, you need to basically have an agreement about it. By the way, by the way. Nowadays, I'm using GPT chat, to develop scenarios and examples. And it's working well. Maybe I'm going to prepare a nerd talk about this, about using behavioural driven development and GPT chat to prepare scenarios. So but this is part of another conversation, but you can try if you if you like, it's working. So it's working well until now. Okay, let's continue. Okay, that's step number three, step number four, of course, as any any kind of meetings, there are, there are some questions right, that maybe we cannot resolve, right? We cannot we cannot answer at the time. So please write write your question, write write equations. That is quite important. Because more or less, depending on your equations, someone needs to take the responsibility to answer the question later, right. It can be there, it can be the product owner, it can be the scrum master, maybe it can be the developer, it can be a tester, you know, so please write your questions, it is quite important. Step number five. And I think that that is something that if you read books about behavioural development, maybe you're not going to find in those books. Step number five is select the samples or scenarios with you know, unify the positive scenarios, and often defy that negative scenario. That is that is important, because remember, this is this is scenarios are going to be part of your of your development, they are going to drive your development and your testing. So it's quite important to differentiate them between positive scenarios. And negative scenarios are examples, right? Especially for automation, especially for automation testing. Okay. And finally, number six, is a combination between example mapping and a slicing, you say story slicing, right? In my experience, when we apply this technique, in most of the cases, we identify that the user story is to be, you know, is to be he's too complicated. For example, if you kind of look if you have a lot of green posits. So I mean, a lot of examples and a lot of business rules. I think it's a good idea to use a slicer right? Because develop that story is going to be too much complicated. So I recommend you first analyse and use a slicing right? So for example, in this case, we apply slicing in this way, right? We create a user is totally a new user story using these three business rules, and then get to more user stories with to obey the rules in Long story. And finally the other one with with with another another news story with only one reasonable, right? It depends on your case, but that is my recommendation. Take advantage of these using of course slicing as well. Yeah, it's a good practice. Okay. And basically, this was our approach. As I said, you know, if you remember, when I when I started this conversation, BDD has three stages, right? First of all is formulation first or either discovery right when we discover the behaviour. Basically, using XML mapping, as I mentioned, right? We've covered the scenarios, we've covered examples, we've covered our business rules, we've covered the the slicing user stories and so forth, that is covering right. And the second stage is the formulation right? We need to formulate our our user is sorry, our A scenarios are examples. We need to formulate them using some specific language in this case gherkin. I recommended use gherkin because, you know gherkin is a kind of nice language where you can use it to structure your examples in a way to business product owners and tester can understand you know, the given windowing approach. So I recommend you to use gherkin to formulate your reducer historys as part of the Cambridge German development. And finally, I think so, yes. And finally, you know, the automation part, you know, as part of Behaviour Driven Development, fears users, you discover the scenarios, you discover the examples, then you formulate your examples in a kind of language. So, anyone in your team can understand the example, using given when them from gherkin, and then you need to automate your examples. So, that is quite important guys, especially, if you are also adopting DevOps, or code or continuous testing ways of working, because at the end, as you know, you want to take all the testing, or the most important testing to your pipeline, right to your DevOps pipeline as part of your DevOps adoption. So yeah, in this case, you can use Selenium for functional testing, cucumber, Serenity BDD, for example, as this approach, so, you can have, you can apply this entire strategy for your functional automation testing, right, as part of, for example, integration, right? In our case, we had, we had a big problem, because the 100% of our regression testing was manually and it takes more or less, two weeks, or more, to do the entire revelation, right, which is a lot of effort, people doing a lot of manual stuff. So that is why we try to apply this agile testing approach with BDD for our functional regression testing, you know, as part of our approach, okay. And something that is quite important, no matter if you are, if you're doing Agile, or DevOps or a combination of both feedback, right, you need to get feedback about your experimentation, about your testing about you about how you are working, right. So that this is an example of how we get testing from our sorry, how we how we get feedback from our testing using Behaviour Driven Development, using in this case, cucumber, you know, so we have these kinds of reports, where we get data about our testing, our testing, our test, fail our bugs, and so forth, you know, that is quite important is information about your, your testing our audio backs, it quite important to fix the problems, right? fix the problems in an easy way in a quick, quickly way, you know, get feedback. Okay. And something extra is that as if you remember, at the beginning, I said that we apply this approach, you know, what we defined and OKR. One chiar was about that MPs, you know, of our clients during the review. Do you remember that? So about that about that point? We use this gherkin approach this example mapping approach for our demos and reviews. So basically, in the demo, we present our examples, you know, with gherkin, because as I said, right, getting is a language that everybody can understand business technical or non technical people. So we use this approach to prepare our to facilitate our reviews and demos with our clients, you know, and actually, to be totally honest with you, they work very well you know, because at the beginning, they are they the business understand what are the the example of a real user using the application and then they can say they can see also the behaviour of the Oh That example in in with real software, right? So that was something very good that it worked well. I know that maybe you're asking a shortcut, but so that means that you run your and your demos in non production environments. In some cases, yes, we run some the most insert in non production environments. But you can run your demos or do reviews in production environments. Also, of course, if you want to do that, you need to apply some techniques, right? These are the management you need, you need to data masking your data, depending of your of your how, how important is your data in your production environment, you can use data masking, in your staging environment, for example, you know, to run your demos and reviews. And of course, you can also apply a B testing Canary release a feature toggle, depending of your approach, right of your DevOps DevOps approach in production. So you can use, you can apply this technique as well, to round your demons and reviews in, in production in a safe environment, right? You can do that as well. Okay. Number eight, as I said, Now that you have all your automated tests, write your functional tests with gherkin with sample mapping with gherkin with Selenium, and so forth. As part of continuous testing approach, you need to create a strategy to put those tests in the pipeline in the debug pipeline, you know, because it's, this is part of our agile testing approach in our case. So this is an example of a strategy that we apply for our pipeline, right? We have we had in our pipeline levels of of testing, level one, level two, level three, and so forth. And in each level, we have some specific kinds of testing, right? In Level One caffine. Testing in level two, you have API testing, integration testing, some part of UI testing as well. In level three, you have API testing, also, UI testing, and level four. In level four, we have our, our functional regression testing, automated functional reason testing. So we apply these, these approach for our, for our, for our testing, it's already in the pipeline. And of course, we move our automated tests to the pipeline as part of continuous testing, okay. Okay. Okay, just got just finished, guys. Thank you very much again, for your time, just to sum up by the BDD, sorry, I should be the which is in Spanish. BDD, you know, we Ansible mapping can help us to improve team collaboration. Please try it simple mapping. It's quite awesome. Having the guy discussing the requirements, the developer, the tester, the business, it's really nice experience, improve team collaboration, improve quality, you see that right? The entire idea testing approach, developer experience? Well, because developers want to share, you know, their point of view about the examples of their behaviour or the functionality, right? They like coding by also they want to participate in the creation of the product, right? And build up customer trust in the product. Right. So I think this is kind of good backlash if you want to improve quality team collaboration, developer experience, and customer satisfaction or customer, NPs or customer trust, right. So yeah, that is the sum up of this conversation. And to finish books, I recommend use those books. Maybe you're asking a Okay, those are many books. Please give me one. Okay, if I can recommend you only one to me, but I will I will recommend this behavioural discovery, explore behaviour using examples. I recommend you these books. Maybe you can read this book at the beginning and then you can read the years okay. I recommend discovery exploring shaper using sample but the rest of the books. Sorry, Lisa. Sorry, Annette. The rest the rest of the book are awesome, you know, are awesome as well. But if you ask me for to read one book at the beginning, I would recommend this one. Okay. Remember that we all have dreams. So please help and share more. Why I'm here, I'm here because I like sharing. I like sharing knowledge. I like building communities, you know, sharing experiences. I know that we can learn from anyone. So please, keep your dreams help and share more. So thank you very much. I really appreciate your time. Thank you so much. Thanks very much for your time. So it's a talk I asked. I asked you before and she said George, but then I hear you're saying Jorge. So how do you say your name Hi, it's Jorge, but I know that it's difficult in some time, so I prepared George, you know? Okay. All right. Thank you very much for your time and sharing that example. And we have time now for some questions. So please, you know, this is a self organised group, feel free to unmute yourself. And if you have any questions to George, feel free to ask. Okay. No questions. This is completely No, not my domain. Normally, I can rescue. And I can I can ask some questions, but this is completely not my domain. So I am I cannot rescue here. So, anybody tried a similar approach before? I don't know. Maybe I can do something about this. I know that as part of my talk, I shared with you some IgG testing and continuous testing approach, right, which demands knowledge of this automation maybe on pipelines and so forth, right. But remember something people as part of behavioural driven development, the first step is the discovery part, you know, and in the discovery part, there is no technique, sorry, there is not technical stuff, right? Basically, facilitation is basically having a discussion in the failure scenarios in defining business rules, having that conversation and so right. So if you are maybe if you're in a group, right, you are, you're someone from the non technical, non clinical, and my recommendation in that scenario is that you can take BDD, you know, but focus on the Discovery parts, you know, the example mapping technique that you can apply as part of your refinements, for example, right. So, so that was my recommendation, because a lot of people asked me, Hey, Jorge, I am not a technical people. I am a scrum master or, you know, an idealist and I want to play this. So I recommend you to focus on the Behaviour Driven Development, the first stage, the discovery parts, no. Thank you. And, George, we have we have a question here. For a company with no automation in place and no automated pipelines, what is the best approach to kicking off the business users and get buy in? Well, in my experience, you know, my opinions, the best way to get that sponsorship, or good, you know, the support from business is with results. You know, with results with evidence of your work, you know, you can you can talk about Agile and Scrum and Kanban all this stuff, right? But you need to present results, right. So, in my recommendation is, you can try Behaviour Driven Development, you can try agile testing, and in both your product owner or business, in your example, mapping sessions, right. So, first of all, in that way, your business can understand how you work, right, in terms of development, right. So they can understand how you work. And also he can share with you the business rules, if he can be part of the discussion, you know, about creating the examples of the soft words. So that was that that will be my recommendation, right, in both your business in your example mobilisation? And of course, of course, in your demo demos and reviews, please use this right? Show them the value of this right, in terms of quality, in terms of speed, right. Something that was good, in my experience was having the business watching the behaviour of a real customer using the application, you know, and something that is, is extra good, is that you work with the business to design that. You know, I it's a real good experience, right? That is my recommendation. If you don't have any pipeline, or any automation stuff. It's right to define your scenarios using example mapping. That was the first approach. Okay. Thank you. Thank you. All right. Yeah, they thank you very much. Thank you. Do we have no more questions? Yet? We have one more. Do you see the product owner or the business analyst as owing the example mapping or do you see it as a shared responsibility? Good question. Actually, actually, that patient is from the trenches. This question is from the trainer from the field. Okay. I'm going to be totally honest with you. If you want to Don't get success with this approach, at least at the beginning, we need to take the onus, right. And when I, when I say we do, right, if you're no matter if you are a developer or you're a tester or you are a scrum master, or whatever is your role. If you like this technique, if you like it right, and you want to try this technique, you need to take the ownership, you need to run the session, you need to talk with your scrum master or whatever Right? Or you need a we need to change our refinements, we need to apply this technique, sample mapping, this is the benefits, we can use this for automation as well for DevOps as well. So you need to sell idea, right to your team. And you need to take the the responsibility, right? If no matter if you're a developer, or tester, or scrum master or whatever, at least at the beginning, at this at the beginning. And after some iterations, you can change that right? You can say, Okay, after four sprints, I think it's time that our scrum master takes the facilitation. Great, you can do it maybe in for two sprints more or less remotes, and then the product owner can take the ownership, I think you can rotate this gonna rotate the facilitation of the of the order of the session about that, about the responsibility in terms of functionality and business needs, of course, in this case, the product owner has a bigger responsibility, right. But to bring, you know, to bring the what the business needs to the session, I think that that is the person right. But in terms of the facilitation in terms of their collaboration, I recommend to first take you take action, you promote this in your team, explain the value explain the benefits, run decisions coach or the teacher beam, and then you can rotate these this row right, the facilitation, that would be my recommendation. Thank you. And we have another question. Great. Yeah, we have actually a good fuel a good fuel. Yeah, it's a record. It's a record me PATIENT Wow. We have a very vibrant community. George. So, you can you know, I was sure we will get some questions for you what is a good way to help the stakeholders to understand the importance of automation you know, automated test, most stakeholders see it as an unnecessary cost and only opt for manual testing, which is high effort for our QA engineers, okay. With a resource, I think first what they recommend you is try to try to analyse data with data and also I recommend you to have conversation with your customers with the clients with your pro coordinates for business and you know, to get information about what are the main constraints about the software right about the delivery, try to gather information more broadly, they are going to share something like okay, you are slow because I want my product a production in three weeks and you are you are you are using four weeks or something like that, you know, so, you need to get that fit that feedback right about the satisfaction of your customers of your business in terms of your delivery, right, it can be quality, it can be velocity, it can be lead time cycle time whatever right. So, fifth recommendation, try to get that information, second recommendation, get data from production, you know, get data from production, the vaccine production, how people is using your product in production, the the time that your application was down in production and so forth data from production, combine this information, that feedback from your customers from from conversation with the data from production, combine them and analyse what are the main constraints in terms of business end? Okay, most probably, you are going to get information about or you're going to get some ideas about okay, we need to improve the quality, we need to reduce the number of tickets about complaints, for example, or we need to improve our velocity in terms of new features for our products and so forth, right. So, that is my approach you need to define your OKRs or your goals using real customer and product situation, from production and for your delivery. And if you work in that and you improve just one key er, you're going to you can present the results to your customers, your customers are going to understand the code because they they feel a problem. They have the problem and you're ready to present a I can I could reduce the number of bucks from I don't know 10 bucks per release to two bucks per So that is the way that you need to convince your your stakeholders with results, you know, will result in terms of quality in terms of velocity in terms of efficiency. That's my recommendation. Thank you very much. And we'll have one more question. And, Bruno, I see you, you went on the camera? Do you want to ask your question? Yeah, that's fine. So hi, George, thanks for your presentation. So my question is regarding like, how we can apply the like BDD and everything to explain it to non functional requirements. So for example, if if I'm working on a back end system that has like scalability or security requirements, I like how can we approach this using the techniques that you're explaining? You can apply, you can apply them? Thank you, thank you for your question. And you can apply them as well. Unfortunately, in this case, Giganta 30, some odd actually, we don't know, unfortunately, in this talk, I not prepare these kinds of nonfunctional scenario, right? I can pick up some, you know, okay, let's do that. Right. Maybe I can share, you know, with the, with the organisation, a real example of how we can apply example, mapping with a non functional requirement, right. But you can use it as well, for example, in my case, we apply that, for example, for the integration part, right, for API's, you know, for DPS, as you know, when you define API's, you need to define the methods, you know, when you get information and when you bring information, right. So we apply, for example, if someone mapping with API's and integration path with Ray services, SOA service and so forth. That is one, I think the approach is similar. But of course, it's more technical, right? So maybe you can apply this for a kind of, in my experience, at least at the beginning, you need to involve your technical testers, if you have it. And your develop developers, right. And you as a, as a developer, or clinical leader, you can be the product owner, or sorry, you can act as a product owner, but more as a kind of technical owner, right? Or something like that. Right? And you can present Okay, guys, this is our API, those are the services that we need to prepare. So let's about okay, the, the scenarios, right this scenario, so examples of these API's working in production, you know, so then you can analyse the number of transactions. What about security? You know, what about if the server is down, and you can print, you can generate the this conversation, right? So if you're okay, Bruno, after this session, I'm going to share with the with the organisers and examples that, you know, they're going to share that with the rest of you after the session. Brilliant. Thanks very much. So please make sure you want to sign up to our newsletter, because we will be distributing all the materials via via newsletter. And George, also, if you would like to send us your presentation, we can also share it with with the participants. Yeah, I'm going to share my presentation. And most what I'm going to add this extra example. Bruno's question about using XML mapping with non functional requirements. Perfect. Thank you very much. George, thank you for your time and sharing your knowledge and practical examples. It's always nice to have a case study not you know, academic book base theory, but actually something which which was implemented and we can learn from your success or, you know, setbacks, learnings, learnings as the word I was looking for. So thanks very much for investing your time in our learning. And enjoy your day. And thanks very much again.