Product Discovery (concepts, tools and practice) with Ricardo Colusso - Agile Lean Ireland

Product Discovery is a mindset and a continuous process for validating hypotheses and gathering empirical evidence on the value of new product features BEFORE developing them. When applied properly it can reduce the risk of losing time, human talent, and money in functionalities that, when implemented, might not be as attractive, feasible, or profitable as expected.

In this session, Ricardo Colusso will describe the key elements of Product Discovery, share some real-world examples and experiences,and then we will work in subteams for applying some key concepts and tools.

Ricardo Colusso is a Senior Product Owner and Product Management coach with 25 years of experience working across Latin America, the United States and European countries.

Find us here: www.agileleanireland.org

Transcript

So welcome everybody to our second alley meetup this year, we feel very privileged to be in position to serve you with amazing learning opportunities. And we are looking forward to hear from you, what other speakers would you like to hear? And what other topics would you like us to cover. So if you have any ideas, any special requests, please put it in the chat. And we'll look through it later on. And we'll do our best to reach those speakers and cover those topics. We are very excited you could join us today during the session. And you can see, it's a really big meetup with truly global participation. And for, you know, everybody's information that Meetup is recorded, but also it's streamed across a few location of JP Morgan offices. So welcome, everybody. And let me introduce Ricardo Calusa, our speaker today, who is no stranger to ollie, we've been working with Ricardo to deliver amazing talks to you for the past few years. And Ricardo is a Senior Product Owner and product management coach with over 25 years of experience. And he's been working across Latin America, United States and Europe. So he's well travelled, and he speaks multiple languages. And we would like to welcome you all to product discovery session. Excellent. Thanks for such a nice presentation. We will have, as I mentioned, to the ones who got into this meeting, first, we will try to make the best use of our time. I repeat that I know that most of you probably were in meeting stakeholders, or are in in the middle of different online meetings. So we appreciate a lot. You're lucky this time for this experience for this session. So first, what I will do is to send a request a phrase in chat to complete I say, today I'm connecting, connecting from an I will answer greystones Ireland, or we are where we are connecting us Poland, Ireland, UK, Argentina, Seattle, NAn De Mayo by Dubai. Yeah. Dallas, checking if I was lying. We have fever covering all parts of the world today tonight, now that we know we're connecting where we are, or at least how much of the globe we are covering, we are ready to start. So this this idea of price discovery comes from the experience of having a sometimes a deployment features and software in production that sometimes is not used. It's become it's becoming wasted, because people don't do not use it. Because it is not attractive, because it is not profitable. So, this is why I got into the world of product discovery some years ago. So, there will be a few slides with concepts and then there will be two instances of practice on price discovery, so I will go into slideshow. And here is the agenda for today, we will talk about price discovery and product delivery then why we are talking about and we are thinking about being in product discovery mode mode and then a what what is the product discovery mindset. Then, we will talk about how to move from product features to Product A to to outcomes to metrics of impact and outcomes. And finally we will see an apply product discovery toolbox. We'll see about that later. Okay. Then first distinction is between product discovery and product delivery. price discovery is related to build the right thing is like let's explore opportunities do experiments to prototypes before developing new new features or improving features or even doing a new product and products discovery movers to measure outcomes. On the other hand, product delivery is related to build the right build the thing Right. And in this side of product delivery, it's like we already have a good practices and good ways, agile ways to do it. Like, we know that doing iterations slicing, incremental delivery, delivery of features, and early getting early feedback of work already done. It's something that we were doing for for many years. So, today, we will focus in the left side is like how we could ensure that we are not facing risks of doing things that will not be attractive, will not be fit will not be feasible, technically, technically feasible, or will not be profitable for our companies there in the Agile principles, we learn that working software is the primary measure of progress, but here in this context of product discovery, we will we will say, we are not fully sure should work in software, be our primary measure of progress, perhaps do not, because before investing on doing something new, or improving or developing a new feature, we should always need to answer these questions, are we going to solve a relevant problem? Our solution will be useable, attractive? Do we have the technical knowledge to implement our great supposedly great ideas? And if we put new software or improved features in production, in the word of in the, in the hands of customers and users? Will it help that to meet our business goals? So before investing time, money, and talent, human talent, let's try to ask these questions. And for there, we first need to think differently, it's like, I don't know, if for many of you, it happens that many times we have backlogs and we would like to do, the more you do, the more successful we are and we measure how our teams work based on how many story points or how many features they put in production. But in the new product discovery, we have different minds. First, we are we think that a in a in a user centred design, we would like to have to move and evolve our product teams, to teams that have direct communication with end users and with customers. It's like we don't like to have product owners who are in the middle between all the time in the middle between what customers do and what customers need and a it covering developers testers and people who deploy the software a learning a second hand what this users in the real world needs. So, as much as possible, we are trying to create micro activities that make possible direct communication. So, the development teams are able to see in action or have and to get analytics and understand the usage of what they are building or what they will build in the case of if we create prototypes. So and we also try to validate problems and validate solutions creating them with our users. Then second value or second part of the mindset is learning and collaboration is like we need to create like an environment where people are safe and able to experiment and consider that sometimes hypotheses will not be validated and things that perhaps were like considered great ideas perhaps will not be done and create a shared understanding of what are the problems and what are the solutions we will apply to those problems. Then we will need to think on discovery of something that does continue is continuous across the all the product lifecycle. It is not something we do once a year at the beginning. I have a new milestone for the product. And finally, there is something related to intellectual humility is like accept that if we do prototypes, and if we do hypothesis validation, using the toolbox, we will see later, there will be one times where we will not be right. And there will be times where we will go and detects that we were not a we didn't have knowledge on what what was going on and what our users and customer meet. So there is a phrase that is, we don't know what we don't know, it's like, perhaps you will usually go to interview customers and to show them prototypes for evaluation and testing. To, to learn what we don't know. But sometimes we discover that we don't know what we don't know. So it's like, we need to be extremely humbled with that. Then here, there is something important about product discovery, that is, the way the metrics we will use for our for ourselves and our teams is like, we would like to measure progress based on outcomes. In this picture, you see, a this girl could throw the arrows, and there are several arrows here on the floor, that is like they were they were a waste because they, they the arrows didn't hit the target. What we will like to do in product discovery is to do think, a little more or a lot more before throwing the arrow to the target. So for doing that, the key thing here is before doing any development, try to gather a data and validate your hypotheses and answer the question. Therefore, questions we mentioned before, don't make assumptions. First step in product discovery is just a move from thinking on product features that are outputs to move to think on product outcomes, and business results. So every time we think there is a good feature, we could look, let's see if when used by customers or users, they will produce any change in the Arbic behaviour or some problem that will be solved. And then after we solve a problem or changed a behaviour of a user customer, let's think on the business results we will get. So think of this before deciding to put that product feature in our backlog. So I would like to know how we are doing. Please turn the cameras on tell me if we are doing okay to like thumbs up or not. If I'm going too slow or too fast. Let me know. Alternatively, you can use the emojis so if you don't want to or can't switch on the camera, please feel free to use emojis which comes up okay, let's do a reaction break reactions break to hit the reaction button and you have several icons there and use them and see. Okay, we got some thumbs up moving screens. Yeah. Thumbs up. Thank you. Thank you. Thank you. Now, it will be the time for you to start working. Okay, share screen again. But I will move to other place. We move to the practice part. So here there is I included a couple of examples on how this activity works is like how we can think on a product feature. And then it's our outcome and then if it's busy business results. So let's take for example, Google Drive. Let's imagine that we are part of the Google Drive product team and we have And feature idea that this integrate the chat GPT bot into Google Slides. And the outcome, what we expect is that users will be able to tell about how they, their slides should look like. And Google Slides will build the slides. But we think that that will generate an increase in Google's lies market share by 20%. So then, in the second iteration of our work, we will start asking questions are valid, and we'll see how we can validate this hypothesis, that A, we will be able to change the behaviour of users and customers and that we will get business results. Anyway, by now let's consider that these are our intentions. The second example is for a Spotify we have we are now we moved for to be part of the spot one of the Spotify, a product teams, a who and we are responsible for moving a free customers customer who get the service for free to pay the customer. So we think that we could create like a fandom community for a big band, some big singers. And we could provide provide some special features to pay the users. So that will generate customers that are or that currently are not paying will start getting a in the paid subscriptions to become part of their communities. And that will generate 1 million new paid customers from those fandoms every month. So these are two examples on how we could think not just in the features, but also on their outcomes and their business results. So now, Ricardo, yeah, okay. Sorry, I forgot to ask you. When do you want questions? Because there is already one question in the chat. So would you like to answer now, will you? Do you want to wait to the end? Yes. Is now it will be a great time for questions. I'm going to chat. So I might read it out loud to those who who might might be you know, driving, for example, and listening to it. Or hopefully you're not driving and listening, hopefully. But you know, if you cannot, if you cannot just read let me read it. This flies. Okay, this law seems to indicate the development process should flow from output to outcome to results isn't a tradition, the opposite. The business is communicating it goals, there are opportunities, outcomes, that will drive that behaviour, and outputs solutions, ideas that are validated against changing those outcomes. That's okay, it's like the Dr. There. We put this order. But this does not mean that the this should be completed from left to right. But it's like we are going to validate our assumptions and try to mitigate our risks from left to right. But it's okay. Yeah, it's if we do for example, opportunity maps. Opportunity solution trees, or opportunity maps, usually we have chaos and the chaos are a hopefully, if you've well done they they are metrics of outcome. So, if they are even better they are predictive metrics of outcome anyway. If we follow if we have chaos, probably we will not have this. We will not complete this this chart from left to right, probably we will have the the right and then the very left and then we complete the middle for example, but there is no other day. Thanks, Matthew. Tell me Matthew, Did Did I answer your your your question or your comment? Yes. Okay. Okay. Any any other questions for now? Before the practice? Not for now. Okay, let's go to practice then. Anyway, I will be paying attention to the chat while you practice. So here we have the Pro This part I here I include it to two products or two products that you know and then there there are several slides, we can add more a for for the practice what we will do is to start adding new slides, I don't know how many people so, I have 19 I have 19 breakout sessions the top at the moment and then for everybody if you get into the breakout session and for example people who are with you in the breakout sessions are inactive please leave the breakout session and I will put you to another active room okay. Okay, here, I will share with you this this link for you to practice what we would like to the work to do is to to go into your your slide depending on the on the team you are in and fill the poultice either with the three or four of you in each group will agree on what are the product a you would like to talk about, it could be that someone in the in the group in the breakout room would like to talk and to add a feature idea of his or her own product or you would like to to have some product this three or four of you know, like Facebook, Twitter or or Google or some or Google Drive for example. And then complete this, this both tests and do just two or even three product features with their blood outcomes and their business results. So, what I will do is to share the the link in chat slides for practice I put practice with s it could be with us or with C depending on the type of English we speak and then there you have the link, please tell me if you were able to accept the link and what I will do is to to put the the example in the very last slide. So, your team will will be the number of a slide. Okay. And the again, the idea is to have you will have seven minutes to agree with your teammates on one product feature of a known product and think on the outcome you would like to get from that feature and then the business impact or the business results you expect and probably Yeah, I think on future future features right. Let's see the work done here we have a feature this is a feature a scheduled travel with reward points for using your credit card or more often and the promises are they expected what is expected big a potential growth in the use of credit cards and gain market share right. So here I will go to other examples. So ways change to existing feature mobile notifications to different areas of Apple CarPlay This is okay okay, notifications and directions all directions or notifications all together and we hope customers will get happier. Okay what talks and again, the search function user will be able to view to view their desired search query earlier less frustrated use Yeah, we could measure their frustration. So here we have other examples for Netflix able to comment to insert comments or movies. You started watching more content, increase engagement, nice simplifying Netflix user interface, less buttons, opening your business and meant for non techie I'm thick is excellent. So here we have more examples. Here, remote control from my phone I'll just spend some time searching with this remote production and reduce the shipping costs, reduce the carbon footprint. college application essays being written with chat ship at 99. application essays, integrity, higher rates of situations higher projected production rates. Good. Excellent. So here we have another one. Okay, it's work in progress. So anyone that would like to learn to tell us about the learnings or the discussions? If this is something you already do all the time with your business people and with your teams, something or not tell me would like to, to comment on the activity? Yeah, I think usually the discussion doesn't get further than the product outcome. Quite often, it's the the business outcome is a bit neglected, sort of assumed. It was the better. So yeah, are you you make a big jump, there is what I've seen a lot, too. It's like, oh, well, clearly, once we do this, then it will make us a tonne of money. And it's like, wait, but but how? And so this really kind of pulls out your thinking a little more on that. Now, we will question if these assumptions are right, but this will be in this in the next iteration. So Laura will Lau you will have a question or you would like to make a comment. Go ahead. I get read a question. Uh, yeah, it was a comment was like, I noticed that we were quick to find the business results rather than the outcome. I was like, okay, yeah, let's focus back on what would mean for the user? So yeah, I noticed that one. We have here in chat, Kristen, a, I'm thinking could you use this format mindset shift to convince your organisation of the value of product discovery and or to start incorporating those practices into your teams? Okay, yeah, it's like, you know, one way to introduce product discovery is like, going into retrospective, trying to measure if what was done, for example, during last year, or during last two years by the buyer product team, delivered the expected impact in the business and the expected outcomes in and changes in behaviour of customers and users. That is a good a good, a good way to introduce product discovery is like going going back and trying to measure if the yard if they if the investment on developing new features. is paid or not. I have a question. Yeah. Hey, hi, Calvin. How are you? Amara, how you doing? Thanks for the class I. So the one of my question is basically like, so like, how flexible can this be? Should we should we basically kind of do user research before having this? You know, before kind of having this go have the features we need to implement? Or do we just have that assumptions, and then validate those assumptions using user research? We need to do the research and the discovery before even writing the first half line of code, because so we are going to go in that direction. So I'll share screen again. Alright, thanks. And perhaps this will answer your question. It's like the next step. You have this slide. But the next slide is to put a big, big question marks. And a, actually a, we, we start asking, Hey, if we have this feature, this new, great idea of feature in production When used by customers, our end users, they will produce will they produce the product outcome we expect that means are we going to solve a relevant problem is our proposed solution variable is going to be attractive. And then the here if considering that, after we validate that this will happen, we will see how then we again will ask ourselves that product our outcome will generate the business result we expect. So, here, so, when we say Don't make assumptions, what we are thinking is on trying to validate, if the feature will produce this outcome that means, jumping from this from here from to here and then if the product outcome will produce the expected business results, so, here we get a toolbox This is there a I will I will I will post the slides in the meetup, but anyway is you could take also printscreen of these but not necessarily because because we will mostly focus on this part they they they they evaluate the ideation and validation, but you have many discovery tools that you could use to think on new ideas and validate and experiment to validate the hypothesis you have in order to answer these questions in the orange squares before adding the features into your backlog. So, here mostly of all this discovery toolbox, I would like to put like emphasis in this doing user and customer interviews is like this is qualitative exploratory interviews to to think to start learning what we need to know about a basic or key or relevant problems our users are trying to solve. So, and then, we have some ways to validate if what we are trying to do it will it will change the behaviour of users and I will produce the business impact we expect for example, to see if there will be a change the expected change in the behaviours of users and customers, we could have the concierge and we serve of us it tools is like a basically not implementing anything and doing by hand is like for example, if we if we have a thinking on adding a new chatbot a chatbot to the to our hours product a rather than developing the Chatbot we could have a few people following scripts and do a product and I have a like people interacting with the chat box on the Indian side you have real people this will be Wizard of Oz the difference between Wizard of Oz and concierge is that in concierge, the user knows that there are real people doing the work and not an algorithm or on the other hand is someone that who is answering the chatbox or responding to an email and or doing something in in in Backstage in with with result of us people do not do not know that there is no software and there are real people doing the work. So fake doors is like when you get in some websites and they you click and you express interest on something and then he says oh you will have that feature soon is a factor and the smoke test also similar empathy maps personas to customer journeys, how we might, how might we is another another tool that we have prototypes, physical prototypes, digital prototypes, landing pages and AV testing, usability tests, business viability test, for example, what I will do now is to take some some of your examples here, some random Wi Fi one for example, in the I will go to the number four slide and they say and we say here, the remote control TV from my phone Long will produce a change and in the behaviour of customers that a they will not need to buy more batteries, and not spending time searching for a remote, but we don't know really if if they will use it. So, at some point, we will have to do some some prototype to see if their usability of the phone will be good enough compared with the usability of the remote control. And then we in order to see if that changing behaviour that is like users will be using a mobile phone if if really we will reduce the remote production 50% in some way we will need to to validate that at least 50% of our of the people who buy our TVs in this case will be using their phone. So then we will need to do calculations to see if they the shipping will the shipping costs will be reduced and we will need to do see how much are we going to reduce our carbon footprint. So, it's like do some use some of the tools here a prototype for example, and here are some some calculations before spending time on developing remote control for the phones. So, in the next iteration, the idea is to what I will do is to add this toolbox in the in in the last slide and then and then you in the same groups try to see a what what are some of these tools, especially the ones in this square in evaluation and validation and the the ones here in prototyping. And in testing, what are the ones you will use to ensure that the feature you will use you wrote in the slides will go is going to give you the expected outcomes and the expected business value. Okay. That will be the next a the next activity, you will have seven minutes for the conversation. Quick question. So are you going to send these slides over as part of the for us to review offline? Yes, sure. I will post them in the in the meetup. So we are we are all that we are losing people because of the because of the time what we could do perhaps is to to follow up on the in the chat on the in the meetup, this practice. And yeah, continue talking through meetup you could you can also contact me through LinkedIn, or I will give you my my email here is in the chat. So we could continue the talk, but especially let's try to share our comments and our questions. In in, in the in the meetup chat for in the alley, Ireland. Meetup. Okay. So and I will take like last five minutes for questions or comments on what we practised and what we revealed during this call. Or any experience you have, doing prototypes, validating assumptions, doing product discovery, strangely enough, not a lot of experience. The problem is, I think a lot of product discovery proceeds from assumptions of what the outcome was going to be and whether or not as good or bad foundation I think is the Cinderella end of the process. So I think for me anyway, that's the takeaway. That's something that we should focus a lot more on. It's like otherwise if we do not believe validate, we are always against the clock against the time We have big backlogs, plenty of hundreds of items, items that have features that we will never do. And at the same time we we might be implementing things that are not useful or are not attractive, or will not deliver the value we expect further business. Yeah, I guess you see that the biggest risk probably is that there is, you know, it's very expensive to deliver anything, so that we are spending money delivering something which is not needed. And I've been in that position that, you know, we spent half a year building something, and then very few customers were using it. So that's, I think that's the main benefit of validating it when you know, early. Yeah, I got told by a product owner up on stage that their goal was to get 25% of the features in the product being used by most customers. And currently there were at 18%. So that's an awful lot of effort that goes wasted. But if it's on target says like this. I was also in, in a company where we thought that we were always one feature away from success. Like, there is always something else we need to do in order to do the product market fit. But that never happened. I'm very curious of the time and I want to be respectful of everybody's time. So thanks very much for participating. Thank you very much. Oh, okay. Thank you. Bye. Thank you, you