AA252 - The IKEA Effect: Why You Love Your Bad Product Ideas

Stop building products nobody wants by discovering the psychological bias killing your strategy!

Today, Product Manager Brian Orlando and Enterprise Business Agility Coach Om Patel are taking on the "IKEA Effect."

The IKEA effect is the cognitive bias that causes product managers and leaders to overvalue the things they build - simply because they built them! That's right, today we're tackling a bias that's led to so much wasted budget, we're going to end up needing Congressional oversight... not to mentioned the ignored research and "survivorship bias" of trying to be the next Steve Jobs.

Listen or watch as we discuss and review:
- The Scientific research behind the IKEA Effect (people value items 63% higher because they've built those items)
- Why teams ignore expert research and undervalue insights THEY DID NOT SUFFER to obtain
- How traditional review committees designed to kill bad ideas often stifle innovation (and what to do better)
- Balancing the need for intrinsic motivation (Self-Determination Theory) with the necessity of governance
- The "Kill or Nurture" Framework: A new 2x2 decision matrix to evaluate projects based on evidence vs. passion

We also share personal war stories on the product-related IKEA effect, bemoan the struggles of gaining funding for evidence-based ideas, and maybe even distinguish between a wobbly chair and a throne. Tune in if you're interested in ways to stop falling in love with your own bad ideas!

#ProductManagement #Agile #ProductStrategy

Norton, Mochon, & Ariely (2012) IKEA Effect Study, The IKEA Effect (Harvard Business Review), Self-Determination Theory (Ryan & Deci), Gartner Research, Deming's theories on intrinsic motivation

LINKS
YouTube: https://www.youtube.com/@arguingagile
Spotify: https://open.spotify.com/show/362QvYORmtZRKAeTAE57v3
Apple: https://podcasts.apple.com/us/podcast/agile-podcast/id1568557596

INTRO MUSIC
Toronto Is My Beat
By Whitewolf (Source: https://ccmixter.org/files/whitewolf225/60181)
CC BY 4.0 DEED (https://creativecommons.org/licenses/by/4.0/deed.en)

Transcript

0:00

know what really grinds my gears sorry, people and VPs burn through 6 to 12 months worth of time and budget millions of dollars of budget, right on products that every single customer interview, if they had done any, would have said no to, and it was clear as day to anyone on the product team, and when you ask them, you know, hey, why did you keep going um they're probably gonna give some, some advice, some, some rote advice rote, rote, rote advice, uh, well, you gotta have intuition, oh you gotta have intuition, but under hood, like you, if you clean off the floor a little bit, so your shoes aren't sticking to it a little bit you'll find, it's all the Ikea effect describing the survivors who failed but you're right, what about the ones who ignored the murder wards and Steve Jobs didn't look at focus groups or listen to them Slack was but just a failed game sometimes the wobbly chair becomes the throne well, we're gonna talk about wobbly chairs, but but you know what we're not gonna do? we're not gonna be playing the game of thrones, that's what we're not gonna do because ignoring survivorship bias is like a you're saying to me right everyone's banking I've watched product on being the next steve jobs, and that is not a strategy that is everyone looking for the winning lottery ticket something the in this void we need strategy can't be kill everything that doesn't have a spreadsheet right that's not product management that's accounting at best it's private equity it's private equity yeah, yeah, yeah but arguing agile if this is your first time welcome i'm your host product manager brian orlando this is my co -host enterprise business agility consultant mr om patel and the luminary of low hanging fruit, oh that's one of the better titles I've had, are you feeling quite fruity today, I am very, very fruity, oh yes, welcome back to indeed well, I guess the audience will decide podcast, I know it looks like we have a lot of subscribers, we're a tiny, tiny that's because we've been grinding, we've been running up that hill for a long time, so we're trying to get to a thousand, that's our next goal uh, every like uh, I think, and also known as uh, sends a puppy to heaven, I think that's, I think those are the rules actually, yeah, yeah, yeah, please do like and subscribe, uh, and if you have multiple emails, do that with each of them I didn't say that alright, tackling something that's killed more products, more careers more company cultures uh, than bad market timing or market downturns or whatever ever have, and it's called the Ikea effect um, and that is the scientifically proven, we have some papers we're going to talk about it, bias where you overvalue things just because you built them whatever so today, uh, we're it is, like a wobbly bookshelf a wobbly product strategy the aforementioned wobbly chair I got aforementioned in there that's a good term, I like that term you should get some points for that, I'm just saying although, sadly, the points don't matter so, this podcast the promise here is, you'll learn uh, how to spot when you or your team are in love with a bad idea right, the, the Ikea effect in yourself and your team, you'll, uh, be able to spot, when to so by the end of protect passion projects and when to kill them, hopefully and, uh, how to build review processes that don't crush innovation, or let your company ship garbage I mean, business of shipping garbage it's a lucrative unless you're in the business, I hear um, so, so this isn't just theory folks, right, this is the stuff that determines whether your next big bet makes you a hero or a zero let's there was a study that proved that people will pay 63 % for something that they built themselves even if it's objectively worse than the what if I told you professional version of the same thing wow, 63 % more and yet it's still broken? yeah, yes ok, I'm in, count me in 100 % more broken than the real thing but you paid 63 % only right, so you're still ahead, sort of statistics statistics, Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Mohan Moh demonstrated, uh, Mohan Mohan Mohan that participants valued self-assembled Ikea furniture 63 % higher than the identical pre-assembled versions and that's where this 63 % number comes from, we didn't just make it, I mean like I love making up statistical numbers throwing them on the internet, executives can't tell especially, sorry nope, nobody can tell, right I love doing that, but in this case it's a real thing, you can go read the study this is it is what yeah, I mean, look, it is, I kind of agree with the premise though, that you value more things that you build yourself I count myself in that group, because when I'm building stuff, I want it to be sturdy enough for, not just the fit for purpose, but beyond like, what if, and I'll overbuild it, probably but even if I didn't overbuild it because I built it I would think, I tightened everything up so much that it can't possibly fail, it can't possibly wobble, right? so we'll talk more about that I love the statistics though 100 % of the ships that don't fly never crash listen boom, there you have it 60 % of the time, it works every time yeah, but you know, it's a category that we're talking about so you know, there's gonna be a steel man absolutely there's two points I have here for you in the steel man number one hey, ownership creates accountability and follow through right, which is funny the way, you can tell that I wrote this slide, because I wrote it I wrote it with ownership creates accountability not, not like, accountability is demanded or given or forced on someone, right that's, that's the way that, somebody that thinks like that is like, oh, accountability is a four letter word for them, you know, no if somebody's not accountable, then nobody's accountable and then, and then the other one is detached analysis produces mediocre committee driven results basically, or I have it here as product, but if you design the product by committee, you're gonna get mediocre results, that, that's what the steel man here is gonna say, both, both, both ways yeah, I mean you know, there, those are very real steel man points I've heard people use them quite often, right off the bat, that I want to kick the first thing, off the, this section of the podcast, is pointing out that whole ownership creates accountability thing, well that's that's just your opinion, man you haven't given me any evidence at that point of saying, oh, this person owns, owns it it's their baby, it's, a red flag than uh, communist China over here that's even more of me, well now someone's gonna own it, and I'm gonna enforce that they because you've told own it and check in on them, so now the pressure's on for them to make it seem like it's good at all costs, at all costs, that's right but, but they don't have the same level of attachment to it that you do you know, you expect others to kind of, to value your own creation equally, if not more and often, they won't and, and you know, you know what's funny is, I've that hand off big ideas they're the worst at this they're the most at risk for this kind of bias that we're talking about is oh, Brian, I told you the product leaders that most customers want to see the button in red I don't know why you haven't done you know, red all throughout, though I don't know why you hadn't made the whole app in red, or or, oh, I have a great story about, sorry, I didn't know you gotta put that in the right spot I didn't know it was story time yet, but like, I had a great I had a great moment where, I was a product manager and the director of product had a great relationship at the time with the director of product, and I remember, um bunch of VPs and stuff that had real strong, they would always give me quote, product advice we'll say, we'll say advice we'll say advice, the issue with advice in me is like, you know, I do the podcast on this stuff, so I'm constantly talking about evidence, and I can, you know I can do the book and I can do the, you know, hot seat entrepreneur, you know skin of my teeth, kind of you know what I mean, up at night I can do that, I can do both I can go to two full extremes but I like to, you know a balanced approach is really better than both of those, and it doesn't burn everyone out so, I, I specifically remember I had the, like the VP of sales telling me everybody wants dark mode you should make your app dark mode, and then I had the I had the VP of some business, I don't remember exactly what the VP was, but basically the VP of sales versus like the VP of operations or something like that at this workplace, I I don't exactly remember what their roles were and the VP of operations was like the app needs to look professional in front of, business users and dark mode looks like a like a kid's college project a Silicon Valley MVP or something like that, make it a standard business application white screen, black font that kind of thing, and of course, they never talk to each other of course they don't, they only argued with me about what to make it and i built from scratch by the way zero to one and i remember this is an app that uh somehow they both went to a demo with the director of product who i worked for that i was not invited to where they were looking at my product again, communist parade here, that's red flag number two by the way, anyway, that's ok, that's ok, cause I trusted the director of product, I've already stretched ego enough in this story, but the point was one of the first thing, after they did the whole song and dance demo, right, for this like very important customer or whatever the first question they asked you can make this dark mode? cause which, which, by the is there a way that way when I first developed the app we developed it, me and the development team developed it dark mode only, because myself and development team we know that most technical users that are like solo operators or whatever, they prefer dark mode, that's what the evidence that we researched with our UX researcher found, so the business or what, business function, whatever, and the, the VP of changed to light mode over the sales wanted it to be guy because siloed operation at that company, so we changed it based on sales was like a this VP's opinion because sometimes in development you just want to, you don't want to get in fights, you know, yo, whatever it's, we'll go over, we'll change all the CSS from like, boomerang right, so the minute you get a customer in place to but sometimes it's a look at this stuff not just internal VPs and whatnot uh, yeah, they're gonna ask questions like, what about accessibility and this and that it was so funny, it was so funny, it was, our research is the reason we created the app in dark mode to start, and failure on my part, more than it is anything else in the story, is that I this is really a just like said look, ok, whatever, we don't really have any customers anyway, so we'll change it to dark mode, or we'll change it to light mode from dark mode and just, like, get out of my office, like whatever, it's not, I don't want to fight about it we don't have any users, we have no preference we just have our research that tells us most people want this and then the very first customer they got done selling it to said, hey, we love it but, we want Dark Mode we want Dark Mode it was, it was hilarious it was hilarious hilarious and also, I think I cried I didn't really cry, but, you know I metaphorically cried, on just a fun little diatribe, I mean they are fun diatribes when I do them, but I do have a takeaway, my takeaway here is say, start every major decision review with the Ikea check now, my, the story I just told, like wouldn't really fall into this bucket, because I didn't really think it was a major decision you know, and in retrospect, like it did take, line as major? what do you draw the so my story was not right, I mean this was a one line change, yeah, well theoretically it was one line change one times five, maybe I would have just hooked it to system settings myself, but whatever yeah, should have been a you're right it one line change, but that's a story for a different day, it's like the end of Conan the Barbarian that's a story for another time that's a, I start every major decision review with the Ikea check ooh, what is the Ikea check, Brian, well that's ok, I'm gonna tell you right now, so I've got a couple things to ask, I have a few questions to ask, is the Ikea check, which is, number one how much of my conviction comes from the work I put in, versus the evidence I found conviction, of course, meaning you're sure that this is what everybody wants it's what plants crave number two would I fund this if I had, and then, question funding authority right, or funding responsibility it wasn't my idea, right, yeah someone blocked me cold, would I fund this and then number three is, you know, number three is probably the strongest one on for this time is this list uh, write down your confidence level before you present it, and then check it again after you get feedback your confidence level, and that, that should tell you, I mean, if your confidence is like oh, I'm at 10, everybody wants this everyone's gonna be crawling over each other, whatever and then you check your feedback with either somebody else, or against the market or with evidence or whatever, and maybe you get knocked down a peg or two you know, and the example I gave probably isn't the best example because in that example I went the opposite direction, I was like I don't really think anyone really cares too much about light mode or dark mode I think people probably prefer it, but like if it's gonna be a big thing with the executives I'm gonna let this one go that's called capitulation in retrospect, I should have not let that one go like, actually, our research says you are wrong, Mr. and Mrs. Executive um, what research do you have uh, and then, oh, look there's none yeah yeah, and you know, I would have taken the 12 line change or whatever it was and said, if we go with your decision here's the cost of that decision because the evidence points otherwise so we're going to incur this cost right, are you ok with that? oh, and we did, and they weren't that's how that story ends that's the nice thing about being an executive is it, um I guess, I don't know alright, you stayed up till 2am to create, isn't worth more because you're more tired it's worth more to you because you're more tired think about that you you just sunk every consultant's battleship right there I did, consultant's MO absolutely um, here um, it's, it's about avoiding bad ideas in the first place because you you gotta take away so that deck that were involved like the Pareto Principle if we're gonna do on this one, the 80-20 and you take nothing else in this podcast, take this one it's because you are involved just be self -aware enough to know, oh, maybe I have some bias, maybe I need to check this with somebody else, so what do you think about this one, what do you think about overvaluing your own work or being self-aware drop a note in the comments uh, I guarantee you're not gonna be alone on this one yeah, would love to hear from you ok, exists but here's we know that bias where it gets really dangerous, um we're products to beliefs when it spreads from wow, yeah, now, we're gonna talk about a, Ikea effect right, an aspect of the the epistemic Ikea effect I'm glad you can say that word, because I can't, I can't say why, why does your team ignore expert research at that's what we're gonna talk about next in this segment, expert research, is done by experts they're, some other team they don't know anything, right? yeah, they probably did great work, but uh, it really doesn't impact me, because, you know, I have conviction in what I believe, and that's all there is that matters well, gonna be the hardest category for me, this is probably because I've seen this a bunch of times, I probably have fallen victim to this one, this bias myself, which is, uh another team did excellent research work, but you would never know, because you don't take it seriously, because you, you get used to your team doing all the experiments on their own you do your own scrappy little startup inside of a larger group or whatever you know, and, and why you know, because you discover ideas, the, the, the ideas that you discover for yourself they feel more true than the ideas that get handed to you, regardless of like how meticulous the research and the and the, and the experiments and everything that went into it are yes, I agree, yes, and so this is the yes and um, yes and and uh, they will disregard those ideas all the more if they didn't pay anything for them, because they're free so it's like, well, I'm getting these ideas, these sources of opinions from, you know, sources that don't cost me anything, therefore they can't be worth anything, and that is very dangerous because sometimes, quite often maybe you can uh, you can throw out the baby with the bathwater there so there's academic research here that talks about the um, Ikea effect uh, with regard to beliefs and analyses showing people overvalue self -constructed arguments and undervalue free expert knowledge regardless of actual quality it's what I'm showing on the screen right now you can research this I, I, I'm this will be another podcast where I try to put the the research links into the show notes, because i've got a ton of research links for this one um and this one as well but and I'll try to link this one is interesting because I, I once built a product off of uh, analysis conducted by an entire, I had a whole team in-house so it was like services and products together and the services was the analysis side of the house and what they would write and the reports that they would write would go into a, basically a database and get crawled by a ton of other people in AI and a bunch of other stuff like that and the whole quote system was fueled by what these people were feeding into the engine right, but, but, because because people would, like, get, like, a constant stream of updates in their email or whatever else, platform email, whatever wherever they would get it, it seems free to them and, of product even look weirder I don't know a better word for it more challenging is uh, the person that purchases the product is not the what makes this type user of the product so to the user, it's extremely valuable of things that came to my brain right there when we were talking about this this was a, this was a tough one and this what makes this even is a tough one I will tell you, it's a tough one and, and, and at that company with those teams we never really settled on any one thing but I had a few ideas that never really got tested against the market I had a few ideas about like maybe we can build research to a point and like, we'll do the 80 % and let the customer do the other 20 % or maybe we'll feed the customer with some things and let them take it over the finish line somehow but we basically stopped working on that product just like another two, three months dedicated effort of the whole team but sorry, I had a lot that team was so fractured because it's in the world of like geopolitical analysis is so much going on at any one time that you're kind of, if you have a small team you're thrashing at any things context matters like generic research, generic like free things you got off the internet or whatever like, that's not specific uh, there's two to our specific situation, that's not that's not, that doesn't hone in on our specific situation and, and also learning by doing well that, that creates a deeper understanding than reading or giving you know being handed stuff, well, by some of the team, like research, like another team did user research, they handed it over to you you're supposed to read it and say oh x percent of users do whatever or whatever and you're but do they yeah, and also, yeah, is that the market we want to serve, or do we just go after the y percent, right yeah, absolutely agree, both of these are solid steel man arguments context is king, in this case if you're one of those people that are subscribing to these third party, um research um, sources, if you will like you know, there's so many out there so i don't want to name names but let's say you look at something like the gartner research and say they evaluated 500 companies in the space yeah well you're in that space but only just like you you're you're broadly in that space but specifically you have a nuance that you're serving right and so maybe their research conclusions don't really apply to you i think it's an it's just an input for me that's a great example look at it as an input and go yeah this is great if we were that but we're not right so what do we do with this and that ranges from well we just throw it away or we take it and we just hone it a little more for our own market, right? uh, I went through this whole thing, um, during my past in the, in the publishing sphere, same thing, back in the day, when the company I worked for invented smart terminals, when I say that, I mean, they were really putting, you know, code into dumb terminals, so the, the the compute, the processing wasn't happening on some mini computer that's centralized, it was happening localized to every machine, well of course performance is no longer an issue now, right, and they were ahead of the curve but distributed computing as it became known wasn't a thing at the time and nobody wanted to hear about it, it was all centralized, we want mainframe you know, mainframes, um, tethered with all these dumb machines. So at that point the company had to make a decision. What do we do? Do we go with what the research says the customer wants or, or broadly speaking, or do we just venture out on our own? Thankfully they decided to do the latter. so the two points, learning by doing creates deeper understanding deeper than it would if you got a report or whatever, I don't know there's a whole, like, well well, it's not invented here, like, even part of your example is not invented here, kind of that's what everyone else is doing, but we're special like, I'd be very careful about that you know, I've seen we're special burn a lot of companies no doubt, absolutely no doubt you're not really that special, if you look at the macro trend in your industry and you take the blinders off a little bit and realize that your industry is a little bit bigger than you might have thought, right especially if you're playing games with a total addressable market, these kind of terms like playing games um, does matter, um but also, you're, you're scrappy experiments with your one product team yes, strong, yes, context it's important that you touch things and find them out for yourself like, you, like, as small children, you don't learn until you, like touch things and you get burned or whatever you're not to touch them again, yeah, like that's how you learn, I understand that's where the deep learning comes from, but I would hope, like I would hope that the wisdom of being an adult is you don't need to, to trip over every you know, route in the yard to learn how to step properly I would hope that you can take advantage of other teams' learnings and learn yourself without actually making the mistakes that, that's, that's what this, that's what the first point oh, well, context matters, you know like, your generic research just misses my specific situation and, but, so basically what you're saying is well, even the company I was talking about before was we had a service side and we had a product side, right, and you kind of had to blend the two to make it totally work, and you say no other company is doing services and products together we're so unique, but anybody who's been in tech, any, like a year, yeah, knows that that's complete nonsense, it is, it is we used to call them solutions, right, solution equals service plus product, yes, every tech company does that, undervalue UX research, uh data analysis research right, compiled by analysts on specific topics or trends, and they undervalue those, longitudinal reports because again, mostly they're done like, internally or teams systematically by a consultant or whatever and you don't pay for it and you get it for free and you undervalue it but there's real wisdom and insights in there, it's just, you didn't you, there's so much bias you're bringing to it when you're reading it and you're getting it for free, and there's and this weird thing about the human mind, about this whole section of the podcast we're talking about now, is kicking in stopping you from really like, realizing that, oh, maybe I should change something, or maybe I should do something and yourself out of the evidence I'm seeing and being presented with, even if I step back and ask didn't do it myself, what existing research are we ignoring? and what could we do about that yeah, I think in totality you can't just dismiss, you know, that longitudinal research, people do it all the time though, they do absolutely they do, and you know because they, it's two reasons probably right, because it's free, and they didn't do it themselves, yeah, somebody else did it so what do they know about our industry was special, that syndrome, that's right I think there's a middle ground somewhere look at it and say, how much of that applies to us yeah and, and just run with that, because not everything will apply to you, right, broadly speaking, you're not gonna be in the middle of the bell curve all of the time so, I think what we're saying is, uh you didn't reject the research, because the research because it was wrong you rejected it because you didn't suffer to find it which is funny, because you have another team member or a whole team, or a series of teams or you spent a bunch of money, so that you didn't have to suffer, to gain that wisdom right but here you are rejecting it because you didn't suffer i think i'd extend that a little and say maybe it's not you didn't reject even your anything it's not your team or maybe it's one of your competitors that suffer there's learning in there right and that's completely free ignore it at your peril in the case of the gartner uh polls and yeah you were talking about before and and like there's a much more other than gartner but like sure everybody's buying those studies right, they did it once, your competitors are getting them, why are you not learning from them, like you know, exactly yeah, we need to move on, um, need to have a there's a lot here, takeaway in this section, because we've been ranting a lot, and usually the first category in the podcast is the longest one, I think this is gonna end up being the longest one, I agree, uh, because we both had stories in this section but so the, the, there's a takeaway here, I'm gonna highlight it quickly it says, add a prior art section to every product brief. Well, prior art section, Brian, what do you mean? I'm glad you came to the podcast because I got some, I got some things here for you. So, before proposing new research, you list out what internal teams have already learned because your internal teams, they might have access to these studies, these papers, um, if you're in AI, uh, and like if you, not just like using, if you're actually like researching models and training models yeah, you can, you can download all these like all the papers and everything, they're all freely available you can go get them all, number one and number two, uh, you can assign a steward of learning so this could be a UX person, UX researcher I mean, get a UI UX designer could do this work too, like the people person, people, people, people they're people, people, they're also like deep research folks, like the UX people are like very underutilized in the field, especially this this should be a, we should get a UX person on the podcast, just to talk about like, what is happening what is happening in the UX side of the universe right now in tech, that all these people are so undervalued, because if you think about a product manager and what, what, what the modern podcasts and people and the linked influencers out there are like asking from product people, oh, you should be your own development team and be your own you know, AWS like DevOps expert and everything and you, and you should do your own stunts and do the, and do the product stuff and do the business strategy stuff, right ridiculous, right, um UX people were already doing this stuff before the whole market decided like, well, we don't need UX people you know, there's no AI inside of UX those are different letters completely sorry, that was the amount of thought that went into this, but you add them to every review meeting steward of learning, in the way that you boy, it's starting to sound like a scrum master isn't it, right, like somebody that goes to most of your review meetings and kind of connects the dots yeah pb hermit style together to figure out, oh, well this team's doing that, and this other team's doing that, and they're not, they're not there to like supervise, or tell people what to do, they're just there to help teams say, communicate with each other right, yeah, and in organizations that have a matrix structure this can easily be supplanted into your teams, right, it's not a full time thing for a team to own one of these resources, if you will, or people for any length of time yeah but this second point here, you know, a steward of prior learning I think it's a great idea, and again that doesn't have to be a dedicated full -time role with that title, right you can rotate if you want, but these are people that are just, like you said are, they have their eyes peeled and ears open, and they're learning from what's going on in the past, right briar friars, whatever this is also a good candidate to throw over to an agent because, again, an agent that's documenting things in, a shared learning area something, like an L if you have nothing,&D type of whatever, it could be anything it could be a file, it could be a file on a server yeah, um, just to, like write down what the learnings are and what you tried and what you're learning, uh, and then the final one I have here you ask, uh, what would have to be true for this existing research to change our plans which, that, that should be easy to answer if you're in product, right, because you you already have expectations you already have, KPIs whatever, you know, the goals that you're looking out for, um that should be easy to answer so those are some takeaways sorry, interesting category again, we could have stayed here way longer, so, what do you think uh, would you, would you have liked us to stay here longer, what do you think has your team ever ignored perfectly good research, or, I mean, maybe for a good reason, I don't know, sure let us know in the comments like and subscribe while you're there man, bias uh, the obvious solution is some kind of review panel that can murder these ideas and projects whenever they need to whenever they deem them to be poor investments let's this is an talk about murder boards, review committees designed to kill bad ideas, but here is where it gets complicated murder boards exist for one reason and we've talked about remember where in software you had to go in front of a review panel on a regular basis to do I am old enough to things or to, like, green light projects or whatever so, murder boards exist for one reason to kill your bad ideas before they kill the company but here's a question that boards are killing your good ideas too, so what if the cure is worse than the disease, right, well what a what if murder fantastic segment this one is um, alright, so first of all, we should probably define for the younger generation what murder boards actually are I probably like, uh, titled this as like a losing proposition of murder boards it's, it's, they're review boards so, my career and I'm sure you can talk to the early stages of this as well like, you, you, in software you used to have to go in front of a review panel they before you even started a software project you'd have to go in front of a review panel you'd have to say, oh, you know, this is my plan for how many people it's gonna take, and how much money it's gonna take, and how long it's gonna take and these are the things we're gonna accomplish, and here's the KPIs, and whatever made made up metrics that you dreamt, dreamt out of the air, and uh, you put them down on paper and people are like, oh, great job going straight to the top, that's right, I'm doing the transatlantic accent once again I hope it made it into the intro of this podcast in some way, shape, or form, but in case it didn't, I good job kid you're going straight to the top, the they originated in high stakes domains, like satellite operations and mission planning for like DOD and NASA and things like that a big, multi-million dollar type of deals, where we're gonna waste a lot of money and what are we gonna get out of it, so you had to aggressively surface the flaws and that's where these what I call murder boards just project review boards, basically that's where they came from yeah, and that was, I mean, you mentioned some of the high stakes environments where these were used, but they were also used in software, because software used to be fairly high stakes back in the day not anymore, but it used to be, right so they were actually prevalent back then as well, and it was at the outset as well as throughout the SDLC sure, alright, it's fairly ironclad uh, listen, uh, you need a rigorous review because that prevents expensive failures downstream especially when you're talking about multi about next generation fighter jet multi -billion the steel man here dollar failures and then, weak ideas they deserve to die early yes, this section is actually very good I can get behind these as well because the steelman here is, listen, rigorous reviews, they present expensive downstream failures and also weak ideas that like, they deserve to die, like, forget Dave Chappelle for a second because we may or may not have to pause the podcast to watch a funny clip is like, you want to kill them early before they waste a bunch of money um, that's a good thing and yeah, like, to do I yeah, the weak idea mean, look, it's hard to argue against these um, you know, if you're in that situation see if you can compartmentalize your weak ideas from some of the ones that you want to explore further versus the ones that you're really hedging a good bet on, right the weak ideas, the sooner they die the more funds are available to go check out some of the other ones, because funding at the end of the day is a finite pool sure, if you have this notion of weak ideas can just disappear, but also not because uh, they're external, they, they could be your ideas, right, so you've got to have that detachment from the ideas it could be your baby well, also, I mean, you know, the other thing, the unspoken thing in this category is, like we're all rowing in the same boat hmm if we're all on one team, one business team, there's no way I as a single product manager can be exposed to every idea in the business so, if we have a review board like this, like we want that review board who, who are exposed to much broader swaths of ideas, right many more ideas than I can be the steelman for exposed to we want them to be identifying and then redirecting resources, quote resources right, that's saying chairs and staplers, that's what I'm saying money is what I'm saying, right to the best ideas, and I might not see that, I might, I might have blinders on so, in this one, what's our stance on this what do we believe one, I mean, the traditional quote, murder boards like what most people listening to this podcast will have been exposed to with regard to traditional quote murder boards is the people that sit on those boards that have been there in perpetuity forever that are just there for the prestige of ooh, I, I greenlit that idea and I invested in this super early and they're the hipsters of business investment is what I'm saying they're there to make a thousand bets so they can claim that one bet, they're so smart because they made that one bet a million years ago or whatever when they made like, 50 bets that year, and like 49 of them did not pan out, and that one did so they can claim they're so smart, these are the Marc Andreessen's of the world, that's what I'm saying right now that's exactly right, so the other 49 were the fault of other people, like product managers etc, yeah, they're playing games they're playing prestige games and they're playing games basically, um, if the review board is run with some like basic product management practices that you're not looking for yes or no's here you're looking for the next milestone you're looking for the next, you know, funding round or whatever, you're looking for like micro -budgeting, you're looking to secure your micro-budgeting yeah, you're looking to see what needs testing, what warrants testing next, possibly right, these are the micro -budget expenses that you can endure without you know, breaking the bank uh, there'll be some that are just basically in that other echelon yeah, they're greenlit, we're good we're gonna pursue those absolutely but they're probably also further along in your product life cycle um, in your discovery cycle I think what you're pointing out is number three here the default of this type of board should be, what is the next experiment we need to see cleared for you to secure your next round of funding and it's not just like blankly no or, you know, something that there's no way that you could do or whatever yeah, but that decision yeah, you're right, this is the point I was making but that decision also comes with the guardrails of micro -budgeting you can go so much, right you go spend this learn what you can learn and then we'll come back and review if we need to, you um, expend more money on that if you're a finance bro listening to the podcast or like PE or whatever listening to the podcast like, we're giving you the playbook right now exactly like, how to use your money to get the best and not be controlling but how to get the best out of your products and how to, like, how to merge R &D with product uh, what we're talking this is it like, about right now yep, yep as opposed to playing product roulette yeah, I mean, or, or the product hunger games, I mean, the last one here is on this review board you want to rotate the membership on a regular basis to prevent any kind of like political slash ideological slash group think capture of the group, because regardless of who they are, where they come from, whatever um, they're stopping ideas through the controlling of funding and this could have a real this could have a real permanent impact on your business here yeah, this is a valid point too, I feel like most organizations that are not new, uh, tend to have the same static board so, that's actually a good point I this, section that we're talking about like, in, that we're the sad part about hypothesizing about is that, I think, at the very start of my career it used to be big like, project management heavy review panels you would have to go in front of and I think now, in the later part of my career it's morphed into like like one person that controls all the budget which I think is a real disservice to the concept that we're talking about it's become more like, shark tanky kind of like, there's one person we gotta convince of, I don't know if it's better now than it was before when everything was looked at as like resources and project timelines and all that kind of stuff I don't know if it's better that's, that's what I'm saying yeah, I agree with that, I think that one person also tends to be far removed from the product side of the house, they're more you know, on the finance side and so they don't really have a purview of what's really happening, like what are the latest things you've learned about and how they inform what you should be doing next all they're looking at is the cost side of the equation for the most part, I'm not saying they don't look at the other side but the other side is more of a, an extrapolation anyway but yeah, the cost side yeah so yeah, this is a valid point so membership on this review board it should be diverse is what we're saying and it should not be just finance people and it should also not be the same people every single time yeah takeaways ok, transform your murder board, I probably at this point, I've said murder board so many times it loses all meaning it should be, rather than a uh, review board rather than a project review board, which is the way I think about this kind of thing, we have some turn it into an experiment board, or funding board, the funding board would be a better way to word it, right um the board should be tied to a testable claim or a gap in every criticism from data, so if the board is critiquing you there should be uh, there should be guardrails on that critique, right, they shouldn't just like, open -ended, throwing you under the bus or whatever, not that the old project boards ever did that anyway not much, um not much, yeah, no, I agree so I think, part of the experimentation mindset here is also, what do we need to find out, and what can we spend at the most, on that right, right and that's where I think the collaboration between product, finance, anybody else who's interested in it, right maybe there's some I don't know, regulatory impact and so they, they can be on the board too but they're informed with the budget that's needed to find out what you don't know with a certain time frame in mind and then you reconvene yeah you know, this is interesting too because because your, your board themselves cost a certain amount of money yeah so, it's like the, the, function in some product management kind of like, budgetary control but, you, you know, you, you get some kind of like, shepherding along of your products or features or oversight or whatever, whatever, governance whatever you want to call it, right? yeah the controls here are, it's not personal it's about the work it's about the investment of money in the business, that's why you're tying criticism to testable scenarios it's not about the person oh, your idea is terrible, oh, he's got terrible ideas, yep, which is true most times, but I think it's, as long as it's evidence based as well, right so you're looking at the evidence that you can present and say, it's not my idea the evidence shows this and we need to find out xyz and here's what i propose we need to spend on it and then we can talk about that right but it isn't just it's great because it's my idea so i got the the default outcome here is uh you need to test next to get more funny it's not just no right that's number two here this is number three is uh you only kill when there's a clear disqualifier meaning you know there's a legal hurdle that you can't overcome, there's an ethical hurdle you can't overcome, or there's unit economics, in the case of in the case of the current world of AI, where yes, you could um, let every customer in the world um, go out and generate a million videos of infinite length with AI but, we're gonna pay a bajillion dollars um, the unit economics just don't work out think it does two things, right the first thing it does is it gives you a ceiling limit and that's very, very I important I mean, if you're even close to it you should really re -examine the model, right because, you know, there's a there's a delta there, and you don't want to be right up against that ceiling limit so that's one, and then the other thing is when you have multiple things that you're you're, you're testing out, right if each of those have ceiling limits combined what are you going to spend, what are you going to not do out of four things so that you can get somewhere with one or two things that you want to do, right oh man, unless it's strategic oh yeah strategic just means there's more money there that's all that really means, but in this category is down here at number four, uh, all reviews are time boxed they're all, that's a great the great equalizer equalizer, because like, the money to a certain point, and especially depending on what level of seniority in the organization cares about that feature, money might not be the hurdle but I guarantee you, the time box will be the hurdle yeah, absolutely, yeah the time box is just another way of spelling money though, at the end of the day yeah, yeah, right, yes, yeah category ended up being super hot fire like, I category up, ok, a murder board that only knows how to say no, like they're not protecting quality, they're protecting the status quo that's exactly right, yes so, we gotta move on to the next category so let me know what you think about this one our murder boards our sorry, I'm gonna try not to call them this throwaway murder boards anymore, ever, but we're done so I'm never gonna call them again anyway, so are they helping or hurting your organization share your war stories in the comments with us let us know, like, listen, to wrap this yell at me please, talk to me all about murder boards, tell me not to say murder boards that's what I'm asking you right now, in the comments yeah, boards aren't the answer, then what is the opposite approach what is an alternate approach what if we let people build their wobbly chairs so, self-determination theory research is about if murder clear autonomy drives motivation and, and, and you're talking to the that's right, what people at this point, I feel talking to the two people at this point, that's that we did the Deming podcast where, like, intrinsic motivation obviously is the way to motivate people like you, like the carrot and the stick like that, they don't work, like intrinsic motivation is the way to do it um it shouldn't be a controversial take but sometimes you have to let your team build the wobbly chair even if you know it's going to be wobbly when it's done you have to let them do it, the question is how do you know that they're how do you know when they're done and they've got a wobbly chair that's ready for market and they haven't burned down the house somehow they're building a chair and they're playing with fire I don't even understand how I would say, let them build it to the point where it looks like a chair, it wobbles like a chair, and let them sit on it, because you should definitely not expose that chair to the market, right, and I'm not even worried about the financial damage that's caused as a result of that, think about the reputation damage, you don't want to be the company that delivers wobbly chairs, so I mean, it's Haiku 3 was a thing at one point I'm just saying, it was terrible Haiku 3 was terrible, like I remember getting in a knock -down, drag-out fight with a data scientist, saying that, like you can't present anything from Haiku 3 to a customer directly, like a pastor on a website or whatever, you can't do it, because Haiku 3 is terrible it's bad it barely follows prompts, it's bad bad, bad, and she looked at me like I came to her house and I kicked her dog straight off the second story she looked at me like I remember that, professionally emotionally, philosophically I remember it some of your team members that say this chair is not wobbly, it shakes a bit, this is not a dead parrot oh, it is a dead, it is an ex -parrot this parrot is no more discussion but I'm, like, I'm gonna take you straight to the steel, man, listen, without guardrails, unlimited autonomy, like that review board we were talking about before, without guardrails, that creates chaos, it spends all of our money resources this is a great spent on pet projects they have opportunity costs, they have the, they have the cost of what else could we be doing, this is our own our own arguing point from another podcast in years past or whatever, brought back against us, and you know I hate arguing with myself because you might lose, that's right, I lose every time you don't know who wins, fucking i hate losing more than i love winning which means it cancels anyway so a review board that has no autonomy is really just a puppet more yeah yeah right at the end of the day to your point resources get spent on projects of course they do right and then what happens is your company is going to build stuff that potentially potentially isn't needed by the market because it's somebody's idea is why you're building this thing right if you're in a company like that and we're gonna do this for only one time this particular podcast keep that resume updated folks um, because it ain't going anywhere fast right means freedom within constraints that scoped autonomy that time, budget, blast radius, whatever like scoped autonomy uh, innovation sandboxes they, they let founders run wild with a capped downside so you, you basically take this budgetary amount you put it aside in the notes here i put 10 to 15 percent discretionary time right with a light review right that goes along with it and i put these three things in here because the alternate is crushing every passion project stopping your development team from doing all this like refactoring or technical debt review or whatever, that's a great way to run all these very passionate people, run them straight out of your organization, absolutely, or have against the market, if they don't work out they don't work out, throw it away cool, but if we're doing the 80 -20 thing then we're saying the 80 % solution is enough to me that means the 20 % can fly under the radar to be like, hey just do, just deal with it yeah, I think in practice what happens though is you spend the discretionary budget 15, 20, whatever it might be and often times the end result isn't to fund the rest of it. The end result is, what do we test next? It just depends on the complexity, I guess, of your product too, but, you know, do we continue along this line, or do we stop now? Right? And if we continue, it isn't like, ok, well, we're just gonna invest all of the money now that it takes to bring this thing to market and beyond. It might be the next experiment, right? So sometimes, it kind of goes that way, but you gotta figure that out, right? Because presumably if you're going from one to the next to the next experiment there's enough of a market that eventually is going to make it worthwhile for you risking the farm, so you're not that's my, I guess my diatribe is leading to that, you're not risking the farm by doing it this way it's guarded, and you have the option to stop at any point well, reason that supports the quote review board murder boards, that's another thing that's another that supports the murder boards, but governance is such a four letter word you know, and so is like project review they're so four letter words, it's hard to find they're panel, that kind of thing sorry, I know I'm trying to give it other names to soften it, to kind product review of like direct it into what it should be but without a solid product management function, I find these these things that are they fall through the cracks and they're done by many other functions they don't work very well, and that's that's a lot of what we're talking about today, is like we don't have solid product practices but to that, I don't know if we have a specific, like, area or section of the podcast where we test these things talk about some of the practical maybe takeaways or tips but this, one of the things on that last point you raised is maybe you could assign an impartial facilitator for this product review or whatever you want to call it right and that facilitator is an impartial metrics owner so they don't to, have skin in the game they're just simply looking at the evidence and based on that they're making a decision it, I say it like that it makes it sound like it's one person ideally it should not be one person ideally it might be two to three people yeah and it wouldn't be the same two to three people every time you meet yeah well that was, I mean, in the previous category we said, well this board should evolve slash revolve so that it doesn't get captured by any one person or entity or group or whatever yes and that's still solid advice here solid advice, it's going to be, uh, your, your innovation sandbox with guardrails let's talk about if we have other that, ok, because budget and the timeline, we want to do those two things, right yep, because we want to cap the again we talked about micro-budging, this is a way, like, hey, you can apply for this budget, you can apply for this timeline you want to define the blast radius, the worst thing that can happen if this thing completely explodes doesn't they go together, work, you want to define that up front um, that probably would be a condition of your applying for this money to start, right, you want to, um, come up with some kind of pre-committed success metrics, which is saying you move the needle in whatever area it is, right ideally you want to develop something and then prove that you can move the needle in a certain area and and then you want to make it visible, want to make it visible you want some kind of light touch review by this board that we're talking about um, not a preemptive veto you, what I'm trying to set up is not a, you know absolutely not, do things until, you know, this you get the stamp of this committee or whatever that's not what i'm trying to do i understand different size organizations operate differently but like again all i'm trying to do with these takeaways here is saying some kind of panel that has some kind of cross -functional expertise into different areas of the business that of pool of funding that has some kind of, like, is that of companies um, have this sort of a yeah, I mean, a lot structure in place under different guises you know, um, I'm thinking about Silicon Valley's kind of skunk works um, where sometimes something will come out of it, but the amount of money that gets expended there, it's incredible and you have innovation labs and things like that in large companies that come up with tools, initially these tools are always, almost always um, grown as internal and then, if there's a market we can sell it that that's how they do it, right um, so, I, I agree with that I, I think, you know, if you're having those kinds of structures in place figure out if there's a better way because there is a better way well, serendipity but you can budget for it absolutely yep section, let us know you can't schedule does your org give you room to experiment, uh, do they give you budget to experiment, do let us know in the comments, like, I'm, I'm very interested in different people's hoops they have to jump through to get funding for so with this this kind of thing, we would love to know thank you for, uh, staying this far and also don't forget, while you're here like and subscribe alright, ohm, we've talked about all the biases problems with murder boards and the case for autonomy, now we're now we're gonna pull it all together and we're gonna talk about something you can actually use, we're gonna talk about the kill or nurture framework the decision matrix for all your projects I love me a framework or two listen, you and Boston BCG and consultants all around the world love a two by two so I saved the best for last and we're gonna go up with a two by two here we go so, loves a 2x2, that's what I'm saying every consultant loves a 2x2 and gonna do now, we're that's what we're gonna talk about the kill, nurture framework, it's a decision matrix that, I kind of, mocked up for this section of the podcast and, we're gonna talk about it now hang man right away is ohm, uh, frameworks they oversimplify complex human market dynamics but also, great products often then, and the steel defy rational analysis and you can't stick them in a 2x2 great products can't be put in a 2x2 which I'll say, hey get out of my office yeah, so the first one um, frameworks oversimplify complex human and market dynamics sometimes that's true oftentimes that's a cop -out if you really think about it yes, of course you have human and market dynamics, but you should be able to create some sort of a framework, now granted it could be a little bit fickle, but it's better than nothing, you should be able to come up with that the second one, great products often defy rational analysis early on um, typically people that are not finance minded they really don't like budget meetings they'll say this, they'll say, but, you know I have a gut feel, I know this is gonna work, trust me, uh yeah, trust me bro, right listen, in refutation of the steel man here, I'm gonna, I'm just gonna point out, that frameworks like, they don't make decisions those, those, those are just frameworks they're, they're frameworks to help you like, a two by two is not gonna make a decision for you it's just gonna help you kind of like, guide your decision and you're gonna make a decision anyway so like, don't blame the framework because you don't like what you're looking at right don't blame the backlog because you don't like what you're looking at you make the decision for yourself have an adult conversation with yourself I want to show on the screen, the, the quad like, I created this quad, by the way like, this is not something I just randomly found on the internet I created this, and you can tell because in the evidence category, it says the evidence is, can't find any and on the right it says, tons and tons every product leader tons and tons so it's, so the people listening in the x-axis the x -axis of the passion evidence matrix the x -axis is evidence and on the left you have you can't find any evidence at all and on the right is tons and tons of evidence and on the y -axis the y -axis is passion meaning your company or the people in your company or your department or whatever passion for this thing whatever it is and on the lower part it says we don't like it at all and then the higher part is we like it we love it we love it yeah we like it we love it categories here are pretty straightforward, in so the four the, in the lower left category of, I can't find any evidence, and I don't like it anyway you kill those ideas immediately, you kill those ideas, we're talking about ideas here, the idea of Hunger Games on the podcast, and in the lower right, where you have tons of evidence but you don't like it or, or maybe you don't have any experts in your company right, cause that could go in here as well sure in passion then you find a champion or you go outside your company and you hire a champion in the, in the case of AI right, a lot of people are doing this and, and to this podcast and you're in the Agile community and you know something about this, this like weird yellowish find if you're listening a champion, this, this, this vomit yellow, find a champion sorry, I purposely tried to, color these in the worst colors in the world what are the are the 80s Stranger Things colors I could get to name these, I absolutely did that, yeah, I did that on purpose find a champion, like this could be you outside the company, and in fact I would pause it is you are that companies right now champion you don't know it you don't this is a lot of know it right you just haven't networked you haven't reached out you haven't done whatever to find out that you're that person that that that you have tons and tons of evidence and they internally don't like this thing but you've got the passion for it right so this is stopping you from moving it up a category into the green box right so if you've got tons and tons of evidence and you've got the passion for it you love this thing you're ready to take this thing to the moon, you're ready to scale it, you're in the green box here you might not be directly there, you could go via the purple box, but you certainly can move in the top tier I feel the purple box is the one where oh, well, we're gonna give you these micro budgets, and we're gonna spoon feed you the budget because we know you really like doing this thing, and you're really good at it, but the market hasn't responded yet I've, I've been here in this category as a product manager where I'm trying to educate and build a market, which is the, that purple box, by the way, I'll put will tell you right now, I'll, I'll make a claim on the podcast, um, I can't stand being that purple box, because in that purple box, I hate being there I oh thing your team believes in the thing your executives and your sales people they you believe in the don't believe in the thing the people funding, they don't believe in the thing you're running on a limited budget of willpower and enthusiasm yeah and if you don't make it into a a stream of revenue, that nobody can deny, if you don't move it over to the green box, please let me know in the comments, if you have another take on this one takeaways here, I'm gonna hit them real quick, you know, plot every initiative before you go into that review panel, right, you just wanna have them written down, the sandbox area that we said, which was the, uh, upper left that I said I was super scared of, right, that, that is the, that's the cheapest, what you want to do if you're in that area is you want to define what is the cheapest experiment that we can do to move this purple box forward and keep going, right, before we get killed, right, cause you, you, you're still gonna get killed in that purple box, you're just gonna get assaulted by people internally that are trying to get your money to send it to other boxes, that's right yeah, so don't be, uh, under a fallacy that if you're in the purple box and you have just enough leeway to um, do something that you're gonna end up in the green box directly it could be small steps yeah you'll still stay in the purple box and you may have to go through iterations of this i think that's why i don't like it i it becomes a hunger games where like people can see you're successful and like it's it's it's it's a cutting off at the knees that begins when you're in that like partial success box and they want to take your money and move it somewhere, anyway define the champion box, the lower right box that we pointed out, if no one wants to own it, uh, passionately like, evidence is not gonna save it evidence alone, like, oh, there's definitely a market for this, but we don't have the talent, we don't there's some have the passion, we don't care about it that much, it's really not, our ICP isn't really in that market, yeah, that money's not gonna stay with you yeah, you're really not, and then, you these boxes, really you need a, you need and then for any of real evidence, you need, you need a customer or market signal because the committee alone, even though the committee kind of, like feels like it's going to be a thing because alone is not going to move the needle yeah, you need hard evidence yes, yes, this category the best the best murder so I think out of but their opinion board it's not a room full of executives it's a room full of customers who won't buy your stuff right, right, well charisma with causality founders can be persuasive yeah, um, but the market is the final jury sure, sure we covered a lot of ground don't confuse Ikea effect proven bias here boards, project review boards, scientifically the we covered the product review boards, yeah the intent is not for them to kill initiatives, it's to try to get the best initiatives to move the funding to so that those things have the best chance whether or not they're like well talked about murder executed right that's a that's another conversation which we kind of talked about a little bit here we probably could talk about more we, we talked about letting people build the wobbly chairs right, autonomy you know, that intrinsic motivation which lot of different things and i'm, i'm really wondering what, what people are going to resonate with, with this podcast yeah, yeah, let us know uh, once you get through this because uh, as you said, a lot of things i'd like to know about your thoughts on the 2x2 you know, it's, we talked about a you've seen a lot of those but let us know if you think this is useful if you could use it in your environment if not, what would you use instead? also, let us know about other topics that you want us to delve into and lastly, please like and subscribe we'll see you next time