The Product Experience

How GenAI has improved GoDaddy's product experience - Laka Sriram (VP Generative AI, GoDaddy)

• Mind the Product

Share episode

In this week's podcast episode, we speak to Laka Sriram, VP of Generative AI at GoDaddy, discussing leveraging generative AI to streamline the business setup process for entrepreneurs. We explore Laka's background, the development process of GenAI, customer engagement strategies, the ethical considerations of AI, and the importance of fostering a culture of experimentation within product teams.

Featured Links: Follow Laka on LinkedIn | GoDaddy | GoDaddy Airo 

Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. She’s worked on a diverse range of products – leading the product teams through discovery, prototyping, testing and delivery. Lily also founded ProductTank Bristol and runs ProductCamp in Bristol and Bath.

Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.

Lily Smith :

This week on Product Experience, I'm joined by Lekha Shriram, the VP of Generative AI at GoDaddy. Lekha and his team focus on leveraging AI tech to build products that help new businesses scale and be more efficient with their time. In this chat, we cover his background at Alexa, how his team at GoDaddy set their AI strategy and how they launched a generative AI tool. The product experience hosts are me, lily Smith, host by night and chief product officer by day.

Randy Silver:

And me Randy Silver also host by night, and I spend my days working with product and leadership teams, helping their teams to do amazing work. Louron Pratt is our producer and Luke Smith is our editor, and our theme music is from product community legend Arnie Kittler's band Pow. Thanks to them for letting us use their track.

Lily Smith :

Hi Luka, really great to meet you. Welcome to the podcast.

Laka Sriram:

Thank you, Lily. Thank you Randy, Excited to be here.

Lily Smith :

So before we get stuck into our topic for today, which I'm very excited to talk about because I always love a new product launch, it would be great to give us a quick intro to who you are and your background in product and what you're doing today.

Laka Sriram:

Of course. So I am Laka Sridam. I am a VP of product at GoDaddy and I lead a brand new experience and product stream called Edo at the company. I have been here for three years now and prior to that I was at Amazon for a bunch of years seven years and overall I have many years of work experience in the software industry. I have been in four companies in this span of 20 years, which I'm thrilled about that Like I go into one and spend enough time there and I want to make a difference in each one and then I exit whenever I see the right time. So four companies overall 20 years, including the GoDaddy one. Been in some of the large-scale tech companies that have been obsessed with innovation, making customer impact and delivering customer value at a rapid pace. So that has been a constant theme during my different companies that I've worked with.

Lily Smith :

Awesome. Thanks very much. I always think it's really interesting that dynamic between, or the difference between, working in lots of companies for a short amount of time, which has mostly been my time working in tech, versus, you know, getting really deep into a company and you know spending a lot of time there and really understanding it really well and and stuff. So it's interesting to you know to hear your perspective today and and to see how how you've worked in products and how you've kind of grown in your in your product career, taking that kind of path. So tell us that you mentioned you worked at amazon. I know I know you worked on alexa. Tell us you mentioned you worked at Amazon. I know you worked on Alexa. Tell us a little bit about what that was like.

Laka Sriram:

Yeah, it was a fascinating period of my career. So when I joined Alexa, we were a very small group and it started growing at a rapid pace in terms of number of customers in terms of number of features in terms of expanding at scale, multiple markets and so on, and the number of people as well Within the odd.

Laka Sriram:

We were expanding at a tremendous pace. By the time I left, we were at more than 10,000 people in Alexa. So it was a fascinating time of growth, especially when I was at Alexa. I was in the speech organization which was responsible for the models that would take in the audio that comes in from the customer through the device onto the cloud, processing it, identifying the right slots and tokens and determining what needs to, how to respond back to the customer. And I was wearing bits and hats over the period of time, Like I was there, like I said, I was there for seven years in that same hall and at one point I was leading, making Alexa faster, and at one point I was looking at geo location of customers and how do we handle the input based on the location that we were in.

Laka Sriram:

I was part of our launch outside of the US and also into mobile apps and so on. So there was a lot of different hats and went through different timelines as well. There was this initial massive growth period. It was really exciting, especially like a real startup and with a lot of funding and so we had to go fast, and that was exciting. And then came a phase where, okay, you've grown really big, now let's make sure that you have the right accountability, and so on. So it was being in those two phases in the same org, in the same org, in a company like amazon, which is known for its amazing leadership principles and the way it addresses customers and focuses on the customers. It was.

Lily Smith :

It was an impactful period of my career and I I've taken a lot of things from it for sure and it would be great to kind of understand sort of what you've taken from that period into go daddy, because obviously you're working on kind of groundbreaking products at Amazon like Alexa. When it came out there was nothing else like it was there. So now working with Gen AI at GoDaddy, it's a similar sort of thing where Gen AI is such a new technology for for everyone, I think for most people, majority of people.

Lily Smith :

So it'd be really interesting to hear sort of the, the kinds of things that you learned, and the, the principles or the, the practices that you you learned at Amazon and how you've taken that into GoDaddy.

Laka Sriram:

Yeah, it's been a huge influence and coming in that was, I think, part of my as well to bring in some new ideas into the company and to make us move faster. And so some of the main things were the focus on customer what is the customer impact? And just working backwards from that, it was in a different form when I was working at Cypress Semiconductors, initially as a software engineer as well, like my first 10 years. But at Amazon, that's treated as a very different scale. Like it truly everyone starts from the customer and unless you have a customer impact and that's significant and you're making a difference for the customer, things don't move forward at all.

Laka Sriram:

So, bringing that in into a landscape like Gen AI, to moving really rapidly and there's so many options for us to choose from. Like it's not a lack of ideas, which, once you're into using Gen AI, there's so many ideas and so many shiny objects and the options are plenty. So it's more the anchor of how do you add value to the customer. Like is it a true value or is it like a one-time thing that they would use and then they would go away? Right, Like a meaningful, sustained customer value. I think that was a huge thing. The invent and simplify is something that's really close to me. It's doing something new, but doing it in a way where we actually simplify. Simplify things for us, simplify things for the customer and that I use that term quite a bit in my teams and people that I work with is how do we simplify stuff and combination of simplification, moving fast and truly delivering meaningful customer impact. In a landscape like GenAI, I think that helps us keep our focus on as well as meaningful stuff.

Lily Smith :

And it would be great if you could just tell us a bit more about Aero, the product. I'm sure lots of people, the majority of people, probably know GoDaddy as a business. But yeah, tell us a bit about Aero and how that came to be.

Laka Sriram:

Yeah. So Aero fundamentally is an experience that creates a magical, seamless set of things for a customer. And I say set of things, it's a set of products, set of features that get auto-generated with customized content that's relevant for the business idea that they came in with. So, like you said, godaddy is well known for its domain business. A lot of people out there when they think of GoDaddy they think of okay, that's the place where I go to buy a domain, and so that is our biggest fun coming in.

Laka Sriram:

But we are much more than just a domain business. We have a set of product and features that help the entrepreneur every aspect of their business journey. We call it the entrepreneur's view. So the entrepreneur's view circles around identity, presence and commerce aspects of a small micro business owner, and so what we really wanted to do with Arrow was okay, the biggest funnel still remains as domain, and so when the customer comes in, when they complete the purchase of a domain, that is most likely one of the starting points of them trying to establish their identity or their presence.

Laka Sriram:

So we even simplified that even more. When a customer comes in, we ask them tell me your business idea, what is it that you're thinking about Explain that in plain English. And then Aero kicks in. It's able to help the customer land on the right domain for them using generative AI, which is all in models and so we do have some in-house models as well and then, after they complete the domain purchase of the domain, the rest of the products and features they get auto-generated and just laid out in front of them Like. Imagine what Aero does is create a brand new website for them, creates a community page and a full website there are two versions of it and it creates a set of logos that resonates with their business idea.

Laka Sriram:

It creates an email account and email signature. It creates a marketing calendar with a large set of social posts that's's ready for them to post, email marketing, and so on and so forth. There are plenty more. There's social handles, there is site optimizations. It just keeps on going. So, as a customer, we see that the main bottleneck for a small micro-business owner who has like one to ten employees in the business which is our main customer cohort, but their main bottleneck is time and effort, right, and so we were anchoring around how do we save time, how do we save effort for our customers and help them get more customers? And Aero does that in a splendid way.

Lily Smith :

And you've kind of talked a little bit about this. But how and when did you decide to invest in arrow, like, how did you come up with the? I mean, you've talked a bit about how you came up with the concept of you know the, the domain being like the starting point for the new business idea which absolutely, I have said had so many new business ideas and just gone straight to finding a domain. So I can definitely relate to that as the starting point of like a business is you know, going to find that domain and stuff. But you know what made you decide that this product with GenAI was like the right next step, I guess, for GoDaddy.

Laka Sriram:

Yeah, so it was about simplification for the customer. I talked about helping the customer save time, save effort and helping them grow and get more customers. You're thinking about what does the customer actually want? How can we help them? And before Aero started, we did a full seven months where we just asked our entire teams to stop whatever else they were doing and focus on how can we best use Genkiai to help the customer with these three things that I said.

Laka Sriram:

That was again, I think one of the reasons why we were able to move this fast, we were able to have this impact with Arrow is the alignment that we've had within the company, across the leaders, across the execs, starting from the CEO to our president, to like okay, yes, we have the green light, this is an important area for us to focus on. Let's clear the roadmaps, let's focus on this and the reason why also lined up with that. And so a lot of people came together really quickly with a common goal and common value add, and then we were able to figure that out to okay, I just need the X number of people to focus on this for now and then let's see how fast we can go with it. So the core group became really small, but then we were able to go together and move fast towards a common goal with no roadblocks in sight, like from the company entirely, and we were just able to solve multiple features that added value to the customers and that was just spread across all our products and features. That was the start.

Laka Sriram:

It was how can we use Android AI in so many areas of our products and features that helps the customer overall? And then came okay, we were able to prove ourselves that all of these things work and we've added value to customers. Let's bring them together into a seamless experience across the entrepreneur's view, which again is a strength of GoDaddy because we're able to have so many products and features. That touches the customer in every part of their journey, and we just brought all of that together. Touches the customer in every part of their journey, and we just brought all of that together and built that experience called Aero. So I think overall it took several steps for us to get there, but in the end it just came together as a magical experience.

Lily Smith :

I think this is really interesting because so often we are as product people, we're kind of like coach, to like always focus on sort of solving customer problems and, you know, let the data guide you all of that. But when you throw in quite a big shift, or in terms of like what, the generative AI capabilities, or like when technology suddenly advances in that way, you kind of just have to have a bit of a reset, don't you? Because otherwise you can just keep on moving at the pace that you're moving with the technology that you're kind of using on a day-to-day basis and not sort of like stopping and reflecting. And it it's great to hear you know such a big organization like just saying, yeah, we actually we just stopped and we just went right. We know our customers really well, we know our customer journeys, like we've got this new tool, how do we explore that? And so it's really interesting to hear that that's the process. Was that kind of the leadership team that were saying that, or was it just everyone? Or how did that kind of come about?

Laka Sriram:

It was a few of us saying it and we had to prove ourselves. We did a proof of concept to showcase that, hey, this is going to add value. We took one area. We took our Biffy area. Specifically, we took our Do it For you area, where we were able to infuse generative eye in there and help our own agents. So this is a feature set of products that we have that our agents, or GoDaddy agents, help external customers with the things for them. So let's do it for you. So I chose that field because we had a middle layer which is within our ecosystem right. So we were able to test all of these, several of these new Genitiv AR features. In that space.

Laka Sriram:

We were able to get feedback from these agents before they went in front of customers, so it was like a human layer before it went to customers. And once we were able to do that really well and we got the feedback that, hey, this is adding tremendous value in terms of productivity, in terms of saving time, in terms of the quality of output that we saw. So it was a combination of all of that. So the proof of concept worked. So we were able to get the data out of it. So we ran experiments. Certain experiments won and we were able to use that to author a document to convince our execs, like our GLT members, our execs even further about it.

Lily Smith :

Yeah, and was part of that process as well also a kind of upskilling the development team that you have and just like playing around with the technology and like learning its capability and and how to apply it within your own sort of like tech stack and everything yeah, that's.

Laka Sriram:

That was a really interesting part when I was going through it. Right, if I look back, we barely hired anybody new as we were kicking off this whole new wing brand, new experience for our customers, whole new process. It was the same people or it continues to be the same people that we had in GoDaddy. Like GoDaddy has been evolving and changing over the last five years especially, we've rapidly evolved and changed since the last five years. Five years ago we had a new CEO and from then on we've changed quite a bit. Our brand, our company has changed a lot compared to the old GoDaddy in terms of culture, in terms of people, in terms of everything. But over the last two years we didn't hire anybody new. We didn't hire that many people, you know, just for the Aero experience.

Laka Sriram:

So it was upscaling. Is that I do it? It was upscaling existing folks, yes, but for me it was easier than what I had expected because of the synergy that came in with. Hey, there was a grind that we all had. There was, there was this huge positive attitude and motivational factor that, hey, there is this new technology and we are able to use it. We've seen proof points of it. Oh, this is really exciting. There is a lot of potential in it. Let's all get to it. So the fact that we were able, we are able to do something special and add a lot of value, new value in an area that hasn't been done before kind of self-motivated, all of us, we're able to rise up I think it just happened naturally after that absolutely offer that Product people.

Randy Silver:

are you ready? The word on the street is true. Mtp Con London is back in 2025.

Lily Smith :

We're very excited for Mind the Product's return to the Barbican next March. Whenever I hear people talking about the best product conferences, mine the product is always top of the list. If you've been before, you know what's in store. Oh, that rhymes. New insights, strategies, hands-on learnings from the absolute best in the field, plus great networking opportunities. And if you're joining us for the first time, I promise you won't be disappointed.

Randy Silver:

We've got one speaker already announced. That's Leah Tarrin, who you've heard on this very podcast. With more to come. From the likes of WhatsApp, the Financial Times and Google, you know real people working in the field who will share real actionable insights to level up your game as a product manager.

Lily Smith :

Whether you're coming to the Barbican in person on March 10th and 11th or tuning in digitally, join us and get inspired at MTPCon. London. Tickets are on sale now. Check out mindtheproductcom forward slash MTPCon to find out more, or just click on events at the top. Since it launched, how has it been received? Like how? How much have you learned about explaining ai tools to the general public? And you know how. How has that worked? Aero?

Laka Sriram:

as an experience is seamless. It sounds like the customer doesn't. They just see many things getting auto-generated for them and it just resonates with what they want to do. So it's not like we are putting it in their faces that, hey, we're using AI or you are using AI, so it's just helping them along the way to get to that next job to be done for them to do whatever they need as their next step. So that is one thing, but overall, the usage of AI we've had plenty of data points that are encouraging and shows how the productivity value has increased when they use AI and small and micro businesses owners have been.

Laka Sriram:

The percentage of them using AI has been growing consistently. Initially it was less than 50% were using AI in some shape or form, either knowingly or unknowingly, and that has risen quite a bit since then. I think it's a combination of just seeing value being added and saving time for them and doing it in a seamless manner. It's just more and more people are just using it and getting value out of it. From stats, we know 4 million customers have used the ILO experience so far this year and more than 50% of them have engaged with the auto-generated product and feature in a deep manner. So those are really encouraging positive metrics that we have and the wide, huge number of people that are actually using it and how they're using it. As we look into our own data sets and pull up all these metrics around what works, what doesn't work it helps us go deeper into specific areas Like logos. In the area that we're continuing to invest and we're building more, we've already built a more advanced logo generator and launched it recently.

Laka Sriram:

We found that when a customer uses a logo in their presence, whether it's a social post or it is a website they get again just a super engagement metric improved by more than 20%. Their legitimacy, or them having a professional brand value, just increased significantly because they now have a logo. So, we've gone deeper into that. We've gone deeper into social posting. We've gone deeper into social posting. We've gone deeper into site optimization, for example.

Lily Smith :

And has it gone wrong in any way at all Initially?

Laka Sriram:

we were really careful about building an image generator, that we've launched an advanced image generator recently. But we didn't do it until recently because of the skepticism that we had about, first, initially, the quality of the image that was being generated and the relevance of it. So we were waiting for the technology to improve quite a bit and make sure that it meets our bar so that we're producing the right things for our customers. So for the first at least a year or so, even though the models were out there, they were multimodal and had images in it, we refrained from exposing it to our customers. We've done that. A month ago We've launched the advanced image generator, advanced logo generator, because it passed our bar. We now have the confidence that we're able to do it right for our customers. But we had to go through that journey.

Lily Smith :

And actually that's a really interesting point. You know, when you're testing AI, like how do you approach QA and testing for a product like this?

Laka Sriram:

So I think there are multiple angles of tests that we use. We use technology-based tests to begin score for the outputs, and so the scoring needs to meet a certain bar that we have. And there is user testing that we do Like once we have a certain type of feature, we run tests with a specific user group and you get their feedback and they they go at it from all angles and represent like the variety of customer cohorts that we have in the wire, and so we get that feedback. And obviously we go through our own internal engineering testing, whether it's UX testing or software testing. So it's a combination of those for us to be confident enough that hey, it's adding value, it's at the right quality, it covers edge cases and so on.

Lily Smith :

Which kind of sounds like your normal software testing, right? So there's nothing special about testing AI products basically.

Laka Sriram:

It sounds like it's become normal software testing. It's part of what especially a good idea. Like AI, general AI is infused in every team who's looking at using it in order to again either improve internal productivity or improving value for customers. So certainly a part of general software testing right now, but using the AI scores the quality of the AI output. That is something new and we're adding more and more dimensions to the software testing.

Lily Smith :

Okay, sorry, I maybe missed that a minute ago when you mentioned it. So how have you created scoring for the AI? So scoring.

Laka Sriram:

Like a lot of these external models that we use, they also produce a score for it. So there is a score we use only when it's above a certain score and the bar is relatively high for us. And so when it's above a certain score and the bar is really high for us, and so when it's below a certain score, we discard it or we figure out how to get to that fine-tuned space where, again, we use RAC, we use other things where we're able to inject additional context, additional data points to produce a much higher quality output. And, yeah, so those are starting points. But we also again have certain scorings within our own platform layer.

Laka Sriram:

By the way, we have a platform layer that we call GoCast, which is GoDaddy content as a service. It acts as a proxy for all of the AI models that are external as well as internal. That has been a huge leverage that all of our teams use in order to accelerate the user-generated AI. So for us, whenever there is a new model, whether it's external or built by our ML team, it's just available behind that proxy layer. So the way any team would use an ML model is through the same path, right? So that has certainly helped simplify how teams can use Gen AI models and how teams can stay up to date with newer and versions of models. And that layer also has its own scores and reliability and all of those metrics in it that prevent something from going from the bottom layer to a top layer interesting cool.

Lily Smith :

So I had a question about ethics as well. You know, whenever we talk about ai these days, I feel like we're always talking about ethics in the same breath. One of the things that I thought was really interesting about this is obviously they're at the moment, or were maybe in the past have been people that are doing freelance work and they're doing their logo generation. They're building these micro sites or landing pages for these small business owners probably people on fiverr and and things like that and uh, and freelance web builders. I guess, from an ethics point of view, like, how have you approached this question of like? You know, you're displacing, like some people's income or or work, and it's quite a loaded, loaded question. So I apologize, but it's interesting to hear how you're thinking about that and how a product like this, like, really changes the landscape for a certain group of people.

Laka Sriram:

Yeah, it's a great question. I love the space of ethics and AI in general. I'd love to talk about what actually ethics means in AI and what areas that we should dive into. But to your question, I strongly believe that generative AI, when used in the right manner, it enhances the individual's ability to actually produce quality stuff.

Laka Sriram:

It's not a replacement for the individual's ability to actually produce quality stuff. It's not a replacement for the individual, but it's a tool where every individual should use in order to enhance what they are capable of, especially in terms of content generation, like you mentioned logo. So I believe that it's not going to replace jobs by itself, but a person who is using generative AI in order to produce that output is going to replace another person's job who's not using it at all. I see it as an accelerator, as an additional tool and an enhancement vehicle for individuals as well as agencies, to get better at what they currently do and produce even better quality outputs at a faster time to grow their individual business, whether they are a solopreneur or an agency like so awesome, yeah, and it's a very good point, and, and so I guess the other angle to this is like what you use to train your models to help come up with these landing pages and with the logo designs.

Laka Sriram:

It's about being responsible, being fair and producing output that benefits humanity in general. So the sense of accountability and sense of social impact has to be intruded when you are using generative AI at scale and especially, we take that seriously at our company because it's intruded in, like I said, almost every aspect of what we do right now, because it's integrated in, like I said, almost every aspect of what we do right now. So I think, just making sure that, again, going through our rounds of testing and making sure that our outputs are fair and buy are fair and we handle, make sure they don't have bias in certain things, and so that going through the round of testing, going through that hard output, it meets our quality, bar the emphasis that we do over there, I think it helps us be in the right spot in terms of doing the right thing using AI. But, having said that, we also don't push anything out there into the internet or into the wild without the customer viewing it, looking at it and approving it. So everything that we produce, it's being produced while being engaged with the customer.

Laka Sriram:

We handhold the customer.

Laka Sriram:

Like we handhold the customer, we try to understand in a deeper way about what it is that they want to do, like, for example, the advanced level generation, and then we provide recommendations about, like a set of logos or different font styles, different colors, things like that, and different icons, that again the customer can choose what to do next with it.

Laka Sriram:

They can say, okay, this does not resonate with what I want, or this doesn't include either the gender or a specific area that they are interested in or thinking of, and they can say, hey, give me something else. Or they can add more information, like I want this to include this particular variety, and then we we refresh based on that and then then they choose what to do with it. Then they choose once they're happy with the product, they can use it. Same thing with the website, right, whatever we are. We auto generate a full website in a matter of seconds and it's ready for the customer to review. But it's, they are in the driver's suite, right, right. So they need to approve it in order for the website to get published.

Lily Smith :

They can not spare the regenerate specific sections or even regenerate the whole thing, and it will do it for them one of the other things that I know that you are very passionate about is developing or creating a culture of experimentation. I'm sorry we're deviating a little bit away from the AI and the ethics, but we're running out of time, so I just wanted to make sure that we touched on this as well. It would be great. You know you've got 20 years of experience working in some great companies. It would be great to hear a little bit as well about how you experiment within the organization that you work in and how you create that culture.

Laka Sriram:

Yeah for sure. So it's evolved quite a bit. If you look at my journey at different companies, the culture of experimentation has evolved quite a bit, like when I was at Cypress Semiconductors I was in the software industry, a chip manufacturing company the experimentation culture there was. There's heavy guardrails around experimentation and the experiments where it's slow and methodical and systematic because you don't want to get a chip problem in a hardware.

Randy Silver:

And at.

Laka Sriram:

Amazon, it was much more rapid than that, obviously, and there was this huge principle around fail fast. So, amazon, I was anchored around sure, it's okay to fail, and when you know you're failing, just fail fast and get the learnings out of it and move on to the next one. So that helped moving fast and empowering us and empowering me to experiment. And, godaddy, it's full on experimentation, like when I say that every piece of software that we push out goes through an experiment first and we measure. It's an A-B test or sometimes brand new ones, it's an A-A test as well and we measure the outputs between the treatment control and we prove to ourselves that the treatment cohort is the winner and then we push it out. So it's a full-on experimentation culture and it's fascinating to be a part of it and how it empowers us to move fast and move in a more meaningful manner, because every change that we do we're not guessing that hey it's going to have a positive impact.

Lily Smith :

We know it's going to have a positive impact before we get there it's really interesting to hear you talking about it because, you know, I've been trying to build this culture of experimentation as well at bbc maestro, and I think I had like a proud man moment, which I've got three boys, so I always, you know, think of like my team, as my, my children which is terrible, I shouldn't say that but I definitely had this sort of proud man moment when we were talking about this feature and the team were like, oh yeah, we really want to test this so that we can prove that it's, you know, prove the impact that it's going to have, and we want to be able to show to the rest of the business, like you know, it uplifted conversion by this amount.

Lily Smith :

I can't remember what the feature was now actually and I was just like, no, no, we haven't got time and I know people are definitely going to want it, we just need to put it out. It was a bit more of a sort of table stakes sort of weakness, but I was so excited that they were, you know, so passionate about wanting to test it so that they could demonstrate to everyone the impact of some. You know that something was going to have and even though it was something fairly obvious. They still wanted to do it anyway, which was um, which was really great.

Laka Sriram:

And it leads to a data-driven culture, right, like you're using data to actually look at results. And also even there, just like what we did recently or what we've been doing here, is celebrating the failures as well in these experiments, and we naturally tend to celebrate successes, obviously, because the impact is there and we have the data to prove it. But failures are such a critical part of this experimentation culture. It helps us. Our hypothesis becomes a much bigger swing. The bar moves higher if we celebrate failures as well, like we learn a ton of things from these failures and helps us do better experiments overall, and that has been something that we're focusing on here as well yeah, it's.

Lily Smith :

Uh. Yeah, I did. We did a test the other day that negatively impacted our conversion by 20 percent. We were like, okay, let's learn from that one. Yeah, it is great, like unfortunately we've run out of time, but it's been so great chatting to you today.

Randy Silver:

Thank you so much for joining us on the podcast yeah, thank you so much, it's fantastic the product experience hosts are me, lily smith, host by night and chief product officer by day and me randy silver also host by night, and I spend my days working with product and leadership teams, helping their teams to do amazing work.

Lily Smith :

Luran Pratt is our producer and Luke Smith is our editor.

Randy Silver:

And our theme music is from product community legend Arnie Kittler's band Pow. Thanks to them for letting us use their track. Thank you.