Google VP of Product on The Future of Search and AI Mode | Robby Stein | E287

In this episode, Carlos Gonzalez de Villaumbrosia, CEO at Product School, sits down with Robby Stein, VP of Product at Google Search. Google Search serves billions of users and holds over 90% of the global market share, acting as the engine behind Alphabet's $400 billion in annual revenue.

Robby is steering the most significant shift in Search history: the transition to AI. He oversees a massive portfolio, including the new AI Mode, which has already scaled to 75 million daily active users. Drawing from his time building Stories and Reels at Instagram, Robby breaks down how to build zero-to-one products inside a tech giant and why you need high conviction to push through early bad data.

What you’ll learn:

• Building Like a Startup: How to maintain crazy speed and focus on zero-to-one initiatives while protecting a massive core business.
• True Product-Market Fit: Why flat or J-curve retention in early cohorts is the only reliable indicator that a product is actually working.
• The Value of Colossal Disasters: The untold story of how early failures with Instagram Reels and Close Friends were necessary steps to global success.
• Agentic Search: How Google is moving beyond providing links to executing complex, multi-step tasks by deeply understanding personal context.

Key takeaways:

• Start Small to Win Big: Even at Google's scale, massive AI products begin with just 500 trusted testers and a focus on solving specific user complaints.
• Look for the Golf Shot: When building AI, look for that rare moment when the whole system works perfectly to build the conviction needed to keep iterating.
• Leaders Must Co-Create: To move fast in a large org, leaders shouldn't just approve from the top; they need to form working groups and operate in the details.

Credits:
Host: Carlos Gonzalez de Villaumbrosia
Guest: Robby Stein

Social Links:

• Find out more about Product School here
• Follow our Podcast on TikTok here
• Follow Product School on LinkedIn here

Transcript

Introduction

Robby Stein | Google 00:00:00

As a founder, as a leader, everyone needs to think what's going to be the most important thing that if we get it right, this is potentially a five or 10-year opportunity of growth, of value, of helpfulness that we can give people. Our journey with AI mode was like starting with AI overviews, building more sophisticated models and letting you ask these more harder questions.

Robby Stein | Google 00:00:21

And the first versions—nothing starts great. I would ask it like one or two questions that were really hard and it just nailed it. It's kind of like when you hit a golf ball and you just hit a perfect golf shot, it all comes together and then you don't do that again for a while, but you're kind of like, I know what is possible to have a system that can get stuff done, book restaurant reservations for you, but it knows your tastes well.

Robby Stein | Google 00:00:44

Search can be really useful for you specifically in a way that really just didn't exist many years ago.

Carlos González de Villaumbrosia | Product School 00:00:51

Hey, this is Carlos, CEO at Product School and your host on the Product Podcast. Today's guest is Robbie Stein, VP of Product at Google Search. Google Search is arguably the most significant product of the 21st century, serving billions of users and holding over 90% of the global market share as Alphabet surpassed 400 billion in annual revenue in 2025.

Carlos González de Villaumbrosia | Product School 00:01:10

Search remains its engine accounting for over 50% of that number. Robbie is currently steering the most significant shift in search history. The transition to AI, he oversees a massive portfolio including AI overviews, Google Lens, and the new AI mode, which has already scaled to 75 million daily active users.

Carlos González de Villaumbrosia | Product School 00:01:28

Before returning to Google, Robbie was the head of consumer product at Instagram where he led the teams that built stories and reels. In our conversation, we skip the high level fluff and get into the weeds of how you build like a startup even inside a giant company, and the specific metrics that actually matter why flat retention in your early cohorts is the only real sign of product market fit, and the untold story of how colossal disasters with Instagram reels and close friends were the necessary steps to eventual global success.

Carlos González de Villaumbrosia | Product School 00:01:56

This is a masterclass in building products at the largest scale on earth. Let's get into it. Welcome to the product podcast, Robbie.

Robby Stein | Google 00:02:03

Thanks for having me.

The "Boomerang" Journey

Carlos González de Villaumbrosia | Product School 00:02:04

Okay, so you are a Google Boomerang, right?

Robby Stein | Google 00:02:08

It's what they say. Yeah.

Carlos González de Villaumbrosia | Product School 00:02:11

Well, currently he is the VP of product for Google Search. I'd love to learn a little bit more about kind of what happened between your previous run at Google and your current one.

Robby Stein | Google 00:02:21

Yeah, well, I started in Google in 2007 as rotational program and worked on Gmail and ads and then I left. And, you know, I've always dreamed of founding a company. Wanted to try that. And, you know, I was kind of obsessed with the various ideas around recommendations and kind of the ranking space at the time.

Robby Stein | Google 00:02:39

And so I founded a company, it was called Stamped back in 2010 or so. It was mostly former Google team. And, you know, we basically, over the years now, I was ended up kind of fast forward. There were about two startups that I, I ended up working on, the other one being the most recent one, called Artifact, which was a AI powered newsfeed that I worked on with the Instagram founders.

Robby Stein | Google 00:03:02

And then I also led consumer products at Instagram, building out a lot of the kind of consumer product suite across stories, reels, ranking the DM and messaging system. So really focused since that little ad stint at Google back at the first time around on consumer products and really building things around information and connection over pretty much my entire career.

AI Transformation of Search

Carlos González de Villaumbrosia | Product School 00:03:26

Got it. So for the last two years, you've been leading search at Google and one of the biggest highlights is now this AI mode that we cannot see.

Robby Stein | Google 00:03:35

Yeah, exactly. Oh, I, I, coming up on almost two years and, you know, and really spending a lot of my time on the AI transformation of search. 'Cause there's obviously this incredible moment in time where it's possible for Google to just do so much for people and really live up to this very timeless mission around organizing the world's information and making it accessible to anyone. I mean, that's, it's only now truly possible.

Robby Stein | Google 00:03:57

And so spent the last two years building out and growing AI overviews, which was kind of at the AI experience at the top of the page lens and the visual search experiences, which I'm happy to talk about. And then the most recent one was run AI mode, which was really a full conversational kind of AI mode to Google search, which has been really exciting to see grow.

Carlos González de Villaumbrosia | Product School 00:04:15

So just to put things in, into perspective, what's the scale of Google search today?

Robby Stein | Google 00:04:21

Yeah, I mean, Google search serves billions and billions of users and, you know, our AI products, are, have grown tremendously to be a billion users as well. And so what's interesting is you kind of, what we're seeing is that at very large scale, people are not just doing search like they usually do, but they're also asking these really natural language specific questions, and those are growing along with visual search where people are taking photos of their natural world around them, of their shoes, of they take screenshots on their phone and ask about where they could buy clothes.

Robby Stein | Google 00:04:54

Similarly, so you've seen this tremendous—and this, again, these are a billion user kind of product scale—new use cases for search and for information that really didn't exist even a couple years ago.

AI Mode vs. Traditional Search

Carlos González de Villaumbrosia | Product School 00:05:07

I think that's one of the things that I, I'm very curious about. It sounds like new use cases in many ways. At the same time, it can be a replacement of an existing use case for some people. Right. So, curious to know how you're thinking about this today. AI mode is, is basically a toggle, right? You can switch between both experiences. Is that the forever intention or do you see a situation where AI mode becomes the default?

Robby Stein | Google 00:05:34

Yeah, I think the way we think about it is, search does so much for people. It's so broad. It's everything from someone trying to file their taxes and file, find a form to looking up a very specific image of a carpet they found. You know, it it's so broad and for many questions, actually AI isn't even particularly helpful.

Robby Stein | Google 00:05:54

And the model, the system actually learns that if you're just doing a very quick search for something or you're trying to find an image, actually just getting directly at that piece of content is most useful. But in terms of where things are growing and expanding, now it's possible for people to ask, you know, really anything that's on their mind and get information.

Robby Stein | Google 00:06:14

So those use cases are growing the fastest. And so what we found is that AI mode is really great for those. But actually think about it, it's a very specific use case. I think people kind of know when they have a more complicated question in mind. And for those we think AI mode is particularly helpful. 'Cause you can ask for follow-up questions.

Robby Stein | Google 00:06:30

You could put in multiple sentences. Say I'm taking a trip, I have a friend, here's one has an allergy, the other has a dog. We wanna sit outside. We want to know where our hotel is. Like we can Google can handle those kinds of questions now using all of the real time information systems and Google Maps with travel, et cetera.

Robby Stein | Google 00:06:44

And that's really neat. But that's a very specific use case. And so our goal is that for people who have these power user kind of needs, they can go right to AI mode and there's a way to do that. But for other people, you should be able just to put in whatever you want right into Google search.

Robby Stein | Google 00:06:59

And if AI is helpful, you'll see that little preview show up at the top of the page. And increasingly, we actually, I just announced an experiment that makes this even easier for people. Once you click into that AI preview, you're effectively gonna be able to go into AI mode and have a back and forth conversation with very little friction.

Robby Stein | Google 00:07:18

So you don't need to know like, oh, should I use the AI mode? Should I put this in search? You can just put anything you want right into the search box, you get this little AI preview and if you click on it, you can have a follow up and a back and forth in AI mode. And that's all one continuous experience.

Robby Stein | Google 00:07:32

So I don't think we're gonna view any change in terms of how we're thinking about the default experience. 'Cause that's what's used at at kind of the billions of users scale at this point. And and so, and it works great for all of these use cases. But it turns out AI is useful for these, this new generation of questions.

Robby Stein | Google 00:07:51

And so if you can give people direct access to AI when it's needed, that's gonna be the most powerful model. It's this really kind of expansionary opportunity for us.

Carlos González de Villaumbrosia | Product School 00:07:59

Yeah, and the way I'm thinking about this in the in the future as AI become even more mainstream, is that we'll probably stop thinking about, is this an AI use case or a non-AI use case? Ultimately have a problem. I come and I find my solution, and it's kind of up to the product builder to decide how they're going to orchestrate that solution.

Measuring Product-Market Fit

Robby Stein | Google 00:08:17

I think that's right and I think that that right now AI has been a power user focused type of technologies. That's very increasingly not the case where everyone kind of, I think has the expectation that you can ask natural language questions, you can have a follow up question. Technology can handle that and can understand that.

Robby Stein | Google 00:08:29

And so I do think that increasingly in the future, people will just put in information. Could be short, could be long, could be a photo, could be an uploaded document that says, Hey, it makes sense of this, it's too technical for me. I don't get it. And you put all of that right into the Google search box.

Robby Stein | Google 00:08:46

And depending on the tools available, it's possible AI and an AI chat like experience is most helpful in helping you resolve your issue. But it's also possible that you wanna actually browse, see what's out there, see images and and you have a different kind of experience.

Carlos González de Villaumbrosia | Product School 00:09:01

I want to geek out with you on AI mode and and based on your own experience, not only at Google, but also drawing connections to your experience leading consumer products at Instagram. And one of the things that I'm very curious about is the real success metrics under the hood, right? Like beyond, okay, how many people find what they're looking for after X amount of queries? If we were to look under the hood, what would be some of those leading indicators for you?

Robby Stein | Google 00:09:28

I think whenever you build a new product, and this has been true, whether I've been working on startups, or working on new features, whether at Instagram or at Google. I mean, I think that the main thing that you wanna look for is that there's a group of people and there's a need that you're solving for them really clearly.

Robby Stein | Google 00:09:36

And the best manifestation of that is first, it's that it sounds silly, but you. Like even AI mode started this way. We gave it to 500 people and many of them were friends and family, and there were people that we kind of could talk to and and engage with. And so, you know, it's a trusted tester kind of environment where it's it's relatively small end.

Robby Stein | Google 00:09:55

As crazy as it sounds for a product as big as Google, every great new thing starts with the first 10 people that use it. And if they don't like it and you can't get 10 people to like it, you're certainly not gonna get a billion people to like it.

Robby Stein | Google 00:10:14

And so we actually started with a really small group and with people internally, and the first kind of moment is when people went from complaining to me to telling me how useful it was. There was a distinct moment where I was like, wow, most of the people who were talking to me in the hallway or you know, we had a couple external people who were on it early who were sending me bugs and things that didn't do were like, wow, I just realized that I'm using AI mode all the time and it's just like completely natural now and it solved like 10 things I would've never been able to get from Google before.

Robby Stein | Google 00:10:45

And I just started—we started seeing that. You look at surveys as well and just the percentage of people who said that they would enjoy it and come back and we just kinda, you kinda see a sense that your kind of default—the people you give it to seem to like it and come back.

Robby Stein | Google 00:11:00

And then I think you kinda wanna verify that more quantitatively. So from then we actually went to labs and it was a very small US Labs launch, where people could opt into it. But we weren't really pushing it, we weren't really trying to market it. It was purely just to get a read on on what people thought.

Robby Stein | Google 00:11:20

And so we made it available, you know, to, let's say the thousands or or hundreds of thousands of people. And then millions. But again, it's a relatively small number relative to the billions of people that use Google, of course. And for there, you're really looking at a few things.

Robby Stein | Google 00:11:39

There's a distinct signature, I think, when you have product market fit, where if you you look at the group of people who are using it, you know, on day zero and you follow them through day 30, 60, 90, and you you wanna see that tapering and you wanna see that curve flatten.

Robby Stein | Google 00:11:58

And then eventually go back up again. So this is called J curve retention or kind of flat retention. And what that shows is that there's some probability per day that the user wants to come back and use it. And then, if the product gets better over time, it's actually an intensifying probability.

Robby Stein | Google 00:12:15

And so some group of people don't use it again, or they do, but they use it maybe a week later, and then another group of people use it six days later, and a small group of people use it every single day. And what'll happen naturally is more people use it more frequently and less people bail. Then you actually end up seeing flat or slightly up towards the end percentage of people coming back. And that's really your your first real market fit moment.

Robby Stein | Google 00:12:35

And then I think the the next phase is you kind of graduate into growth. You know, you give it to a certain group of people, you put it in a system and just, are you actually seeing more users week over week and more usage week over week than the week before without really doing anything unnatural to get that? It's just people like it, they find it useful.

Robby Stein | Google 00:12:59

And we announced in the in a recent earnings that we're at about 75 million DAU on AI mode, you know, it just, just released really, pretty quickly and you have to kind of click in to use it. So there's obviously very promising early, early evidence that people are getting a lot value from these AI products. And you kind of build from there and now you have this base and then the product enters a more mature phase where, you know, you want it to be really useful to everyone who's using Google.

Carlos González de Villaumbrosia | Product School 00:13:15

I remember, one of the interviews with the head of Instagram back in the day, one of the leading indicators for him was the, the images or the posts that are being shared via DM as a much better measure of success than people clicking on likes or comments. Because the level of commitment for you to share something with a friend, obviously it's much higher that way.

Carlos González de Villaumbrosia | Product School 00:13:37

So in your case, I'm, I'm curious if there's any of, of these additional parallels where it's like a non-obvious metric for you to know that, okay, this is actually working and it's outperforming even the traditional search mode.

Robby Stein | Google 00:13:51

I think it still comes down to the amount of questions that we're seeing per day. And then maybe looking also at if people are following up, which we're seeing a much larger propensity to do, obviously, when you can have a conversation with ai. So it's like really that depth of engagement and the recurring engagement.

Robby Stein | Google 00:14:09

Because unlike Instagram, where it's fairly easy to scroll—like you just flick it a little and then you can get an impression—so like if you look at something like impressions, you're like, okay, it's kind of cool to get more impressions, but it's not a huge user commitment.

Robby Stein | Google 00:14:23

Like the user, someone needs to like go proactively—it's like sending a message to a friend—and like send a message to Google, basically to get information. It's a pretty high bar actually, relative to like a consumption product or a consumption app, media-based product. And so that is, I think, still a pretty useful metric.

Robby Stein | Google 00:14:39

And you're looking at a group of people and you basically say, okay, for a given person, are they using it more intensely over time? And I think that actually has been the most instructive versus you get like a lot of people to use it a little bit. And that usually I think, creates conditions, and we see, saw that in Instagram too, where it kind of started small but intense and then like more and more people use it over time.

Learning from Competition and Format Shifts

Carlos González de Villaumbrosia | Product School 00:14:54

Another interesting story I think connects with your background at Instagram is the launch of reels, right? Or, or you you were part of, of that—I mean, you played a key role in that. And as I think about that that feature or that product as well as now AI mode, both cases, I would say you weren't the first one to market right? Snapchat originally launch similar feature. In the AI mode world, there are some companies that didn't have a search functionality before, so their only option was AI mode. How are you thinking about that, those situations where you are not first to market, but somehow then you find ways to win?

Robby Stein | Google 00:15:33

I mean, I think it's all about solving a problem for your users. And I think what happens quickly in technology is there's kind of opportune moments where user expectations shift a lot in terms of what they expect. And they become new standards quickly.

Robby Stein | Google 00:15:51

For stories on Instagram or short form video—I mean these are, they're fairly basic primitives. It's like a short, it's like 30-second video or a ten second video. And there's been a lot of innovation for startups and other companies that have created awesome formats in that space.

Robby Stein | Google 00:16:06

But it's like a feed where Facebook had a feed and then LinkedIn has a feed and my DoorDash has a feed. And so like, it kind of becomes standards and so it becomes less, do you have a feed or not in the world? Now it's like, well, do you have AI or not? It's like, well, kind of, everything's gonna have some level of ai. It's gonna be, well, how does AI make your product amazing? And actually in many cases it doesn't. It might be slower or worse or more expensive. So I think people need to think about it and scrutinize for that.

Robby Stein | Google 00:16:30

But for us in terms of, you know, think about stories to Instagram, people are already posting more of what we thought were highlights at the time. These square kind of photos people would like and stories created this pressure release valve for people to post more casually 'cause they disappeared.

Robby Stein | Google 00:16:38

And you weren't, kind of, didn't feel the pressure of a like, and you could just post what you wanted. You could post as much as you wanted. You wouldn't take up a whole feed from your friend. It would just kind of stack in the feed. So it was an awesome, it turned out to be a great format that aligned naturally to the use case and job of what people wanted Instagram to be.

Robby Stein | Google 00:16:54

And then there's a lot of things that we did at the time to make it great for Instagram. It had Instagram creative tools. It kind had this slightly elevated kind of visual aesthetic. I mean, it was connected to DM and the messaging experience people were already using to send photos and other re, re-shares, to your point earlier, sharing posts with each other. I could write back.

Robby Stein | Google 00:17:11

And so it worked for Instagram and I think everyone would need to think about what that product meant for them. And the equivalent for, you know, how we think about AI search is we're not trying to create another general purpose chat bot that exists. They're great, there's plenty of them.

Robby Stein | Google 00:17:24

But we and, and you know, I think for, for search, people are coming for informational needs and they want very specific things from a Google. They wanna shop, they wanna understand location, they wanna look at restaurants, they wanna know plan a trip. They wanna, you know, they wanna understand what the world has to say about a certain topic.

Robby Stein | Google 00:17:41

They're very specific to Google. And there's ways that we think AI can make those really easy and frictionless for people. So it's like really that—so it's a lot more effortless to get information, especially when that information may not exist anywhere else, you know, on the web or elsewhere.

Robby Stein | Google 00:17:54

And so I think that, you know, that's kind of how you end up with products that I think even though AI mode is relatively new, it feels natural because people are kind of want these kinds of questions out of Google already.

Robby Stein | Google 00:18:01

And in case—and in fact we actually saw this where in a with AI overviews, which was the little AI previews at the top, people were asking harder and harder questions and they were actually adding the AI word at the end to get the—to trigger it. And so it's actually this latent demand in the same way that I felt like on Instagram.

Robby Stein | Google 00:18:18

I think people wish they could just post more on their feed, but they were always sitting there scru, scrutinizing, oh, I don't wanna like be weird and put nine things direct on Instagram in a row. People are gonna think I'm too hungry for likes or whatever.

Robby Stein | Google 00:18:36

And so it kind of just didn't engage more deeply. And then when stories came around like, oh, fine, I can just do it. I can just post it whatever I want. And same thing for Google. It's like, oh, finally I can just ask like this exact thing I was looking to do, put it into Google and have Google figure that out for me. And so it feels natural to Google's use cases, whereas, you know, I think other products will focus elsewhere.

The Ubiquity of Informational Knowledge

Carlos González de Villaumbrosia | Product School 00:18:56

Yeah, you're right. And I, and I think some of these, potentially new use cases be, become commoditized very quickly. I mean, we're seeing this with ai. Like I don't even think if this is an AI search or not. I just, I'm just now used to getting my answer with way less prompts, basically.

Carlos González de Villaumbrosia | Product School 00:19:06

And in some cases, I can allow myself to be lazy because I know that the system knows so much about me that they are out to out to complete my own prompt or at least follow up in, in, in smarter ways. So as you think about the future, and I'm not talking 10 years or anything like that, like short term future. What is next in terms of search?

Robby Stein | Google 00:19:31

I think there's a, a few things that I'm excited about. You know, one is, there's an opportunity for search, I think to be just so knowledgeable relative to, you know, what else is possible. So, for instance, search 'cause of the access and, and, and awareness of what is on the internet and can connect you to any piece of content so you can go deeper, learn, see a diversity of perspectives.

Robby Stein | Google 00:20:00

I think that's where really where knowledge lives. Is really like, you don't really wanna take an AI's word for it necessarily, but you wanna connect to people, you wanna connect to information. So you're trying to buy something: What are people saying about it? How do you understand that? What have the reviews been? What are creators saying about it?

Robby Stein | Google 00:20:17

And what's the price? And like, is that price accurate and real to updated as to the last, you know, 10 seconds and is there a deal and is that deal good relative to its history? These are like, this would take you like dozens of searches. No one would've done this to, to, to that level maybe except for the more, the most expensive purchases you would make.

Robby Stein | Google 00:20:33

But we could just do that. And the AI—under the hood—is actually doing that with Query Fan out and how it's reasoning for users every day. But I think you can take that to the next level. So it's really deeply integrating into all of this real time knowledge that exists and making sense of that for the user.

Robby Stein | Google 00:20:48

So it's like—so that's one big thing. The other one is how people interact with it. It's very much still, Hey, I am like on online. I wanna like ask Google this question. But increasingly we're seeing it, you know, I'm driving and I want to go live and just have a conversation with the same model.

Robby Stein | Google 00:21:00

So the modality is very quickly shifting. To being one that's flexible in all these different environments. So it's with you on your phone: so if you screenshot or if you're on Android, you circle to search. You have the AI right there with you as a companion, right?

Robby Stein | Google 00:21:02

If you're in Chrome, you can connect, um, to AI mode now and have this really nice experience where it can understand the page context and help you with that. If you're in the car, like I said, you can use the Google app now and you can use, search live and you can just ask, Hey, what's going on with this sports team? I heard there was some injury. Tell me about it.

Robby Stein | Google 00:21:19

Well, what's the implication for my fantasy team? Go research that for me. It'll like, go figure that out. And you can have a—you can play on your fantasy team on the drive home. And so the second big theme is that it's, it's really ubiquitous.

Robby Stein | Google 00:21:36

And the third is I think that it feels really like it's possible. It's made for you and can do things for you. And, you know, people have talked a lot about how it can become agentic and you know, we, we've announced that we're working a lot on both, um, these kinds of system tools and also these personalization experiences where.

Robby Stein | Google 00:21:51

You know, for you to have a system that can get stuff done, book restaurant reservations for you, tell you what's, but it knows your tastes well, um, you know, a search can be really useful for you specifically in a way that really just didn't exist many years ago. Where if you search for where to get, you know, eat, and for date night tonight, you get largely the same information.

Robby Stein | Google 00:22:08

Um, then someone searching. Obviously maybe it's geographically constrained, but let's say someone down the hall who's totally different from me. And now that's no longer true. It's really feels like it's for me and it deeply understands me. Those are a couple kind of big pieces that are I'm excited about in terms of where we search is heading.

Empowering the Agentic Experience

Carlos González de Villaumbrosia | Product School 00:22:25

I'm going to double down on the light latest one, the Agentic experience, because I've seen some early versions in shopping, in restaurant reservations, in, in flights. But for some reason it still feels like there's so much human in the loop. So curious to know from your perspective what needs to be true in order for this experience to be more self-sufficient?

Carlos González de Villaumbrosia | Product School 00:22:50

So, of course, the human can still be in the loop for critical moments, but like we can empower the machine a little more so it can make our life a little easier.

Robby Stein | Google 00:23:00

I think there's a few things with this. I mean, it's early, first of all—I think, um, computer-computer interaction, agent connection is still an early protocol. I think just in terms of its ability to seamlessly tap into information. It's just, it's a, it's a young platform overall, so one part of it is just making the thing work. 'Cause it, it does take time.

Robby Stein | Google 00:23:24

I think that—probably takes a fair—probably too much time in some cases, and you also want to work every time. But let's assume that we live in the future—slightly in the future—where these things are easier, more mature, and faster. I actually think a lot of the problem in this space is more in understanding you deeply and in helping you understand the potential considerations with actually you in the loop in the right way versus fully taking you out of the loop.

Robby Stein | Google 00:23:43

Like if you think about buying something, so much of the process is actually in the consideration phase of that. And while it's kind of annoying to put your credit card in at the end, it's actually not—I think the largest problem. I think a lot of people, if you wanna wanna buy a tv, I don't think that the largest problem of buying a TV is paying for it.

Robby Stein | Google 00:24:01

I think it's actually like, well what's the TV and how big is my room? And if I take a picture of my space, is the TV gonna fit there? And how, how do I hang it up? And by the way, if I wanna mount it, who could mount that for me? Those are the things that if I, if I just, if I just was like, I wanna buy a tv.

Robby Stein | Google 00:24:17

And maybe the thing was like, take a picture of your living room. Take a picture of your living room. And it's like, okay, well, and it understands other purchases I've made. It understands its history. And it basically says like, Hey, here's the TLDR for each of these TVs. Here's where you can read about it.

Robby Stein | Google 00:24:30

Here's reviews. You could research. And by the way, these are the live prices. It's Cyber Monday or recently. Based on the price history that I analyzed, this is like a legitly good deal. And like this person who's a local business could install it for you. Do you want me to go do that?

Robby Stein | Google 00:24:51

And it's possible that the answer is like, no, I actually want a smaller tv. Or you'd want to modify it, right? And then you'd want to do it. So I just viewed it as like—so it's like super leverage. It's like you had this thing that knew you so well and could do things for you that you just need the littlest bit of intervention.

Robby Stein | Google 00:25:00

Like, um—I want this tv—that's all. Like, maybe it predicts you need a TV in the future too, I don't know. But I don't see that anytime soon. And then you just judge it a little bit and it's done. And that would've taken you like hours of research, um, kind in the prior, prior world.

Robby Stein | Google 00:25:09

So that's—that's just personally how I see it, that I think people are a bit enamored with the fact that computers can do a lot of things, um, on your behalf. But I, I'm kind of think a little bit more about like what's really useful and if you really think about your time, how it can be harmful for those things that are the most burdensome.

Impact on the Business Model

Carlos González de Villaumbrosia | Product School 00:25:36

How does that new functionality affect your own business model? Meaning now you are able to allow the user to go further into their decision making process, right? So it's not just you presenting information and let them choose.

Robby Stein | Google 00:25:50

I mean, I think it it's, um, a natural extension of everything Google's already done. I mean, if you think about what you know people do today, first of all, it's expansionary. Like I couldn't take a picture and ask about my, my room. But today it's, you could easily do that and it could probably measure the space for me.

Robby Stein | Google 00:26:07

And so you get more, um, interactions, more usage and more value to the user because you can do more for them. The second is theoretically, I'm giving, you know, a lot more information for Google to be helpful. And so to understand the spec or to understand what I'm trying to buy with the, I may have before just Googled like flat screen TVs.

Robby Stein | Google 00:26:29

You know, like—that's kind of like a typical query. But in a world where you, Google has all of this incredible context in terms of exactly what you need, the size, the space, the parameters taste, you know, different considerations you may have said in the past, um, it is a very nice and natural way to help you buy that thing.

Robby Stein | Google 00:26:51

In that, in this example, given many journeys are, you know, commercial nature, but these are natural extensions. Just like how now if you were to ask for TVs, you might see an ad that um, has a great deal for Black Friday or for Cyber Monday, um, in the AI world is exactly the same. You could say, I also got you this TV deal. You know, like—I mean, you could do anything.

Maintaining Speed while Protecting the Core

Carlos González de Villaumbrosia | Product School 00:27:06

So as you think about allocating your own efforts, your team's efforts into all these different initiatives. Some of them feel new and I was like, what? Some people would call zero to one. But then you also have a core business to protect and augment, right?

Carlos González de Villaumbrosia | Product School 00:27:12

So—and you, you've been a founder as well, and you know that when you're a startup, your competitive advantage is pretty much speed. When you are much larger, your competitive advantage of one of them should be distribution and maybe data. So how are you able to still maintain crazy speed to push for the future while protecting your core business?

Robby Stein | Google 00:27:38

I mean, I think one answer is that the models have gotten so good that it's a lot easier to add new functionality. And so I think you're seeing this ramp up in the amount of features impossible 'cause something that would need like special, um, custom model tuning or post training, you can just give the model some information and some context and it'll just do it.

Robby Stein | Google 00:28:00

Uh, so I think it's also—it's, it's one, one part of the story is it's much easier to build and to try new things because it's less work to build an AI than ever before. Um, the second is, I think like as a founder, as a leader, everyone needs to think, you know, what's gonna be the most important thing that if we get it right, this is potentially a five or 10 year opportunity of growth, of value, of helpfulness that we can give people.

Robby Stein | Google 00:28:20

Whereas if we kind of had some other area that didn't go exactly as well as we wanted it to, but it was okay, probably wouldn't make too much of a difference. So that's like one thing. And the second thing is, what can I uniquely help with that the organization or the team wouldn't potentially pursue themself either because it's hard, it's risky, it traverses the organization in a way that's unnatural?

Robby Stein | Google 00:28:43

And those are the best opportunities where those two intersect, where there's a lot of opportunity for future value and then also where it wouldn't naturally occur because—like this maybe cuts across the org in weird ways, or the team, you know, it's a pivot. And if you're a startup and it's not how the team has thought about, they're wed to their kind of first idea.

Robby Stein | Google 00:29:02

And that's where I think leadership intervention is the most useful. So that's how I at least think of focus. Um, and you know, and then eventually not everything is in, its in its—needs a level of attention and, and love. Like, once the thing gets to be at some level of maturity is a lot easier to maintain it versus getting things going is usually really hard.

Robby Stein | Google 00:29:19

So I think for leaders and founders it's usually about finding those engines that are working and, um, creating value for people and getting those up and running. Um, is the hardest work.

Selecting Leadership Bets

Carlos González de Villaumbrosia | Product School 00:29:36

And for you—I mean, as a, as a former founder, I'm a founder as well, right? So I know how important it is to be in the details I believe in, in that in many ways. And showing face, being in the kitchen, cooking with the team—it applies a level of care, not just pressure.

Carlos González de Villaumbrosia | Product School 00:29:54

That it, it's contagious. So when you cannot be in all the kitchens—you have way too many, right? So—how do you pick those bets to ensure that there's still that level of attention by the leader while obviously you also let other people, other readers, all place their own bets?

Robby Stein | Google 00:30:10

Yeah, I mean, I think it comes back to the framework I was mentioning before, and when you pick the area you wanna focus, I kind of think of them as projects. Like I'm really gonna, um, nurture and like we'll meet with the team. I'll meet with the team multiple times a week, potentially daily if it needs to happen until that thing feels like it's on track and try to do it in a way that feels like co-creation.

Robby Stein | Google 00:30:30

I don't think it's about, um, you know, making it challenging for leaders on your team to, you know, have influence, but it feels much more like we're building this thing together. And these, these working groups, these virtual teams—if you're sprinting on something like an AI mode, it feels a lot less, like there's levels of hierarchy and there's like an approval, and then another approval, and another approval and another approval.

Robby Stein | Google 00:30:49

So get something like that out—it's kind of like you have this big coworking team and you have some people steering key aspects of it, and you all come together and you explain how it's going and um, and like, yes, you need a leader to kind of cut through, but I think it ends up being this more collaborative vibe. And that ends up helping a lot.

Robby Stein | Google 00:31:06

So like, everyone's doing a really important thing and within that working group, people are like—so whoever's working on, let's say the voice and audio component right, of that project, like they're gonna make that incredible and be the most detailed in that particular piece. Um, and then when you bring it together, I think you have enough level of specific—I do think leaders need to be deeply in the details.

Robby Stein | Google 00:31:30

And so that's why you have to pick projects carefully. And when you pick a project, you as a leader, you deeply understand it, but you're never gonna know every single detail down to the level of every engineer, obviously. And so there's always that room—you want kind of, um, it to feel like this, this collaboration where the person closest to the work knows the most about it.

Product Culture and Cultural Environments

Carlos González de Villaumbrosia | Product School 00:31:47

Product culture is one of the topics that also very interesting to me. 'Cause, um, every company in a way has their own ways of working. And for you, you've seen I Instagram, you've, you've created your own cultures, you've been part of now Google twice. How did your previous experiences in different cultural environments are now affecting how you build at Google?

Robby Stein | Google 00:32:11

You know what's interesting? Like, I think, um, a lot of what's shaped my thinking is, is really thinking as a founder and thinking as, you know, it's, it's kind of obvious to say, but I think when you kind of ask like, what does it actually mean as a founder, what, like what is that kind of means that like—no one else is gonna solve the problem and you need to be the force of like, what is gonna be the most useful thing for you to go do any moment.

Robby Stein | Google 00:32:35

And it kind of works recursively. So like the leader thinks about it from their altitude level and perspective, but then every individual who's building something thinks about it from their world. Like even if you're working just on this word of using this example of audio, you know, they're thinking, "Okay, well what are the decisions I'm making that, what are the moves I have to make to make the best audio experience?"

Robby Stein | Google 00:32:54

And they're—owning that outcome fully instead of being like, yeah, well, it was hard and you know, oh, I had to work with this other team and that was hard. Everything's hard—it's like this kind of just force of nature of, I guess, this combination of having an outcome and being clear about what you think it should be.

Robby Stein | Google 00:33:10

So you kind of have that point of view of like what your product should be, where the growth is coming, where the value is being created, and being really focused on that. And then you're—forced to just like driving, moving it forward, getting it done. Like not, there is no excuse ultimately; you own the outcome.

Robby Stein | Google 00:33:28

Like a high internal locus of controls—like one way of framing it that people have spoken about—like you're—control of your destiny. Like you just need to figure it out. I think those pieces feel like they most correspond to success from teams I've worked on, both at startups and big companies, they kind of transcend all things, right?

Robby Stein | Google 00:33:46

There's people who are kind of like, gimme the ball and I know where, what we need to do. And and like whether you're founder or you're at a big company, I've seen the, that that kind of attitude just kind of works in all of these different contexts. And you have a view of like what's good and what's bad, and that view is in sync with the world.

Robby Stein | Google 00:34:05

It's like you're getting in touch with the feedback loop of your users, of your people, of the team, and you're like just looping on making it better and better each week, um, until that outcome is achieved and like you're that little force making that happen. Um, and if everyone does that, you're gonna build great stuff.

Lessons from Colossal Disasters

Carlos González de Villaumbrosia | Product School 00:34:23

You know, and when some people hear leaders like you talk like, oh my God, they come from Google, Instagram, of course they're going to win, right? Because they have these large distribution platforms, incredible access to talent and capital. And so if something doesn't work, they'll find something else and eventually that that will work.

Carlos González de Villaumbrosia | Product School 00:34:43

So one of the things that I always try to, to learn from here is colo disasters. Right, like situations where you didn't win and, um, kind of what you actually learn from those experiences to then eventually find, find, find ways to win. So, curious to know, as you go through this AI journey at Google or even before at Instagram, what are some of those close disasters that help you improve?

Robby Stein | Google 00:35:12

What's interesting—I've worked on planet projects that have straight up failed. Um, and those are somewhat interesting, but I actually think less interesting than the projects that failed and we could have just stopped, but then we kept going and they became successful. Because that's actually, even though I wish every project I work on had that signature, it's impossible.

Robby Stein | Google 00:35:31

I mean—I saw some, I wanna say, um, I saw somewhere about, um, someone saying something like, you need to do three startups to have, have the probability of one working or four startups because it's just so hard. Like you, you have to just keep going at some level and run outta time, and then you kind of do it kind of again with a slightly similar team.

Robby Stein | Google 00:35:48

And so—I I actually think what's interesting is like everyone has this portfolio of shots on goal and like, you're always gonna have a couple just swings and misses. But if your attitude is, I'm gonna try to keep going until I get it right and you give it enough time, some of them actually win.

Robby Stein | Google 00:36:00

And then it's funny 'cause you need just a couple of those big winners and everyone just talks about those big winners. That's all that really matters. Um, it's kinda the slugging percentage versus batting average. And I think that we've talked about that at Instagram with your Instagram leadership at the time.

Robby Stein | Google 00:36:10

And like that was a really crucial concept. And so an example is like—actually close friends on stories and reels for Instagram. Both were colossal failures when they started, which is really interesting. Like—on, on, on close friends, we made this way for you to share with just a small group because people felt this pressure that they were being judged by lots of people.

Robby Stein | Google 00:36:42

It's like, okay, let's share to a smaller group. So we allowed you to share on feed and in stories and, and it had this part of your profile. It was also private and they were marked—these were private items and the whole system was just so confusing.

Robby Stein | Google 00:36:54

And not only that, it was even mistranslated in a bunch of countries. So instead of saying close friends, I think it was called favorites originally. And in some countries it was called like my favorite. People would just put like one person on the list and share effectively a dm.

Robby Stein | Google 00:37:06

But because he didn't send you a push alert, 'cause not a dm, it's a story. They would never even see it. And so they wouldn't know, and so there was no feedback loop; no one got a DM back from their story and it just like died. And the thing was a disaster, but we could've just walked away from it and been like, that sucked.

Robby Stein | Google 00:37:19

But two things happened. One, um, we asked users why they felt uncomfortable sharing. And the number one thing was almost always, I'm worried who's gonna see this thing and I'm gonna get judged for it. Or I have a teacher on here, or my aunt or my uncle, and it's like me goofing off with my friends or whatever.

Robby Stein | Google 00:37:44

And so we just had such conviction that this was a serious problem that was preventing people from getting more joy out of and, and, and exp and value from the product. And the second was, um, you, you know, we didn't understand. Just like how the physics would work and we actually looked at the data and it turned out that the stories part of it was kind of growing a little bit, but only with a small group of people.

Robby Stein | Google 00:38:07

So like what's going on with that? We studied and it turns out if people added enough people to their list, like 20 or 30, and they posted their story, it felt like this little private story that felt good. And then we also had this green ring on it. So it felt like you could see that you had a different like kind of story on there.

Robby Stein | Google 00:38:22

And then if you posted to 20 people, you have some probability one of them is gonna DM you back and go, sweet, that's awesome. And if it feels like a private thing. So it kind of, it worked and so we deleted everything else we did and we just made it a stories thing. And we changed the name from Freight Favorites to close friends.

Robby Stein | Google 00:38:38

We added that green thing and we, we made it clear people add. People to your list. And we made it easy to add like 20, 30 people and it was like, "You're not done until that's kind of at a sufficient number ." And then it worked.

Robby Stein | Google 00:38:50

Um, and there was an equivalent thing with, with um, reels where we launched, I think it was in Brazil, and it was integrated to stories. The thought was, "Hey, it's a full screen viewer. It's video. It's something we think like younger people are excited about and they love stories."

Robby Stein | Google 00:39:02

It seems just like a stories feature. Why are we building a whole separate thing? It's like there's already, Instagram's already so complicated. Let's just make something simple. Turns out that was also a bad idea 'cause why would you put all of this effort. Into, you know, building, like posting, um, a story when, um, it goes away in a day, right?

Robby Stein | Google 00:39:20

Like, you're a creator. You want it to live, you want it to go viral, you want lots of people to see it. And so—that led to a set of whole set of changes that were pretty meaningful. Um, and in both cases, I think it was this belief that the thing you're working on is pretty, it needs to matter and be important.

Robby Stein | Google 00:39:41

That gives you the desire to keep pushing. And then you, you kind of like—it's actually just like how AI works and machine learning. It's like you initialize the first thing you see its losses, you see how it's sucking and then you go and it's actually in those, the next two to three moves, you make the iteration cycles that actually produce whether the thing's gonna work or not.

Robby Stein | Google 00:39:58

And then at some point you give up of course. Um, but it took years for some of those projects to go, um, and maybe four or five attempts. Which is, I think, shocking to a lot of people when they, they just see it and they're like, oh, you launched this thing and it was successful, or whatever, like you're Instagram or your Google. But, you know, many, many of the stories kind of started that way.

Commitment and Over-Rotating on Failures

Carlos González de Villaumbrosia | Product School 00:40:18

I mean, one of my takeaways just from hearing you, um, share this is that it's—takes crazy conviction sometimes before the numbers, right? Because especially when you're operating on a scale where you have other things that are working for you, why would you over rotate on something that is not working?

Carlos González de Villaumbrosia | Product School 00:40:34

And um, and I love that because that's one of these things that I, I do as a, as a founder—I mean, it's not that we also don't have a plan B, so it has to work, right? But like in, in a place where you have a, a more diversified risk approach, it's still incredible to see that you can. Focus on things that are not working and pushing however it takes until at least you get to some level of success.

Carlos González de Villaumbrosia | Product School 00:40:58

I mean, we've seen the extreme case maybe with meta, with uh, AR and all of these things, but, but in reality, something good will eventually happen one way or the other.

Robby Stein | Google 00:41:08

Totally agree. Like if you think about what our journey with AI mode was like—starting with AI overviews, building more sophisticated models and letting you ask these more harder questions. And the first versions, you know, they're not, nothing starts great.

Robby Stein | Google 00:41:16

And so you're kind of working on it and you're like, "Oh, I don't know, like it's kind of confusing or it's getting confused with these kinds of questions, or it's not thinking or reacting to what I already said it told it to do ." And you could just give up right there.

Robby Stein | Google 00:41:31

But I think what happens is usually—you have this intense conviction, like for, for me and for us, it's like you should be able to ask Google these questions that are hard and that are about travel or learning or homework, and we should use all of this incredible context and knowledge at Google and get it right and help you.

Robby Stein | Google 00:41:47

And so it was this very high conviction. And then usually you have this like spark of working. So like I ask, I would ask it like one or two questions that were really hard and it just nailed it and you're like—Oh my God. It's kinda like when you, you hit a golf ball.

Robby Stein | Google 00:42:00

If anyone has played golf and you just hit a perfect golf shot, it all comes together and then you never do that—you don't do that again for a while, but you're kind of like, "I know it's what, what is is possible and it feels so good when the whole system works and then you're just, that motivates you to keep going basically."

Robby Stein | Google 00:42:15

Those are the two reliable things that I've seen every time. Anything like that I've been a part of working on. That's, that's—gotten to that level of escape velocity had those two starting points. 'Cause again, everything starts not great and it's up to the leader to make it great. Or the founders or the initial, the founding team, um, that you know, the real people doing the work, um, to make it possible.

Carlos González de Villaumbrosia | Product School 00:42:40

I love that golf analogy. I got it—I call it the golf course because indefinitely one of those people, everyone. True white—

Robby Stein | Google 00:42:47

Also, that's also true.

Distribution Advantages and AI Adoption

Carlos González de Villaumbrosia | Product School 00:42:49

Um, I want to switch, uh, shift gears for a second and, and talk about distribution. I mean, what an incredible comeback story like Google—like when we saw the rise of new AI tools, uh, there were articles out there about Google is dead, it's going to die, and suddenly it took a few iterations.

Carlos González de Villaumbrosia | Product School 00:43:07

I mean, especially now with Gemini three, boom, it starts clicking and like—Use as a user, I start getting real value. And like—I see also the mechanisms that are encouraging more users, such as now you're bonding that ing that as part of Google Workspace, right? So I don't have to pay more to use that.

Carlos González de Villaumbrosia | Product School 00:43:24

Um, you are better, better integrating with other, other Google products. So—I've seen that movie maybe in the Microsoft world when, uh, internet Explorer was already installed in some, uh, computers and, and even Microsoft teams started taking more te, uh, headwinds, uh, head tailwinds after, um, slack had to be installed for new users.

Carlos González de Villaumbrosia | Product School 00:43:47

So I, I'm curious to know about how you're leveraging your own distribution advantage to get more adoption of your own AI products.

Robby Stein | Google 00:43:55

I mean, the good news is people are coming to Google and they're asking these kinds of questions, so I think it's more actually about how do you meet people where they are. Solve their problem really well, honestly, because, you know, I've been part of many things where if you put a product in front of someone, they won't use it.

Robby Stein | Google 00:44:09

Like, if it doesn't add value to what they're doing, it's just like dead. Um, you know, there's been many experiments where we've tried making really radical changes to products before. Um, and—like, there's just no reason why anyone would care.

Robby Stein | Google 00:44:23

And so I think a lot of it is, you know, you know, we have the signature where people are looking for information with increasing complexity, and then the more that you find that moment where you've invented the format that works for you, like we talked about Instagram and stories and. AI overviews, AI mode for search.

Robby Stein | Google 00:44:37

It's like the informational AI that people want because they want this context and ability to ask natural language and ask follow-up questions. Um, you make that possible and easy for people to use. Good things happen. So I think that like—One piece is having that, um, system work really well.

Robby Stein | Google 00:44:58

And I think we're starting to see that now with AI overviews and AI mode, especially how they're, you know, coming together in a more fluid way in the kind of ways that we're thinking about these experiments. And I think the other piece of it is this, this relentless continued improvement in the models themselves.

Robby Stein | Google 00:45:12

I mean, you said yourself, like, you know, think about where models were three years ago and then two years ago and then one year ago. And then now you're in this world where, um, the model can code and can produce little simulations that explain things for you and create, you know, mortgage calculators for you on the fly.

Robby Stein | Google 00:45:29

And so you have—someone coming, trying to figure out their mortgage. You have a model that can do all this incredible stuff. How do you bring those together and match that. Um—User need. And that's really, that's really, I think, the, the most critical opportunity that we've been, you know, really pursuing.

Closing

Carlos González de Villaumbrosia | Product School 00:45:46

Rob, it's been a pleasure to geek out with you on all things AI and how you are building from the trenches. Thank you so much for your time.

Robby Stein | Google 00:45:53

Absolutely. Thanks so much for having me.