We Built What?
The podcast for engineering leaders building in the agentic era. Brought to you by Augment Code.
We Built What?
What separates the teams winning in the AI era? With Wei-Wei Wu, CEO at Momentic
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Everyone is shipping faster. But speed alone isn't what separates the teams winning right now. Wei-Wei Wu, CEO at Momentic, has a clear take on what does: a tighter feedback loop and the discipline to verify what you're building against what you actually intended to build.
In this episode, Wei Wei and host Emma Webb get into why experimentation is the only way to find out what's possible, why the spec needs to be your source of truth and not just a design artifact, and why more code output is not the same as more value delivered.
They also cover what 2 billion test steps have taught him about how teams are actually shipping software, why ambiguity is your biggest enemy when working with AI agents, and why the QA engineer skill set turns out to be exactly what the agentic era needs.
Wei-Wei's take: the price you pay for not experimenting is you don't actually know what's possible. But you still need to know what you were trying to build in the first place.
If you lead an engineering team thinking about quality, velocity, and what it means to ship with confidence, this one's for you. Have someone you want us to interview or a topic you want us to cover? Let us know on X or LinkedIn.
In the age of AI, everyone wants to tighten the feedback. Like being able to adapt, be flexible, and be willing to change my opinion, instead of waiting weeks to get user feedback, we can just ship it and see what happens. You know, the price you pay for not experimenting is like you don't actually know what's possible.
SPEAKER_01You're listening to We Built What? The podcast for engineering leaders building in the agencera. I'm Emma Webb from Augment, and I'm so pleased to be here today with Weiwei Wu, CEO of Momentic. Thank you so much for joining us.
SPEAKER_00Thanks for having me.
SPEAKER_01So, Weiwei, you just recently closed your series A. Congratulations. Can you tell me a little bit about the thesis behind Momentic? What are you building?
SPEAKER_00Yeah, yeah. Momentic is the AI verification layer for software. We are the guardrails to make sure your software actually works the way you intend it to, you know, for your end user. So, you know, powering some of our customers like Notion, like Built, Quora, Webflow, Xero and more.
SPEAKER_01So this is such an important problem right now, and I'm really curious why do we need a standalone company to do it?
SPEAKER_00Yeah, I think one of the things that's really interesting is as we see teams adopt more and more AI tools, is they're shifting towards plain English specs for how they talk to their AI coding agent for coding. We want to make that spec also the way you would verify it, you know, along with all those assess criteria, all the edge cases, you know, just the way you would describe how you would build it, that's exactly how you would verify the AI agents' work as well.
SPEAKER_01So the spec is in the verification step as well, it's not just the design stage.
SPEAKER_00Exactly. So like the spec is kind of how we think of it as the source of truth for how your product or feature is supposed to behave.
SPEAKER_01So if I'm thinking about kind of how we build software in the agentic era, one of the things I think about or that people talk about is you really don't want the same agent that wrote the code to also be verifying it. Is that part of why Momentic as well?
SPEAKER_00Yeah, yeah. I think I am tasking all of my little, you know, AI robots to write code for me. Um, you know, we have them all hooked up in Slack, for example, internally at Momentic. But at the same time, I also want proof that it did its work correctly. And, you know, this goes beyond just static code review, linters, you know, unit tests and things like that. I actually want to see it work from our user's perspective. And I think it's important that it's a separate tool that's actually verifying the work, but it also works well with the AI that's building the feature in the first place. You know, I think it's a you know an ecosystem that kind of grows together.
SPEAKER_01Yeah, it's so interesting. I think there's this really big conversation right now about what it means to be AI native. What does that look like for you at Momentic? How are you how is your team building? Can you tell me a little bit about how do we build now?
SPEAKER_00Yeah, yeah, that's a that's a good question. You know, when I when I first started coding uh in in high school, you know, we were using you know Notepad Plus Plus, or like I think in college I was using like Sublime Text. That was like before, you know, Cursor or any other AI coding tool existed. I would not consider myself AI native. AI native is like someone who grew up with AI, you know. I think the way we approach using and adopting AI is also quite different as well. But I think for me, like I would consider myself very flexible. It's like, you know, especially as the the founder of an AI native company, you know, being able to adapt, be flexible, and kind of be willing to change my opinion about like what's possible.
SPEAKER_01It's really interesting to me because basically the way you describe it is there's kind of a before and after. And in my mind, if we're thinking AI native is someone who's always built that way, gosh, what a small cohort of people that is, right?
SPEAKER_00I I agree. I agree. I think having experience in a non-AI native uh is is important for context setting. You know, instead of me prompting in my editor, um, you know, I used to have to type things out by hand. You know, that's insane. You know, it was just you know actual code autocomplete, not just you know, tab completes and things like that.
SPEAKER_01In my day. Yeah, but I'm like picturing you around the fire in the future, you know, telling your grandkids in my day, we coded by hand.
SPEAKER_00You know, like in the future, I imagine you know, I have an electrode attached to my temple. I'm just thinking the you know software into existence.
SPEAKER_01Yeah, it's become a bit of a problem because I build so many apps now that nobody like I don't have nobody wants them. You know what I mean? And I'm like, look at this cool thing I built. And they're like, you can you can send me one per week. Like that that's your maximum app distribution. So I think about that too. Like, what is our appetite for this, for kind of this creation? Where's the limit of that? I don't know.
SPEAKER_00We should be testing the latest and greatest and see what are the things that are actually bringing us, you know, 10x, 100x productivity gains. And I think part of that means that we have to experiment a lot, maybe build a lot of different apps, build a lot of different integrations and see what works because you know the price you pay for not experimenting is like you don't actually know what's possible. And I think it's important to build things, test things out, even say, you know, if 99% of them fail, like 1% is going to be, you know, a huge lever for your team.
SPEAKER_01Yeah, and I think the part of that discipline then becomes too like, how do we how do we find so much more comfort than we're used to in killing those apps or killing that code or saying, we try this thing, it's kind of dumb, it didn't work out, or good idea, let's remix it and put it over here. And I I think about um even that cleanup stage, I think that's something that we haven't fully figured out yet because we've never had this proliferation of application or proliferation of code that is so disposable.
SPEAKER_00Instead of saying, like, say, waiting weeks to get user feedback, you know, we publish Momentic multiple times a day. We can just ship it and see what happens. There's a lot of value in just getting it out there, experimenting and you know, see what works or not, because it's very easy to fix or update.
SPEAKER_01Yeah, it totally makes sense. At Momentic, I know you're you're thinking a lot about this validation step. Um, and you're a team that's really kind of at the forefront of thinking about that idea. Your customers have run two billion test steps. Is there anything interesting that's happened in this data that you're seeing in their behavior? Obviously, you know, they're coming to because they're in this AI moment looking for an AI solution. What are you observing from what you're seeing customers do?
SPEAKER_00Yeah, I think what we're seeing is that the rate of change, you know, the rate of code output obviously has increased a lot. But I think depending on how teams adopt certain tools, um, like more output is not necessarily better, you know, because I think it's there's like uh it's important that more output, say like more code output directly translates to say like, you know, new features, better use user experience. Everyone wants to tighten the feedback loop. How can you make it faster? And I think the form of that is through tests, say like a critical flow like authentication or or billing or anything like that. And I think the people who are shipping the most value are the ones who are optimizing for that feedback loop.
SPEAKER_01How do you think about the quality of the signal? You know, just in terms of if I think about kind of in a pre-AI world, if I think about A-B testing, for example, you know, part of the discipline of an A-B test is, okay, well, how long do I have to run it to get to any kind of significant signal? How do you think about that challenge now? Are people thinking about kind of like the volume of data that you need to get to significance? How do we decide?
SPEAKER_00Yeah, that's a that's a good question. I think, you know, I'm just thinking back when we first started, when Jeff and I first started Momentic, we were in YC, we had maybe like two paying customers. You know, like what's this, what does statistically significant mean? You know, we're gonna survey all of them, like, you know, like two, two two out of two. Two out of two is you know, not not a lot of you know high signal data. So I think how we approach it today is like, for example, in Slack, you know, most of our customers have shared Slack channels with us. You know, we collaborate with them, we meet very often with them, you know, to get feedback, show them the latest and greatest. And I think feedback that we collect is both like quantitative and qualitative. And like, you know, for example, like we have session recordings. We can see how they're actually interacting with certain things. Like that doesn't necessarily require us to, you know, have a survey or like, you know, hey, what are the frictions of using Momentic today? How we shape product at Momentic is vibe-based, right? It's like it's based off conversations we have with prospects, with customers, you know, they're giving us feedback, you know, we're seeing how they use Momentic in the wild. Do we have to survey, you know, 300 users to get 95% with plus or minus 5% error bar? Like, probably not, but I think you know, based on all these conversations we've had with our customers, you know, I'm pretty confident that this change will make their life better.
SPEAKER_01So interesting, too, because we talk so much about the attributes of taste, of judgment, of kind of having conviction of what you're building, of intent. And so I think in many ways the ability to decide based on vibes, based on taste is becomes even more important. Um, because if we're experimenting at this rate, if we're shipping at this rate, the data in many cases will never catch up to tell us. Um, or it'll be too noisy. Right. It'll be very hard to distinguish signals. So yeah, I don't know. I I think about um just that taste quality. And the other thing I think about is how do you teach taste and how do you uh develop that? But part of it is just shipping all the time, I think, and getting a feel for is this good or not?
SPEAKER_00We're essentially building the tools that we wish we had when we were engineers. Well, some of them are still engineers. Um, and I think there's like incredible value in that because you know you may not necessarily be able to quantify it, but you kind of have an intuition for oh, what is the workflow I would expect as an engineer for this specific developer tool?
SPEAKER_01And momentec, you spent a lot of time, you know, thinking about this validation step, thinking about this testing step. How is the role of that QA engineer changing? What are they doing now? Like how is their how does their day-to-day shift? Yeah. Tell me more about that.
SPEAKER_00I think the lines between roles are blurring more and more. And I think the barrier for entry into what we used to consider software is also blurring as well. So I think the titles I think will need to change in the very near future because you know, what is an engineer? I think technically anyone could be an engineer today, you know, if the bar is only getting higher and higher.
SPEAKER_01But in a sense, I imagine those folks sitting uh, you know, as a QA engineer, uh evaluating, judging output as kind of your professional career may be very well placed. Like they have may have this very well-honed sense of judgment and taste because of that role of, you know, my role has been historically to look for bugs, to look for issues, to find problems. That feels like a kind of a useful skill set at this moment.
SPEAKER_00Def definitely. And I think that it's even more powerful when you know, now you're like, say I used to be a QA. Now I'm actually building full-fledged features. Like I have a much higher bar than say someone who has ever no experience, you know, testing or verifying product. And I think that skill set is incredibly valuable. It's like it's it's almost like kind of like thinking from first principles, like, you know, it's cool, we built this really cool software, but you know, how are all the people gonna misuse it now? You know, it's a text box. What are people gonna type into it?
SPEAKER_01Oh god.
SPEAKER_00Um, and I think that type of thinking, I think is incredibly valuable.
SPEAKER_01It's so interesting too, because I think in some, especially bigger organizations, maybe that person really wanted to go. They're like, I see it, I can fix it, I have the skills to fix it, but for whatever reason, not allowed to.
SPEAKER_00Yeah, I think we want to make that loop as tight as possible with as mid little middlemen as possible. And you know, I think that's how we can iterate really quickly and just move really fast.
SPEAKER_01So I'm curious, can you tell me more about how your engineering team is structured? I think this is it, it's sounds like such a kind of basic tactical question, but people are very interested in it at this moment because you know, if I can, if I can output at such a higher rate, if I have this feedback loop that's so much tighter, if I can be so much closer to both the customers and the code. Okay, well, what does my team need to look like? How have you thought about that? How are y'all starting to do that?
SPEAKER_00Yeah, that's a that's a good question. And I think you know, today we're you know, Momentic is still pretty small. We're you know, 13 people, about to be 14, we're F7 engineers. The amount of parallelization we we can unlock with AI is massive. So like now instead of executing on one or two, we can execute, you know, like four or five X more actual projects that are massive. So for example, um, we just launched Android support back in uh October. Like iOS is coming in like a week. If you ask me, like, you know, two, three years ago, like how if we wanted to do this, it was like we're gonna it's gonna take like six months to a year. Now it took like you know, you know, a few weeks, maybe a month, a month and a half. It's like it's incredible.
SPEAKER_01What is the most overhyped thing in AI testing right now?
SPEAKER_00I think one of the most overhyped AI testing is really uh focusing on like I'm gonna crawl your whole app, we're gonna figure out every single thing your users are doing on your app, and we're gonna generate tests for you. And I think that approach is, you know, it sounds great on marketing, it's like, you know, oh, I don't have to do anything. The AI is gonna just figure out everything my user is doing, what's broken. But I think the the problem is I think there is like kind of like a chicken and egg thing, you know, like what you have live in production is you know a function of like what you what code you have deployed. And you know, just the nature of software, you know, there's going to be bugs, you know, and I anticipate there are going to be bugs for you know quite a bit, even as you know, AI continues to improve. Um, you know, you know, that's why we have all these you know code review bots, there's like 20 of them, you know.
SPEAKER_01We have one, yeah.
SPEAKER_00And and for sure. So like using production as your source of truth is just fundamentally flawed because production will have bugs. Saying that we're just gonna use AI to crawl your app and figure out what's wrong is just like, well, how do you actually know what's wrong? You know, because you know, what's actually live in production might not actually be what you want.
SPEAKER_01This is so interesting. I've been thinking about this too, just in terms of your kind of what exists may not be your ideal. And so kind of your opportunity space is the delta between your ideal and what's out there today. But I guess this is why you guys go back to the spec. You're like, the spec is what we're trying to get to, just because the code is this, that doesn't mean you know, that's what you were trying to achieve.
SPEAKER_00Yeah. I think if anything, being specific in what you want and what you don't want, that's incredibly important because whatever you tell AI, anything that's ambiguous, it's definitely gonna go there. So I think, you know, in that sense, like we're thinking about like specs as the source of truth. Like these specs are kind of your north star, what you want your software to look like, how you want your product to behave. I think it's important to make that distinction.
SPEAKER_01I think we're so used to just imagining that the thing that has been built is the thing that we meant to be building the whole time. And in fact, like we don't we may not know if we're we're trying to go.
SPEAKER_00It was like, you know, when a test fails, how do you know that it's the test's fault? Or maybe the feature change was intentional, or maybe it was a breakage, and they're like, you know, we don't know. Support docs are out of date, PRDs got out of date, the Jiro ticket got implemented, you know, two months ago. There's been like a few thousand commits since then. Figuring out what it's actually supposed to do, how it's supposed to work, I think is is going to become more and more important, especially as we delegate more stuff to AI.
SPEAKER_01It sounds really basic, but it's not.
SPEAKER_00Yeah.
SPEAKER_01Yeah. It's there's so much that's really interesting in that idea. You founded a company about quality. Um, and there's this whole conversation right now about the tension between slop and the tension between quality. Like we have so much more throughput, we can ship so much faster, we can do so much more. How should engineering leaders think about where the quality bar is?
SPEAKER_00Yeah, I don't want to dictate where the bar should be set, but I think the bar is higher than before. You know, I think just because you can ship a lot more code, um, it doesn't necessarily mean you're shipping higher quality code. I think there's incredible value in kind of stepping back to think, like, you know, from first principles, like what are the problems we're actually trying to solve for our customers? How do we want to solve it? And like how we actually present it to your end user. I think one of the things that we're very excited about is that you know the ground is shifting beneath our feet and you know, quality is more important than we think. And, you know, I think, and to have high quality, you need to be specific. I find myself when I'm creating linear tickets, I just put in the title and then I'm just thinking about okay, if I gave this to say some AI agent to build it, will it actually know what I want it to build? I have this random idea in my head, but it's not like you know, my engineers or our AI can read my mind. I need to be specific about what I want and what I what it needs to look like.
SPEAKER_01What do I really mean?
SPEAKER_00Yeah, exactly. Exactly. You know, you have to be clear and exact to the AI.
SPEAKER_01To your point, ambiguity is not is not a good thing for your agents.
SPEAKER_00Yeah, you can it can spend money now, it can send emails. I'm like, I don't know if I want it to do that.
SPEAKER_01You're like, how well did you prompt it? What are the guardrails?
SPEAKER_00Yeah, there's all this hype about, oh, AI agents are gonna be able to do everything for us. And there's not as much thought on like, well, do you actually want it to do that for you? It's like I don't really want it to, you know, pay my rent for me, like or like sign a lease for me. Like, you know, I want to tour the apartment kind of thing, right? Um yeah, it's like, you know, I don't really want to be in like the the Wally movie, you know, sitting there on the chair and be like, think about that movie so much.
SPEAKER_01Like it's a lot. It's in my head all the time. Wait, wait, it's been really fun chatting with you.
SPEAKER_00Thank you.
SPEAKER_01You've been listening to We Built What? If you like what you heard, please like, subscribe, share with your friends, humans, or agent. And if there's someone you think we should interview, let us know. Thanks so much.