.jpg)
The Neon Show
Hi, I am your host Siddhartha! I have been an entrepreneur from 2012-2017 building two products AddoDoc and Babygogo. After selling my company to SHEROES, I and my partner Nansi decided to start up again. But we felt unequipped in our skillset in 2018 to build a large company. We had known 0-1 journey from our startups but lacked the experience of building 1-10 journeys.
Hence was born the Neon Show (Earlier 100x Entrepreneur) to learn from founders and investors, the mindset to scale yourself and your company. This quest still keeps us excited even after 5 years and doing 200+ episodes.
We welcome you to our journey to understand what goes behind building a super successful company. Every episode is done with a very selfish motive, that I and Nansi should come out as a better entrepreneur and professional after absorbing the learnings.
The Neon Show
94% CAGR: What the Inference Boom means for your AI costs | Vamshi Ambati
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Vamshi Ambati has spent more than two decades in AI, through the symbolic era, statistical era, and the neural wave we're experiencing today. A CMU PhD, founder of LatentStructure and Predera (which was acquired), now an investor at Virama Ventures, he's one of the sharper voices on what's actually happening under the hood of the AI boom.
We discuss a simple question: Who wins when models become cheaper and more abundant?
And try to answer this by looking at how inference spend v/s compute spend is shifting, and why inference may become the biggest infrastructure opportunity of the next decade.Vamshi explains what actually goes into the cost of a token, why AI is simultaneously getting cheaper and more expensive, and why the inference market alone could reach $1.3 trillion by 2030.
If you're building in AI or someone who wants a clear mental model of where this industry is headed, this conversation is for you.
00:00 - Trailer
0:45 - How an AI researcher thinks after 20 years
05:53 - Where enterprise AI adoption is headed
08:35 - Drawing parallels between cloud and AI
11:20 - If building is cheap, what's valuable?
13:37 - Can computing get cheaper?
16:41 - What is inference, really?
22:22 - Why coding and customer support got eaten first?
26:48 - Which technologies are overvalued and undervalued?
29:56 - An accidental entrepreneur's journey
33:15 - Why is healthcare slow to adopt technology?
38:59 - Landing Walmart as a customer
42:36 - Should founders build in services if product isn't visible?
43:47 - Is Palantir a product company or a services company?
44:15 - How to win as a forward-deployed company
46:23 - What it takes to land large enterprise customers
49:20 - Building sales muscles as a technical founder
-------------
India’s talent has built the world’s tech—now it’s time to lead it.
This mission goes beyond startups. It’s about shifting the center of gravity in global tech to include the brilliance rising from India.
What is Neon Fund?
We invest in seed and early-stage founders from India and the diaspora building world-class Enterprise AI companies. We bring capital, conviction, and a community that’s done it before.
Subscribe for real founder stories, investor perspectives, economist breakdowns, and a behind-the-scenes look at how we’re doing it all at Neon.
-------------
Check us out on:
Website: https://neon.fund/
Instagram: https://www.instagram.com/theneonshoww/
LinkedIn: https://www.linkedin.com/company/beneon/
Twitter: https://x.com/TheNeonShoww
Connect with Siddhartha on:
LinkedIn: https://www.linkedin.com/in/siddharthaahluwalia/
Twitter: https://x.com/siddharthaa7
-------------
This video is for informational purposes only. The views expressed are those of the individuals quoted and do not constitute professional advice.
Is there any point in time where instance kind of global is a modern computer one company started research 20 years ago in AI space?
SPEAKER_00The instance market is what the fastest growing tag is. 94% from 2026, we are looking at an $80 billion market. By 2013, it's going to be a $1.3 to 1.4 trillion dollar market. The whole big bet of OpenAI findabilities, target, and other big data centers is not that they just want to train on this. That cost is a one-time cost. The entire infrastructure cost of AI today is largely going into inference.
SPEAKER_01This is when cloud came. Hi, this is Sidhaat Alwaliya. Welcome to the Neon Show. Today I have with me Mamshi Ambati. He has been a founder, researcher, and investor. It's a very unique journey where an individual started in research 20 years ago in AI space and now it's all paying off, right? Welcome, Mamshi, to the Neon Show podcast. Thanks for having me, Sid. Finally it's happening. Yes, yes, finally we are able to make it happen. So glad we are sitting across, you know, and discussing things of our both of our interests, you know. So, Mamshi, if you have to summarize uh uh the current juncture that we are today, what's happening in the macro, in the AI world with anthropic, cloud, open AI, how would you summarize it to a layman and uh which way do you think the industry is going uh in the next couple of years?
SPEAKER_00Got it. Yeah. So um, I mean, to start off, it's a it's the best time to be an AI engineer, right? So you have uh enormous amount of compute at your disposal. You have amazing um capable models that are actually accessible to most people today. Um and you have you know the level of sort of access and distribution that is unprecedented. You know, you build a small app, you know, you think of Clawbot or all these other um apps that people are building today that are going from zero stars to like maybe 100 K stars in less than a month. So all these three things together at a point in time where you know we have this sort of trifecta of things coming together, um, you know, good compute, good data, and good distribution. It's great to be an engineer, you know, building things, experimenting quickly. Um and having said that, I think it's also very difficult to get things right because the same three things are accessible to everyone. So really to stand out and then build something amazing in this day and age is going to be uh you know exciting, uh nevertheless, right? Um and when I look at um anthropic, open AI and others, you know, what has happened is um, you know, uh like to say taking my journey as an example, I've seen um or I've been in AI for about 20 years now, right? So two, two and a half, a little more than two decades, right? So my undergrad, um, and if I have to think of it as, you know, there were about three waves in AI, and I have probably been part of all three of them, right? So and I'll explain uh what I mean by that. Um so typically the whole uh NLP and machine learning space, uh this was more symbolic in nature. So think of the first wave as the symbolic AI wave, right? Um where you're building rules, you're thinking about how to bring a flowchart or a logic to how you want the computer to think, right? So bringing more structure to the computer thinking. And then you have the second phase where you went from the uh the symbolic AI to statistical AI, because you didn't want to teach the computer through a logic diagram, but you wanted to give it a lot of examples and data and say, hey, go figure this out, right? And then the third phase where we are in right now is the neural AI, um, where you're not necessarily changing a lot of things, but the scale that you're providing to these algorithms is gonna, you know, the the loss of scaling and everything coming together, the compute and the data at this scale given to neural machine learning algorithms are is the third wave that you're seeing today, right? And so I've been through all these three, and then when I look at you know what's happening today, the pace of innovation is very different, right? So the first rule engines, you know, you'd build these rule engines, you'd build these expert systems, if you will. And then modifying these was so difficult that it took a long time to change things, right? In the statistical phase, collecting data was difficult. You know, data was just not existing everywhere for people to build, right? So the um, you know, I I mean you a lot of people have access to like you were building vision models and so on, right? So you have these data sets which are like you know, a few hundred images, a few thousand images. Um if you have to change and iterate and move uh things, uh you had to bring more data into the picture. And that was getting you know very expensive as well as time-taking. So you wouldn't find the pace at which these models were rolled out. Uh but what you're seeing today is a whole different beast. Every month, if you will, there is a new model, right? That's beating the old model and visibly adapted already, and people are moving to the new models. Um, and the data is just flowing, like you're generating data, you're sort of getting access to new data, created by old models, and sort of improving on top of it. So it's it's an amazing time to be an AI researcher, as well as for someone to, you know, someone who has seen all these three waves, it's it's like the pace is what is baffling me this time.
SPEAKER_01So And where do you think the industry is going in terms of enterprise adoption? Will Claude eat up all software, as I said, because currently the SaaS prices are down?
SPEAKER_00I know, I know. That's that's definitely on everyone's mind, right? So the uh I think enterprises is a is a whole different beast, right? So um again, this is my take. And having been uh you know the second phase of my life where uh you know I've been an enterprise AI entrepreneur, um I I look at it slightly differently, right? So, you know, getting enterprise right is about um in a couple of things. I would say enterprises don't optimize for accuracy of a model, right? So they're more so optimizing for reliability and trust. So it's very um it's very, I think in today's age, with this new uh level of capability that the models have, getting the accuracy right is available to everyone. Uh but who gets the trust and reliability right in enterprise AI is probably gonna win the you know the the likes of these deals and and the and the customers in enterprise, right? So I think the uh the next two years is gonna be it's a shift that is happening, uh, where you know, like you know, if you zoom back a bit, you know, how did enterprises move from all these waves of um, you know, you have uh you know your on-prem to the cloud shift, you know, the mobile shift, and then there is the the big data shift, and then you had the uh the predictive AI shift, right? So now with generative AI, what has happened is whatever the whole fabric that the enterprises were built on has now shaken because enterprises, at least in the SaaS world, were about capturing workflows, right? So you had a workflow and then you're capturing it in some sense to achieve a task, right? So now generative AI is coming in and saying, I can actually generate that workflow for you, right? So then the basis under which the SaaS companies are lying has been shaken a bit. Now we have to look at SaaS different from enterprise, because at the end of the day, whatever you build, whether it's a clawbot or whether it's um you know generative AI apps, you're still selling it back to enterprises that are going to be using it, right? So getting that right uh is is needed today for all the startups that are coming out. Uh, but SaaS is a whole different conversation. I think it's just a transformation that we are seeing, going from SaaS to maybe call it an intelligence, uh intelligent SaaS or whatever, but it will transform. It will transform into a space where uh generative AI is going to be very integral to you know the workflows and the intelligent workflows that are gonna be defined.
SPEAKER_01So in the previous eras, when cloud came, for example, from on-prem, so on-prem didn't drive die. Else companies like Nutanix wouldn't have been born. And it said that globally cloud is only 10% of all on-prem that is there, right? So in terms of the new, you know, the current uh model acceleration that we are seeing, so would it be similar that you know it it'll be like 10% of all software will belong to the model layer?
SPEAKER_00That's uh that's a tough one to put a number on, uh, but your intuition there is right. That you know, what's happening is you know, going back to the the two pillars on which enterprise uh sales is dependent on, right? You know, trust and reliability, getting these two right takes time, right? So which is why what you have seen in the cloud space, you know, it the adoption curve for these enterprises is gonna be longer than what people would imagine, right? Because getting trust and reliability right uh takes time. It takes time for someone to say, I'm okay moving all my data into the cloud, um, I'm okay, you know, moving all my compute needs to the cloud and not have and own my own data center. And then that could happen 100% in some verticals, um, but it may not happen in some regulated verticals. So I think the uh the the time from new innovation hitting to enterprise adoption is gonna be longer. I don't think that will change. And really that is the opportunity where you know people are playing, right? Um, what percentage will be owned by the model? We are starting to see more evidence that models are gonna own that more than the 10%. Uh you know, whether it's the new plugins that uh Claud has released that shook up the whole SecOps market or the more recent uh sort of clawbot that has come in, that is starting to question productivity for everyone, and then think of organizations that is very flat. Um all these are good innovations. But for that to really uh mature into a reliable and trustworthy technology is really where the current startups will have to really think. Um it's it's tough, but the advantage is towards startups who get this right, because I can't imagine an existing behimat going back and saying, okay, now I already have your trust in reliability and everything, I can take care of that, but I'm going to reinvent and now be an AI company. That's gonna be a tough sell. But a good innovative startup trying to address the trust and uh you know the reliability angle will probably be in a winning side.
SPEAKER_01When building today has become cheap, what has become valuable now?
SPEAKER_00Yeah. Um so when you say building has become cheap, I'm I'm assuming that it's the access to these models that uh you know, a prompt is creating an app, uh, prompt can create a workflow. Um so uh I mean but I want to question that because um you know today the today I think uh yes there is access to models, and then these models require a humongous amount of compute behind the scenes to really take that prompt and convert it into a full-fledged running app, right? Um and let's just talk about that part. Um, if the tokens prices go up by 2x, 3x, will all of us still be building those same throwaway apps or one-time use apps or an app that we already have been using for a while, but we want to rethink about it. Uh I don't think you know the access to compute today is gated by very few providers. Um and you know the models are definitely in the hands of like a couple of them, if you will, right? So I think it it has become easy in the sense that directionally people know that this can be done, but I would still question that the access is not fully available to everyone. Um, you know, I, for one, in the last uh couple of months, uh, when the models have gotten their reasoning capabilities uh you know much to a different notch, um, what we have seen is they're consuming more tokens, and then you run out of these uh token limits, and then you sort of hit these limits, and you can't build your models anymore. Uh you can't build your apps anymore. So I think they're gonna get it more expensive as the models get better. Um, the compute needs are gonna increase, and then you're gonna get better. So it's uh it's not for everyone to build uh these applications. Right now, what we've what we're seeing is uh is sort of a teaser of you know what's possible, uh, but to truly make it accessible to everyone, I think the compute has to be solved, and then we need to make sure that compute and models are actually available to everyone, right?
SPEAKER_01So is there any effort going on where to make compute let's say 100x cheaper or mod the price of model 100x cheaper, which is very counterintuitive to what you said. You said the price of models are going to become 2x or 3x.
SPEAKER_00Yeah, so um the you know, there's so so think of the um the price of a token, right? So if you go into the anatomy of a token and then the the general price of you know what constitutes the cost, right? So there is the model training cost that has gone into the years of training or the number of GPUs that went in and then the scaling loss that they had to hit in order to get a better model out. So that's all baked into it. Uh the second is you know, how uh what sort of chips do you have access to while running the inference layer? Because you've built your model, but now you're you're getting into the inference phase. And then what level of accuracy do you want out of these models? Do you want to run this in a in a lower quantization where it's cheaper for you to get these tokens out, but then you're losing some accuracy. Um there is the cost angle, um, there is the sunk cost of training, uh, then there is the hardware sort of limits of you know what hardware you're running on. Primarily it's been NVDS so far, but then we'll see you know more coming up, right? Um and so you know when you start bringing the the the and then how much throughput, how many users you want to serve and everything else. So when you start bringing all of these together, um you realize that you know you have to sort of give and take somewhere, right? So the trade-offs are like across these four or five dimensional problems where you have to pick for your use case what works best and you know where you sort of uh draw the so when it comes to reasoning models, everyone wants the best model. And so you're looking at you know the cost going up, right? But there's definitely innovation on the software front of making like all the innovation that we've seen with the memory bottlenecks, the uh solving the KV cache, uh the disaggregated serving where you're you know considering multiple kinds of hardware, one for pre-fill and one for decode. Um so there are lots of these techniques. And again, a plug is we've written up some of this as a book uh called Peak Inference. It's on Amazon for anyone who wants to download. Uh a friend of mine, Rajan and I wrote this book. So there's all this innovation on the software side, which is trying to drive the overall cost down, right? And then there's all the innovation on the modeling side where the model's cost is going up. So you have to sort of bake that in. Uh, and then the usage is going through the roof. So, you know, more users using it for more things is obviously going to drive uh sort of a shortage of the tokens, right? So, yes and no in different places. So it's gonna get expensive in some places, the software is gonna make it cheaper, but then the need is gonna you know drive the demand higher. So there is a time in the next year and two where we'll start to see these directionally, yes, uh all the token prices are going down, but as the models are maturing, we are starting to see that you know the the opposite effect as well, where you know the cost is going up for the latest models.
SPEAKER_01So right now, globally, let's say AWS revenue mostly from compute layer is $140 billion.
unknownRight?
SPEAKER_01That's the AWS revenue that's gonna come out recently. Uh assuming the total cloud revenue of the entire globe is $250 billion. Seeing AWS is the leader in that. Uh right. Do you think is there any point in time where inference revenue of the globe or the or the inference end of the globe will become more than compute?
SPEAKER_00Interesting. So um so I've been looking at inference for more than four years now, right? So, you know, right before the um like uh we were building uh Pred error, where uh the LLM ops layer that we built was primarily focused on how do we optimize for inference. And that was the company that we exited to a data center company, where we are also now looking at you know in these similar sort of problems and spaces. So the uh the the inference uh market is one of the largest uh also the fastest growing Kagger. So what we are seeing is about uh 94% uh Kagger. And then you know, today, I think around 2026, we are looking at an $80 billion sort of uh market. But soon by 2030, uh it's gonna be a $1.3 to $1.4 trillion market. So that market is a huge one where we are gonna be seeing, I mean, it's the it's this decade is uh the diffusion decade for AI, right? So you're going from models that people are happy with to where are we gonna figure out the end use cases? And we are starting to see already, you know, two to three of really good use cases, like the coding being the number one. Um you have the voice models in uh you know, customer facing um sort of you know, uh customer support in other observable CS use cases that that we're looking at, right? So we are seeing this decade where a lot more such use cases will be using AI, right? So this is the diffusion decade for AI. So that means uh inference is at the center of the conversation, right? So and that market is growing huge. So the the whole big bet of OpenAI, you know, trying to build these Stargate and other big uh data centers is not that they just want to train on this, because that cost is a one-time cost. So you train ones, you sort of run inference for a lifetime. So that the the entire infrastructure cost of AI today is uh largely going into inference because the diffusion has already started and it's only going to get it's only be it's getting accelerated, I would say.
SPEAKER_01So got it. Let's say for our uh users who don't come or listeners who don't come from AI background, how do you explain them inference?
SPEAKER_00Got it, yeah. So um yeah, so let's take an analogy of uh uh of of some of our brain. The human brain has you know a very big neural network that we have trained over decades. Add the DNA memory as well to it. But let's say the the human brain is now uh you know taking a lot of inputs and then reacting to that could be through one of the senses, right? So you're speaking or you're like, you know, taking some action or you're sort of sensing a uh or or feeling a thing through the skin. So a lot of these um inputs are processed in the uh in your brain. So essentially think of the whole decades-long of learning that you've done as a kid growing up, exposed to language, exposed to visual cues, you know, all the rules and the societal uh contracts that you've been through. All of that is the training phase. And now you have a well-developed mind, and that is now starting to make decisions, right? So that's making decisions, that's taking actions, that's speaking words and everything else. So to me, that's inference, right? So in your own space, there is a, again, people say most of the learning happens in the first few years of your character forms in the four years and so on. But uh, let's say, even in argument's sake, you learn till 25 years. So your model is already built. Um, right, but then think of how much of uh sort of decision making and then you know, sort of activity that you do with your brain. All of that is inference, right? Uh so inference is nothing but you know, how do you take all this input, process it through all your neurons, and then you know get that in the final outcome, right? In the large language model space, so that model training activity is essentially the training phase where you're you know running through thousands or hundreds of thousands of GPUs, where you're training all the parameters of your neural net, right? And that uh essentially during inference, you have to step, do this uh in a forward pass where you're going through all the parameters, computing the math to say, okay, if I see this word and that next to this there's another word, how do I then process that the next word that is going to come out, right? So that prediction of that next word is what we have simplified it to in the case of L11s, and that's inference today. And so the bigger the model size, the more expensive the inference is. Because it's going to be running through that entire list of parameters to compute what is that word coming next, right? It's the next word prediction problem, if you will. But things get complicated when you have other modalities like video and voice coming in. But by and large, think of it as, you know, like when you have a context, predicting what comes next is the inference problem.
SPEAKER_01You mentioned that the biggest applications the world that till now has seen for inference is one is coding and the other is customer support. Why is that, right?
SPEAKER_00Yeah, so both come from, I mean it's a very good question and uh and even something that I think about like why is it that coding has taken off such in such a big way, right? You know, anyone would imagine that software engineers would be the last thing to get automated, but you know, AI went straight for it, right? So you have uh software. So uh so think of um you know just the evolution of programming, right? So you had um, you know, it's it's it's very deterministic in nature, meaning you're going from um, you know, your uh your your grammars that are sort of generating, and then your your uh you know your programs uh are essentially code that adheres to a certain uh certain uh you know context-free grammar, right? So in your in your programming language. Um so you know, when I look at the the coding, there is there are two things that are happening. One is the output is so deterministic that you have a static piece of code that you can actually take and then run it anywhere and validate it, right? So when something can be brought down to that uh where whatever is the level of abstraction that you're dealing with, which is English language, you know, prompts and everything, but then that gets distilled down to an artifact that is static that you can deploy and pretty much validate, uh, it becomes much easier to get that feedback loop right. So to me, that is one example of like why the coding is taken off. Uh and and then the second reason I think the coding has taken off is when you look at the level, the amount of code that people write today, uh, there's a lot of boilerplate code, right? There's a lot of repeatability, there's a lot of reuse. The whole evolution of programming languages happened around the concept of reuse, right? You had these monolithic C programs that then had to be more reused. So you sort of brought in abstractions that became your C, and then so on and so forth, right? That's how uh things evolved. So there I feel like the the two things working towards why you know something like coding, I can understand that part. What I don't understand is you know the human angle part of it, right? So places where mundane work, which is calling people, uh whether it's the sales calls, whether it's um you know customer support calls or like you know, debt collection calls. Uh, in fact, like lately I'm seeing a lot of uh banks reaching out with automated bots reaching out saying like we have these offers and so on, right? So that is like, I think at this point in time it just felt like you know something that's very mundane and and repeatable action that humans don't want to do, they're starting to offload that to this thing. But I don't fully agree to that because anything that has a human-to-human touch point should be the last one that should get automated. Um today we're just probably seeing that first uh sort of spike, but then that should probably you know die out soon, I feel.
SPEAKER_01Um but these are two extremes which customer support has companies like Sierra, Decagan, which are already almost $10 billion companies.
SPEAKER_00Yes, yes. So um, I mean the way to uh I guess it's it's the current spike, if if you think of it. Um I mean the models have to get extremely good uh so that they sort of have that empathy, they emote correctly with humans. Um it's very easy, like when you get a call today from you know one of these robotic or agentic models, uh, don't you sense it within like the first time?
SPEAKER_01Absolutely.
SPEAKER_00It becomes so so after that, what happens to the brand uh association? The next time you get a call from them, you're probably gonna, even if it comes from a human, you're not gonna pick up, right? So I feel like you know, pretty soon, you know, before the spike dies out, uh, we really need to see better models that, you know, sort of MO2, have that empathy angle and everything. And at the same time, have the right uh interventions with humans, right? So if we have both, and I'm betting that I'm companies like Sierra and others will figure this out soon. Um, but if they don't, then I think the spike has to come down.
SPEAKER_01So current in the current cycle, what do you think is overvalued and undervalued?
SPEAKER_00In the current cycle. Um I think in terms of I speak in terms of tech more than the more than the startups per se. Um I think the um agents are overhyped in the short term, but I feel maybe they will be like underhyped in like the five-year term. So in the five-year term, I can clearly see a lot of my day-to-day work being offloaded to agents. But the current hype around agents is such that you know, a clawbot releases or a Hermes agent releases, or uh, you know, or a Nemo Claw releases, and there's so much buzz around, okay, this is going to change everything, every company will be run by agents, uh, you don't need a CEO anymore, you don't need, you know, like the the organizational hierarchy anymore. So that's the the hype part that I don't like. But I can see these sort of things happening soon, uh, but probably in a mid to long term, but not necessarily in the short term. Um undervalued. Um I think the the there was a time where we looked at um like like how difficult voice was to get right. Uh I mean, even though I complain about voice not having the emotions and everything. Um voice to me is still gonna be the frontier uh through which people will interact with AI. So um so it's it's so I was working in TTS uh like text-to-suite systems um maybe 20 years ago as part of my my course project. And that was so difficult to get these things right. But now uh, you know, in 14, like I think uh Google, Gemini is probably like in 140 languages or some such, it's so much easier to do the the voice modality. Uh so I I think that was that is one that is has matured and but it's not yet fully diffused. And it's ready for diffusion, but it's it's underhyped, I feel a bit. It's a voice as well.
SPEAKER_01You think undervalued also then?
SPEAKER_00Undervalued also. I think 11 Labs is doing okay. Um but imagine you know a country like India, uh, how do you think AI will get diffused? Like you have so many languages, so many dialects, uh, you know, so much variations and uh you know how people sort of even say the same thing, right? So forget about the vocabulary, but you know, in the same language you have so much variation. Um, and then the level of literacy that is required for you know what used to be called digital literacy, uh, you're not thinking about AI literacy, right? So you have to go from, you know, how do I type things into WhatsApp versus, you know, how do I use AI, right? So how do I use this chat GPT? Everyone wants to know how to use this thing. And voice is a very good modality to sort of interact with AI, right? Um so I think the use cases are plenty for voice. It's just that I feel it's a little undervalued today.
SPEAKER_01Um you started your entrepreneurial journey with services and then pivoted into MLOs. Like, but not many entrepreneurs who start in services are able to do that or into product also. So what led you to having a successful transition?
SPEAKER_00Right. Yeah, I've uh I I can't well, so my journey as an entrepreneur started in a more accidental way, right? So it's I'm an accidental entrepreneur. I didn't really set out to build a big company. Um I was basically always thinking that I need to test my potential a bit more. Um so when uh in working at companies like you know AT ⁇ D research labs or uh you know PayPal, I felt like you know there's enough learning there, but then I had to push my potential a bit more. Um so I went to another startup that later got acquired by Zendesk. And that's where I felt like, okay, I can understand, I understand what it takes to sort of go from a Series C company to get an exit to a company like Zendesk, right? Because I was leading the data science side of things, which was uh the big thing in 20, you know, 12, 13 timeframe. Um but uh when I was really looking at, you know, I wanted to do something different and something where I could challenge myself. Um I think it was more of a personal thing where my dad had some health issues. Um and so I was like, okay, I understand data science and I want to apply this to a place uh where it's more impactful and things that I could you know bring back to my personal life. Uh so uh that's how I wanted to just you know start or do something in healthcare. And honestly, without understanding anything about healthcare, uh, but I had the passion to learn and do something in that space. So I went to a hospital in LA and said, I'm gonna just work with you for the next three months, teach me everything about healthcare and and sort of uh you know, let me help you because I'm a data science expert. So that's how I just like went, stayed with them uh for about three months as an in-house. Quit your job. Yeah, quit my job and then started off. It's it's pretty stupid to do that. Or when it looked back. This is 2016? This was 2016, and I just had my uh like my uh my kid uh at the same time, and then I quit my work. And but it's just that that passion or madness, whatever you want to call it, I think that drove me to saying, okay, I have to do this at this point. Um, so like what uh started as a good learning exchange between uh me and the hospital, and these guys were no small uh shop, right? So they were uh fifth largest provider in healthcare. So it's called Prime Health out of uh Indian US. They won about 48 or 50 plus hospitals. Um it was a very good learning experience for me. And then in return, I could from close quarters see what does it take to work with an enterprise? Yeah, you know, why do they not trust? It's not like they, you know, like people say that healthcare is slow to adopt, right? So when it comes to tech. Uh but I could see from close quarters why they were resistant for change, like why did they not move to Azure? Why weren't I uh not trusting the predictive models? And what was the reason for that? Why were they slow? It's it's a very different space, right? So talking about healthcare per se, uh, you know, you need to first understand how the roles lay out in the healthcare space. So you have uh a hospital, like in the whole uh they call the four piece, right? So you have the provider, patient, pharma, and then the um and the uh the payer, which is the insurance company, right? So the four piece. In that the equation of how the the providers, which is the hospitals, uh what are they for? Uh, you know, they're looking at you know, sort of maximizing the overall uh occupancy rates, you know, how quickly they can move people, and then the the legal framework around you know what or how uh you know the American hospital system is set up, where you know what are the laws that they need to abide by, and then you know, sort of the metrics that they get penalized on and so on. And so there are different roles for each of these, right? So there was this new role called a CAO role, which I never thought existed. It was the chief administration officer who actually oversees everything, all the metrics around the doctors, right? But when you build something, you typically build with the CTO in mind, thinking, I'm gonna build this and I'm gonna go and sell to the CTO. The CTO is where you sell, but then you actually get a check from CFO, but it's used by the CAO. So all of that mapping, like no one, unless you're inside, you don't really understand some of these things. So I think that that exposure led me to, you know, coming back to your question on the services side. Yeah, they were like, you know, they're starting to trust me, but then they were like, and I didn't have a clear view of a product. And they were like, okay, I want you to take on more work and then you know help with the work.
SPEAKER_01They'll just give you more work for IP. Exactly.
SPEAKER_00So then I was like, okay, so let's start a company around it. And then you're later. Exactly. I start exactly. I started out learning healthcare and then started the company three months later. And that was the latent structure, which is the services. And how big that customer became? So uh that customer, you know, um funny time. Like I you need to think of your first customer, not in terms of revenue, but in terms of credibility, right? So they've added so much credibility to my tag that I could actually I didn't even have a startup at that point. But whatever I started afterwards, with this knowledge that I could gain from watching close quarters, I could go to a company like a GSK, and that became my third customer. Uh I went to another Parkland Health in um in out of Dallas and they became my customer. I had like three or two more hospitals, one in Seattle and others.
SPEAKER_01In in the entire journey at Pradera, right? How many of your percentage of customers were in healthcare?
SPEAKER_00Yeah, so this is where uh you know um like the my my interest in healthcare for the first one and a half year and and like trying to chase and you know land these customers, being a first-time founder, not understanding healthcare, sort of hit limitations, right? Um so the sales cycles are really long. Um, you know, trying to gain trust. And once you are in, you need to know how to sort of leverage that and then go to the next level of revenue unlock, right? So if you don't do that, uh then all the hard work that goes into cracking that enterprise account really doesn't matter much, right? So all of these are learnings and I can look backwards and say this is how it uh worked. But uh, the you know, coming back to the hard part of you know running services and product, you know, when you're running services, you understand you know all this, but you know, you should also invest in how to capitalize on what you've built so far. Whereas what I was capitalizing for is you know the product that I could see, right? When I was working with multiple hospitals, I saw the same problems. Then, you know, because of the long sales cycles, we went to you know fintech as a vertical, so we worked with MasterCard afterwards, um, then we went to Pharma as a vertical, GSK, um, then we went to retail as a vertical, Walmart. So working with different companies, I was optimizing for how do I now build a product out of it, right? So you can go double down into the same space and you know unlock more services revenue, or you could see the common pattern across and then unlock a product value. So I went the other route, you know, trying to build, and that's when Pradera was started in 2019 as a product-only company, right? So we saw all of this and then you know built out Pradera with a view of, okay, this is the MLOps problem that we are solving, and let's give it a tag. And surprisingly enough, 2018, uh-19, when we started out with this product idea, uh, I was testing the waters with, okay, let's go knock on the doors of all the VCs in uh Sandhill Road and see what they think about it, right? We were like, we were welcomed in every single uh, like all the big uh, you know, the folks who were looking at AI, like Anderson Horowitz and you know, General Catalyst and Foundation Capital, one of your um folks on the podcast. Um, all of them looked at us. Uh, they were all pretty interested that we were looking at MLOps as a problem because MLOps as a vertical started much later, like maybe 2020, I think, uh right around that time frame. We saw five or six companies come up. But we were the first wave of MLOps companies that wanted to productize this. So I wasn't in a fully successful, you know, coming to the limitations of running services and product. It's very hard to sort of make the transition into a VC fundable product space. But then I realized that I couldn't make the transition, but I want to continue this services sector that actually feeds the real use cases to me, you know, has the uh, you know, opens the doors to these real customers. And so wanted to continue it as a bootstrapped product. So that takes a toll on you personally, but I think that transition is hopefully uh smoother than the VC funded uh transition.
SPEAKER_01Understood. And and all of this could happen because you back then the term was not coined, but you were a forward-deployed founder in a hospital.
SPEAKER_00Yeah, absolutely. I think it started in a very accidental manner, turned into a services company and a product company after that. Yeah.
SPEAKER_01And when you exited, like how much of the revenues were coming percentage? No need to answer in absolute, but percentage were coming from services versus products.
SPEAKER_00Um so product we exited uh very early, like I was I was telling you, right? So uh we we had to make pivots, even in the product, you know, things change, right? So we started out with this MLOps product, but we were innovating on the monitoring aspect of AI because AI reliability and trust, as I keep going back to that, that seemed like the pillars around which enterprises are harping on. So we said, okay, let's build monitoring dashboards for AI. Let's you know make sure that you know whatever decisions are run through AI, we know exactly why they were uh happening that way. And so uh the MLOps product uh was starting to take shape. And you know, funny enough that the the first customer for our MLOps product was Walmart, the largest giant in retail. And you know, they they bought it not for themselves, but they wanted to open it up for all their suppliers. So it was a very big thing that could have just turned uh into gold for us. It did work for a good year and a half before COVID hit, and then a lot of those uh initiatives at Walmart sort of you know took a backseat. But from every crisis comes an opportunity. So we were watching very closely what was happening throughout this, and you know, we really caught the wave of LLMs, right? So in 2022, when the LLMs came about, we were in the right mind space as a company, as well as founders, where we thought about it as this could be it, because I could clearly see, you know, like just from my journey itself of you know, seeing that symbolic AI transitioning to statistical AI. Yeah. And uh that really helped my PhD shape out as well, where I could turn my PhD was in statistical AI, even though I started my sort of grounding in uh in symbolic AI. Uh so when this happened, I could clearly sense that this is a sort of a shift that doesn't come across too often, right? So 2022, we said the boldest thing that we did was um we did not renew the contract with Walmart because they said uh you know, continue the MLOps product. And we said we'll be an LLM ops company. And Walmart was not ready for LLM ops at that point. But we said, okay, fine, we'll stop this and all in with LLMs. Uh 2023, uh I think like beginning of Jan timeframe, and then the open source models started to come out at that point in a small way that really helped our uh journey as well. So we pivoted to this LLM ops product, and I think that uh shaped out. So we didn't collect too much revenue, but we had enough users on the LLM ops side that was interesting for so most of the revenue was coming from services side. Yeah, so I would say the services to the split was about uh 15% or 20% uh tops for the product uh to the uh the services side. Yeah. And both companies we sort of exited in in two different uh two different transactions. But to the same no, different, different. So we sold the services separately, and then because the product company, the value of the LLM ops platform to the to the buyer was a lot more. So they didn't really care about the rest of the services.
SPEAKER_01So would you advise founder to start with services if no product or problem is visible?
SPEAKER_00Um interesting. So I mean I'm a bit biased, like from what I can see. But um just stepping back and thinking about what's happening with AI today, you know, to your earlier question on if it's become so easy to build things, you know, where does the founder focus on, right? Uh and I think like today's AI, uh, you know, the the problem of AI is not intelligence, but it's an integration problem. So you have to figure out how to integrate this into the right workflows, gain the right end users, and then sort of work with. And this is um this is a classic enterprise sales problem, right? So, which can only be solved by you know forward deployed engineers like the Palantir model, or um, you know, figuring out the right recipes, or you know, let the use cases um sort of come to you where you're working on these use cases and building solutions that you can take to the customer. So all of these are kind of quote unquote solutions and services, or non-recording engineering. So, what do you call Palantir a services company or a product company? Uh it's um it's a good it's a great product disguised as services. So it's I'm sure there's merit to the product, but the launching vehicle is services, right? Because these are hard to understand products that you have to take to the customer. There is a long lead time of getting trust in the product. All of this can only happen when you're embedded with closer to the customer.
SPEAKER_01When you are forward-deployed as a customer like you were, how do you ensure that you are not building a uh, you know, say just for this customer, but you are building a real company, else you can turn out to be a one-man shop for this.
SPEAKER_00Absolutely, absolutely. Yeah, yeah. That's that's the that's the biggest concern that any person subscribing to this direction of let's go build uh this FDE approach of learning the problems with, right? So I think the the two things that we followed, um which kept us a bit more honest to this, were the building the product ambition out. Was one, uh, you know, we said until we get repeatability with five customers, let's not think about a product, right? So that's one rule of thumb we said, okay, let's just build it out, build it out for five customers. Then you know the second time you work with someone in the same vertical, uh, it should not be this, the it should not be as difficult as the first time around. So if we start seeing that, then you know you know that that metric is off, then you clearly don't have a product. So we were very closely guarding those two numbers, right? So we wanted to get uh applicability in a broader range across five customers from different verticals, and so the the MLOps product really helped us get that, right? So we were working with the the top three in every vertical, right? So whether it's a GSK in Pharma or Walmart or a MasterCard and so on, we had we had those bases covered. You pick the verticals we want to play in, we want to be working with the top, one of the top three. Um, and then within that, once you land them, also the GTM becomes easier, right? When you sell to the best guy in that vertical, the second best guy or the third best guy is more open to talking to you, right? And then when we can do that much quicker, uh then you have a much better leverage as thinking about a product. Until then, I wouldn't say don't even pretend that you're building a product because that'll hurt you more than it'll help you. Because you're basically thinking that you're building product, the cycles go there, but then they'll really just start to look like multiple products and not just one single product.
SPEAKER_01Any any other learnings, one thing that you have from your journey that you could share with founders?
SPEAKER_00On the enterprise AI side, um, I think the the biggest learning is so I'll speak about it from the startup angle because you know it's often people, you know, the question that I get asked very often is how is it that you landed some of these big names, right? I didn't have a sales team, I didn't have a marketing team. It was uh it was purely with the products that we were building and then the ability to deliver, right? So I think the the the two simple things I can I can say from my journey of uh you know sort of distilling that sort of uh know-how is one, um enterprises are actually looking for solutions to their problems. Um and quite often what happens is the big companies that they're working with, the the managers, the VP and below levels, like the directors and managers, they actually don't approve of the the call it the big five or you know the different firms that come and pitch uh you know large solutions, big PowerPoint decks, they don't they don't get their trust, right? And and they look at that and say, okay, if you're forced to take this uh company and work with it, we'll do it. But if is there someone else that is solving this, right? So if you can position yourself as someone who goes in with that uh level of authenticity, goes in and you know honestly says, I'm good at this particular piece, I don't know the other pieces. Uh but can you position yourself to solve their problem? Yeah, they will trust you. They will trust you, and then they will come back and tell you, you know, hey, uh, I can you customize it this way? Can you do it that way? Because then they know that they can work with you, they can talk to you, right? So you need to be open to that, find those opportunities where where they're unhappy with these, you know, the the big uh the big the big pitches, right? So that is one that I've learned uh from IGN, if I can distill it. You know, first build that trust, and then you yourselves can be that uh sort of beacon of trust where you're going in and then sort of helping, right? The second thing is you know, don't be shy going to the biggest names in the in the bucket, right? Um it took me nine months to land Walmart, right? But I've I've known people who have spent years and still couldn't get into Walmart, right? But we had the right product. We could approach them with uh in a point of trust where we could work with them and sort of were going in with something that we were really good with and then sort of open to making changes in the way that we wanted, right? So whatever it takes, like how much effort effort is required, just go for the first uh, you know, the top three at least, right? So then good things will fall after.
SPEAKER_01And how did you transition from a uh a technical and a research founder to a sales, uh leading sales in your organization?
SPEAKER_00With a lot of scars on the back. It's uh yeah, so I mean it's good and bad, right? So the like when I walk into a room with a customer, uh they don't look at me as uh someone coming to sell them, but someone coming in to solve their problem. Yeah, right. So that trust comes from my technical background, right? So that definitely helps sort of position myself in that. But then there comes a point where the sale doesn't move forward. There comes a point when you need to step up that game of I'm gonna follow up, I'm gonna understand, I'm gonna sort of read the room correctly, uh, I'm gonna, you know, sort of uh you know, take what it takes to like play what it uh takes to sort of close the deal. Right. So those really the first few, I mean, uh I I traveled the first one and a half years of uh you know running the services firm, I went all across the US. About I drove literally 50,000 miles in my car, uh just going to people and talking to people and trying to understand, and then being very critical about myself and where I'm my shortcomings are, right? So then it was clear that okay, sometimes in some deals you need to go as two people, not one. And sometimes you need to follow up at the right points in time, right? So sometimes you're like uh you shouldn't speak, like you're just you need to be quiet and listen to their pain points. So all these uh took a good year and a half of talking to a lot of uh I think the the failed deals taught me a lot more than the deals that I won. So that's just being self-critical at different points.
SPEAKER_01Well, thank you, Ramshi. It's been phenomenal to record this podcast with you. Uh thank you for sharing your journey super candidly.
SPEAKER_00Thanks. Yeah, I think if at all my journey uh helps you know another entrepreneur in the same space, you know, I'm always happy to sort of help and talk to them. Um and I said I have this thing called Virama Ventures through which I support you know founders who are sort of similar spaces where I sort of identify and I'm passionate about so always happy to talk. Thanks for the opportunity to talk to you guys. Thank you so much.