From Startup to Exit
Welcome to the Startup to Exit podcast where we bring you world-class entrepreneurs and VCs to share their hard-earned success stories and secrets. This podcast has been brought to you by TiE Seattle. TiE is a global non-profit that focuses on fostering entrepreneurship. TiE Seattle offers a range of programs including the GoVertical Startup Creation Weekend, TiE Entrepreneur Institute, and the TiE Seattle Angel Network. We encourage you to become a TiE member so you can gain access to these great programs. To become a member, please visit www.Seattle.tie.org.
From Startup to Exit
Gen AI Series: The past, present and future of Gen AI with leading industry expert Joseph Sirosh - ex- VP Amazon and Microsoft
2023 was a dramatic year for AI with the emergence of Gen AI and ChapGPT. Learn about the evolution of Generative AI from industry leader and expert Joseph Sirosh. Understand how enterprises will deploy Gen AI technology to make their workers more productive. Learn about how Joseph's new company CreatorsAGI will leverage Gen AI to enable creators and authors to interact in new ways with their audience.
Joseph is currently CEO of CreatorsAGI Inc, an AI startup dedicated to empowering creators with Artificial Generative Interaction capabilities. He has been an AI leader all of his career, and served as the Vice President of Alexa Shopping at Amazon.com, the Chief Technology Officer of Compass Inc, and as Microsoft’s Corporate Vice President of AI. During his tenure at Microsoft, he held product and engineering responsibility for all Azure AI, enterprise database, and big data products. He built Azure ML, the first Machine Learning as a Service platform in the cloud. Prior to Microsoft, Joseph spent nine years as a VP at Amazon where he managed the Global Inventory Platform for Amazon’s Consumer Business, and built the machine learning, data warehouse and Transaction Risk Management teams. Joseph holds a PhD in Neural Networks from the University of Texas at Austin. You can follow him on Twitter and read about what he finds exciting in AI on his substack, Generative AI substack.
Brought to you by TiE Seattle
Hosts: Shirish Nadkarni and Gowri Shankar
Producers: Minee Verma and Eesha Jain
YouTube Channel: https://www.youtube.com/@fromstartuptoexitpodcast
Welcome to the startup access podcast. I new World Class entrepreneurs and easy hard end stories and podcast has been brought to you by Type Seattle. TI is a global not process of focuses on the TIP of this process including the article and create editor. We encourage you to become a tie member so you can gain access to this great podcast. To become a member, please visit www.seattle.ti.org.
SPEAKER_02:Hello everybody. My name is Gabri Shankar. I'm uh here with my co-host Sharish Natkarni. And uh we are on with our next edition of our podcast. Uh I'm very excited about today's host uh and uh today's guest, I mean, but first let me introduce my co-host. Uh Shirish uh is uh serial entrepreneur and uh two-time author, so that makes him a serial author. Uh his two books, from Startup Exit and Winner Takes All, are available now for uh purchase wherever books are sold or wherever books are heard. Uh we also want to thank Ty Seattle for uh uh uh producing this podcast, and uh we hope you like and subscribe in all platforms you're hearing us. We are available in all of them. Sharesh, over to you.
SPEAKER_03:Thank you, Gabri. Um, really excited about today's uh podcast. Uh today we have uh um we're welcoming Joseph Sirosh, uh, who is really an industry authority among AMI. Uh Joseph is currently CEO of Creators at AGMI Inc., an AI startup dedicated to empowering creators with artificial generative introduction capabilities. He has been an AI leader for all of his career and has served as vice president of Legends Jumping at Amazon.com, the chief technology offer of uh Compass Inc. and as Microsoft's corporate vice president of AMI. So as you can see, he's very, very uh well versed uh in AI, and we are very pleased to welcome Joseph. Welcome, Joseph.
SPEAKER_01:Thank you, Shinesh. I'm really excited to be here.
SPEAKER_03:That's great. So, uh Joseph, um, as I just described, uh you have been an AI leader all of your career. Uh, you've also applied AI to various industries like retail and real estate. Um, can you talk about some of your key contributions and how you have applied AI to different industries?
SPEAKER_01:Absolutely. And Sri Rish, before I start, I can't wait to publish an AI on your books on the Creators AGI platform. Uh I feel that in the future, uh, authentic generative interactions by creators such as yourself, writers who have incredible thoughts expressed in the past in books, but in the future through an AI, I believe that is going to be very differentiated. So that's what I'm working on. So my company, Creators AGI, is about building a platform for creators and authors uh like yourself with incredible new thoughts to use as a publishing medium. So you can actually publish your thoughts, not just in paper and the printed word, but through an AI. And more about that later. Now, um over the last uh 30 years or so, I've been in the field of AI. So I feel uh somewhat senior, if you may uh uh call it that. I remember taking my first neural networks class back in 91 in UT Austin. Um, and uh the professor Joy Deep Khosh was on my uh thesis committee as well, and I've kept in touch. It's amazing how far it has come. And during my career, I first built uh neural networks for credit card fraud detection at a company in San Diego, Fair Isaac. That was the uh first commercialization of machine learning. Uh, this was 95. The company had gone IPO in 95, and my first job was as a machine learning scientist. And I remember putting in production a model at the Latin American bank uh to catch credit card fraud, and the very first day it stopped a million dollars of fraud. Uh, it was that impactful. And you had to be real-time in credit card fraud detection. So that's how, in the early days, uh, neural networks used to be. Um, I worked in that company for about nine years, worked on everything from search and retrieval to fraud detection and collections, joined Amazon later in 2004, December, and uh I was tasked with building the trust and safety platform using machine learning. And machine learning was critical because at the scale of Amazon doing fraud prevention, whether it be credit card fraud or marketplace fraud, where you know third-party sellers would come and sell fraudulent products, all of those had to be solved in a very scalable way. And neural networks and AI were the only way to actually make decisions on high-volume transaction flows in real time. So I built out all of that from scratch. Uh, and this was one of my scale contributions uh during the career. And um, when I went back to Amazon recently, back in September 22, they uh affectionately called me the grandfather of machine learning at Amazon. Uh and so sort of really kick-started a lot of the efforts, started the machine learning conference, and then I joined um Microsoft. You know, Satya had recruited me over, and I was very passionate about AI as a platform. And I built the perhaps the first uh machine learning as a service in the cloud on Azure. It was called Azure Machine Learning. It's launched in uh early 2014. Um, and then I continued there building big data and data platforms and was a C VP of AI. Lots of products launched at that time, including many of the AI products uh that uh in version three, version four, etc. now um were initiated at that time. Uh there was a product called Azure Cognitive Search, which brought machine learning understanding to search, enterprise search, and that now is Azure AI Search. Those are all examples. And so I felt very proud of what I got started at Microsoft with the team. Uh, obviously, the team did all of the heavy lifting there. Um, and I then got really excited about platforms. I mean, Shrish like you, I believe these platform flywheels uh marketplaces are an amazing phenomenon. I really believe that this decade of the 20s would be dominated by platforms. And I think that has actually happened. And so I joined uh uh Compass. And Compass was a platform, uh, a marketplace uh with real estate agents on uh one side and then buyers and sellers. And then the goal was to spin the flywheel on our platform. So all the transactions between agents and interactions between agents and buyers and sellers would go to the platform, and you would then connect in escrow and title and many of these other ecosystem players into it. So all of those happened in a platform. And during the uh 2018 to 2021 time frame, the company experienced hypergrowth. Uh, we went IP on NICE in 2021. And then after about a year, I returned back to Amazon. AI was all the rage. I ran Alexa Shopping. Uh Alexa Shopping was now introducing AI uh in a significant way, the large language models. Uh the shirt, I love LLMs, you know, uh is from Amazon. Uh and uh I was also uh leading the product and engineering for the Rufus chatbot, which Andy Jesse announced in the last earnings call, which is in beta right now. It is a chatbot for shopping on retail. So uh a bit of a long story, but it's been super fun uh in the world of AI, and it never ceases to amaze me how much it is changing.
SPEAKER_03:That's great. That's an incredible journey that you've gone through. So um, you know, 2023 saw some dramatic changes in AI with the emergence of Gen AI and Chat GPT. Um, what what changed? Uh is uh generative AI somehow uh different?
SPEAKER_01:Yeah, great question. So I I think there has been a dramatic change uh since the invention of Gen AI. And really the change uh was seeded by transformers, the algorithm called transformers invented in Google in 2017 by a small team of researchers. And uh people didn't quite realize it at the time, but that like really drove a big phase transition. Now, prior to that, AI was mostly discriminative, not generative. Discriminative meaning it could help you classify things. You took an input, it allowed you to classify things intelligently. It helped you forecast uh a time series, you could do regression. So these were the dominant paradigms. Nobody thought AI systems could be used to generate output, like in language. Nobody thought it could actually learn a model of the world that was high fidelity enough to generate images, to generate voice, to generate text that is incredibly fluent and start reasoning. And all of that really, you know, face shifted when transformers were invented. So the difference with Gen AI is for the first time, you have a very high-quality generative system. Until now, the only generative systems uh of any substance where humans, um, you could say creatures, you know, parrots are generative. So it was only living creatures that are generative that could actually generate. You could draw, or you could, of course, a computer simulation that you could call generative, but it was not, of course, intelligent generative. So that's a significant difference for the first time. You have the ability to generate output like living creatures.
SPEAKER_03:Got it. So um what made this all possible? I mean, we all heard of large language models. Um, and now large language models uh uh with smaller parameters have existed for some time. So, what is the uh, you know, you talked about transformers being an um a breakthrough uh technology. Um so uh maybe you can talk a little bit more about that and uh how that um how that transformation happened.
unknown:Yeah.
SPEAKER_01:Absolutely. Yeah, and I'm happy to. So in the past, large language models were simple statistical models. It said, hey, let's say you say three words, then what comes next? You can calculate a probability using standard probability calculations. And it just wouldn't do very well because uh word is dependent upon so much of the context on the language before it and world knowledge. Just to take an example, if I tell an LLM of today, how do you say banana in Spanish? Well, it has to answer with the word platana, which is the Spanish word. But to do that answer, you've got to first of all understand all of Spanish, all of English to translate. And then you have to understand that whole sentence is a question. How do you say banana in Spanish? And the right answer is Spanish. So the amount of learning that is required, the context that is required, is so massive, and there was no way a neural network in the past could learn Spanish, English, understand how to reason about that question, and generate that one-word Spanish answer. So when transformers were invented, first of all, there were two things that happened. One, it created an algorithm that you would dynamically pay attention to different parts of the text depending upon the sequence of words, and therefore know which of the context was most relevant to predicting the next word. So that was one invention. The second was the algorithm itself turned out to be so incredibly parallel and scalable that you could run it on these GPUs and these very large clusters of GPUs and scale it up as much as you wanted. So you could now have a neural network which had the capacity, the actual capacity to learn a world model, the capacity to learn Spanish and English and all other languages now. Now, OpenAI or Anthropic, I mean, they cover all the major languages. And it now had the ability, now you had a system that was scalable enough that just by throwing compute at it, now you could digest all of the world's online text. And in uh OpenAI, uh one of the scientists there had that uh gumption, so to speak, right? That thing that, hey, how about we actually go take on this task of taking all of the world's online text and just feed it through the transformer algorithm to predict the next word, right? So the problem was this predict the next word problem that they went after. A lot of others, by the way, even Google, they had this BERT algorithm that came about at the same time. They didn't go after the predict the next word problem. They said, given a sentence, let us try and model a missing word. It was called a masked language modeling. If the if a word was in the middle, you mask it, and from all the other words, try to predict that. Now, those algorithms were very good at modeling language and representing it, but it didn't actually quite have this ability to learn world models by predicting something. So that was a very interesting big step as well. So transformer algorithm gets invented in Google, it's very scalable, people start applying it and first to translation, and then uh in OpenAI, the GPT model, generative pre-trained transformer model that predicts the next word, gets developed and they just go after all of the world's online text. And then when they start generating text with that, it just sounds so real. And people are just amazed internally, right? And so they then went on to develop GPT-2. And when GPT-2 was launched, is when people really started waking up and thinking, oh, this next word prediction problem from you know online data, this has really got something going on that we didn't anticipate. Uh, they called it emergent behavior. Okay, we it was just not a single word prediction uh issue, but just by forcing a neural network to predict the next word, it was now forced to learn everything that you needed to learn, very sophisticated concepts. And that was just a mind-blowing thing, which is why you know LLMs in many ways are not an invention, they're a discovery. What do I mean by that, right? So if you when Galileo invented the telescope and looked out into space, he discovered Jupiter and the moons of Jupiter. Okay. Moons of Jupiter were not created by human beings, we just discovered them, discovered a phenomenon. And you know, that the same thing happened when Edwin Hubble discovered galaxies. In the same way, what just happened was the transformer is sort of the telescope. You know, you invented the algorithm, and suddenly you discover this incredible universe that just opened up of emergent behavior, ability to reason, ability to complete text, ability to now generate images and understand images and any input right now. And so that is the magic of Gen AI today, which is now you have a universal algorithm that's capable, apparently, of learning at scale, learning anything you throw at it. Uh now people are even talking about the online data may not be enough. We may run under the wall of not enough new data. Um it can learn anything, and then it can generate anything practically. It can generate voice, audio, music, um, images, text, DNA sequences, materials, uh, potentially even engineering structures, new algorithms. I mean, who knows?
SPEAKER_03:So uh you know, you talked about um GPT 2 and uh you know prior generations of uh GPT, uh, but it's only through uh when GPT 3.5 and 4.0 came out that you really saw this, you know, uh the world shift, and suddenly you had uh you know the ability to create chat GPT, which would generate realistic responses to any questions we asked. So um what changed uh between GPT 2 and GPT 3.5 and 4 that allowed it to be so much better?
SPEAKER_01:Yeah, uh great question, right? So uh in GPT 3, GPT 3 was a big step change as well because it was uh it went after dramatically larger data sets than GPT 2, so it really started performing well. Uh was eye-opening. I remember um you know that uh coming out in 2020 and getting super excited about it. Um so at that time uh people also understood that large language models uh really just modeled all of the data online and it didn't have alignment with what we liked. So if you told GPD-3 at that time without alignment to generate a recipe for uh poisonous chemical, it would do that. Uh if uh I told it, help me commit suicide, it would tell you that. And how do I build a bomb? It would tell you that. It could it could do all of those things. And people realized, by the way, that you needed to take these models and align it with what we as humans want. So this is like um a rob, uh a kid being born, and the kid uh is rowdy if you ask it to be rowdy and does all sorts of things, and you now have to shape its character so the kid grows up to be a good citizen in the real world. And so they came up with this algorithm called R Eleccia of Reinforcement Learning from Human Feedback, which really um uh uh was driven by a lot of humans creating course material, if you want to think about it, like questions, answers, how to respond, and then using that to really teach the algorithm on how to answer, how to respond to people, and how to have dialogue. And now the models GPT 3.5 and later are aligned. So they're aligned to human preferences, it knows how to talk to human beings and what to say and what not to say. And after building that, they uh OpenAI released ChatGPT. And when ChatGPT first came out, that was uh the first application that really exposed the magic of GPT 3.5 to everybody. Now, many of the researchers and scientists knew about it, and we were building uh applications with it, but until that ChatGPT um hit the ground, nobody realized how uh vast the scope of this generative AI was, all the tasks to which it could be applied. And that then Hawk is taken, and then GPT-4 came out, which was a dramatically larger model. Many people think it is not just one model, but a collection of about eight different models all uh put together called a mixture of experts, and each model having slightly different specialization, and its quality is amazingly good. So that's what's uh so different about the current generation GPD-3, GPT-4, and they are all very well aligned, so they don't hallucinate as much, meaning they don't just make up things. In the past, many of these uh AI models seem to make up things. GPT-4 is grounded and they don't seem to make up. Although you can still get it to make up things, you had to force it, but it it behaves well.
SPEAKER_03:All right. So um it's interesting that the transform model um uh uh algorithms were invented at uh Google, uh, but it was OpenAI that uh made the dramatic improvement. So I'm wondering uh why is uh uh OpenAI GPT-4 so much better? And what about Cloud 3 from Anthropic?
SPEAKER_01:Yeah. OpenAI hired uh some incredible researchers. Uh I think if you look at its founding, it was um Elon Musk, by the way, who founded it and recruited the whole team of Ilia and Greg Brockman and Sam Altman joined as well. Uh Co-founder and Ilya is one of the pioneers of uh large uh neural network models. Um and they uh uh had a really good team and they had this uh bold ambition to go after artificial generative intelligence, artificial general intelligence. Um we'll talk about that more, but that uh and they believed that scale was the key. And they had a conviction that more capacity on a simple algorithm and more data would yield more intelligence. And they pursued that relentlessly in a way that many other researchers didn't, and that really made the fundamental difference. And now it is almost common parlance. Everybody accepts that the capacity of the LLM is one of the most important things uh to drive more intelligent behavior, but uh at that time that wasn't obvious.
SPEAKER_03:Right. And um there are models like slot free that are um approaching uh the performance of uh GPT-4.
SPEAKER_01:Yes, uh the founder of Cloud uh or Anthropic is from OpenAI. He was one of that core team that built the GPT, and they've amassed a really good team of 30 uh scientists with physics backgrounds, etc., to build these LLMs. And the latest release from them, Cloud3, is very close to GPD 4. In some cases, it is better. Um, there is an IQ test I saw online that now people are administering to LLMs, believe it or not. And Cloud3 had uh an IQ of 110, I believe, whereas uh GPD 4 had an IQ just below 100, yeah, somewhere 80 to 100 range. So that that just a tongue-in-cheek demo, but it uh is catching up.
SPEAKER_03:Great. So let me now uh turn it over to Kavri to continue uh the discussion on um Gen AI and LLMs.
SPEAKER_02:Yeah, thanks, uh thanks, Shirish. So let's shift this conversation a bit, Joseph. Right. So you talked about um the evolution, right? Both personally for you and now um what the inventors have discovered, like you put it, right? So it seems like on the business side, it's uh it's just the large players actually can attack and solve it for now. Maybe others will come in and I I want to figure out how you fit in. But the oh the opening uh salvo was hit by the really large players of a lot of money. But the key the key uh ingredient that I want to explore with you on is Nvidia because they made an early bet in their GPU and they fuddled along, and as of uh uh the most recent reporting, they are the fourth uh most valuable company on the planet, right? Shifting at about uh or coming in at about two trillion. How did Nvidia change this growth uh for all the inventors who could build on top of Nvidia when they figured out scale was the only way to go? When they made that bet, they needed GPU or they needed compute. But and along came NVIDIA or they dragged NVIDIA along? Can you explore that history and the and the tech of NVIDIA a little bit?
SPEAKER_01:There is a really good question of what came first, uh chicken or egg. Uh and let me tell you a little bit of history. Nvidia made its bet on neural networks very early on in the 2010 time frame, even 2012. So the very first uh neural network, large deep neural network model that beat um uh the CVPR competition on image classification, uh, that was built on NVDA GPUs. So that was 2012, I believe, the paper came out. I don't remember exactly when the competition was. So uh in the late 2007-2008 time frame onward, there were efforts by researchers to use the capacity of GPUs to do parallel computing, a lot of parallel computing. And three independent researchers in neural networks field, um they in almost in simultaneously in parallel, um, started using GPUs to learn neural networks. And then there was this incredible eye-opening demo in the early 2010 to 2012 timeframe of image net classification, which really, really changed the trajectory of computer vision. Nvidia then took that game on, and it really credit to Jensen. Jensen has been a very visionary leader, and he pushed uh NVIDIA into cutting-edge areas constantly. And uh then he cultivated a relationship with OpenAI as well. And through the ensuing years, uh, from 2012 onwards, Jensen was relentlessly pushing GPUs and parallel computing in HPC scenarios, in supercomputing, in ray tracing at scale, and of course, neural networks and machine learning. And that then hockey sticked when uh GPT models were trained. Um, Google, of course, had that on NVDI chips quite a bit, and then they decided that uh, you know, those were those types of architectures were essential, and they built their own called a TPU. Uh, and then when OpenAI really built GPT models, then Microsoft also came along and built very large GPU clusters to train these models. They provided support, and all of this flywheel started going. And now Nvidia is a king of the hell, absolutely king of the hell, when it comes to training neural networks. Training is the specific time thing where you have to hold the entire neural network in memory, you have to pass enormous amounts of data through it. And when you look at the flops per unit of power, I think Nvidia has an incredible edge compared to any architecture that exists right now. Got it.
SPEAKER_02:So um, two then that leads to two-part question, right? Of course, there were other chip players who could have pushed along. Um they did or they didn't do it fast enough on the one side. But on the other side, um Google and Amazon and Meta had as much uh insight into it, I would argue much more than Microsoft because of what they did, right? Uh Microsoft is a if we classify them as the enterprise software company, largely what they are, compared to say an Amazon or a Meta or a Google, which every day uh interacts with just the average person. Uh how did Microsoft OpenAI combination along with Nvidia work? Uh versus any other three players could have worked. I mean, uh they could have all picked what what led uh Microsoft to make the bet because you were leading it. So in essence, you were making the bet on their behalf uh compared to anybody else.
SPEAKER_01:This is where individuals matter a lot. You know, Tratya's very first tweet, if you go back in Twitter history in 2009, was machine learning exclamation. And then he was a big fan of AI and machine learning for a very long time. So when he hired me, you know, that passion was very evident. Uh and Jensen really was very early to realize that deep neural networks were potentially a very important workload. And Elon Musk was very early to realize that open source AI or open AI was going to be very important because he was worried about Google's incredible advantage in AI internally in research. Uh and then Elon Musk and Satya happened to meet uh at a conference. And uh, by the way, uh Elon Musk had actually, when I was there itself, uh, really asked for an enormous amount of capacity in Azure Cloud for GPUs so that OpenAI could do its work. And Satya was very eager to support. Um, and then later, uh Sam Altman and Satya met at a conference and they kicked off the partnership. And Kevin Scott, the CTO of uh Microsoft AI, uh, had a big role to play in that as well, in facilitating that relationship to come together. Um, and so you see this incredible dynamic of certain very enlightened individuals tearing it, and they um they kicked things off. Uh and inside of OpenAI, Ilya, the scientist, was very responsible for leading the charge into very large-scale models. The yeah, Amode, the current CEO of Anthropic, was part of Ilya's team, who actually got the GPD models along with a couple of other scientists. Uh, and so you see the incredible transformative power of certain individuals taking certain bets, visionary bets, earlier, and those bets paying off. Now, of course, we don't see all the people who took visionary bets and didn't survive, right? But these survived. And so there's a bit of a survival bias here, I'm sure. But these folks are you know responsible for it going the way it did. Now, a couple of more other uh things. Meta, Mark Zuckerberg, uh, also in 2016 realized uh that AI was a very important thing to take a bet on. He went to the Neural Information Processing Systems Conference, the leading research conference in machine learning at the time, and appeared at the workshop and started pitching why Facebook is the best place for researchers to come together. He started the FAIR labs, the Facebook AI research labs in New York, headed by Jan Lakun, who is now a Turing award winner. Now, uh, Google, in the meantime, had hired another um of the leading light, Jeffrey Hinton, yeah, who's another Turing Award winner also for AI. Um, and Jeffrey Hinton and Ilya had worked together. So Google had that strength internally. Ilya had got left Google to join OpenAI, but Jeffrey Hinton remained. So these individuals uh really drove the thought leadership, and now what you're seeing is the flywheel effect of those bets that were taken early on.
SPEAKER_02:But looks like Nvidia um um you know went with OpenAI first, and now they're doing it with everybody. But the fact is they believed in the model so much because they had the lead in their thinking with their visionary CEO to say this is where the world is gonna go. Sure, they succeeded, but the fact is uh if you look back twenty ten years, I bet the market cap of NVIDIA was the market cap Nvidia moved much after their earnings in one day that their entire market cap in the history wasn't that much, you know. So it looks like uh they played a role every day along with all these individuals, because uh you know many, many bets have been played, but very few have succeeded, you know.
SPEAKER_01:Yeah, and that's true. Uh remember Nvidia as a hardware player had low margins and they were very volatile as well, because when Bitcoin was taking off, people were buying GPUs for Bitcoin, and then it started going down, and then the GPU market was not doing well. So if you look at the hardware makers' um life, because they don't have SaaS models, right? They've got to sell a hard every time. It's um you know, groundhog day every time. There's an enormous amount of volatility in the electronics market, which is why consumer electronics companies don't survive very well, right? Very hard. So Jensen was very oriented towards finding the next market all the time. And they now uh saw the demand from OpenAI and so ramped up the capacity, uh, etc., and started making bets, and those ended up aligning and paying off. Um, yeah, so but you know, kudos to Jensen. And so he's actually the longest serving tech CEO, I believe. He started the company in '93. Uh, and so is now uh 30 years into it. And so he was able to have the long-term thinking and take bets to create an outcome as opposed to be um just playing the uh market shop.
SPEAKER_02:So we shift, yeah, yeah. So kudos to that. Let me shift to another hardware player that uh we we should all be talking about is Apple. I mean, there's two billion plus devices out there, right? And um they are the interface of for the world. Um I'm sure they are working on it. We don't hear much, and like every Apple thing, you don't hear till they say, hey, by the way, here's another thing. But they've had this very close relationship with Google, and uh I'm sure they could develop such a relationship with with uh with uh Microsoft also, but Google is also has their own hardware, etc. It's not anymore just um, hey, I can power maps on Apple uh devices uh on iPhone. It's a lot more tricky. Where in your opinion is Apple and because you came from Alexa uh most recently, and Apple has a their version of they have Siri. So where do you see them going um in the gen AI path?
SPEAKER_01:I think Apple uh will nail edge inference. So on the phone, the biggest opportunity is to run the models, not to train the models, to run the models. Already there are very large uh neural network models on the phone for photo processing, uh, and they clean up the images we take without us even knowing that's happening. So I think Apple has this incredible opportunity to go run LLMs, or at that point there might be SLMs, small language models, but optimized for the applications. And I think they are deliberately working towards making that happen because they have this incredible platform called the iPhone and its compute capable capacity. The iPhone's compute capacity is just massive uh compared to anything that you know we are aware of. So I think that's possible. I think we will probably see the inference at the edge being exposed as a platform to developers, app developers. That's what I would do if I were them. Certainly, Siri would improve. And Siri has been, as uh all of you know, an unfortunate disappointment uh for a long period of time. Uh and not just Siri, Google Assistant also. And a part of the reason is it really does take a large language model to be able to do these things well. And large language models are large, so they have to be hosted from the cloud and rendered through an API, but then you can't run it on the phone at the real-time speed you need. I think those things are actually improving. So I wouldn't be surprised if Apple announced um an LLM uh-based approach to CD in the upcoming WWDC. I think they could launch a few uh capabilities for Gen AI for developers. I think they will probably have a new chip, a new AI chip for inferencing uh on the edge. And I think uh from then on, I think it's it's their game. I think I think these chips can be used uh uh for edge processing, potentially down the line, even in an Apple Watch. So it would be very interesting.
SPEAKER_02:We shift to a quick uh another area which is of big interest to everybody, is uh AI and enterprises, right? Everybody in the enterprises allocating budgets or at least uh allocating resources uh in one one of these two. But eventually they have to play with the large LLMs. No enterprise can at the moment can can invest as much as open AI and catch up soon. Maybe they they could, they could, banking and others could. But with your time and compass, and when that was what one of the opportunities that you might have seen of applying AI into probably, you know, if you take one, two, and three, defense, healthcare, then real estate as the top three spends of the United States. How do you see enterprises applying AI given the state of what AI is now, to where they need to go to justify to their own boards the investment versus returns?
unknown:Yeah.
SPEAKER_01:Fantastic question. Let's break that problem down a little bit into use cases and how it can be used. Um so Gen AI as it stands today is great for interactive communication with clients, with uh customers, with patients, um, all of the uh code generation and so on, right? So I actually think where you have those types of opportunities, you'll see adoption first. Now, Clarner recently announced they had replaced uh 700 customer service representatives or so with uh Gen AI and OpenAI-based model. Uh, that I think is just a tip of the iceberg of what is possible. I think um these models have turned out to be much more effective at communication, and there are papers that show the output of Chat GPT or GPT-4 is more empathetic than the way doctors communicate. So when a doctor communicates to a patient, some doctors can be rather short, you know, not very communicative, may not be as empathetic, whereas a large language model can smooth all that over. So in healthcare, there's an enormous opportunity where patient communication is involved. Doctors writing notes are involved, doctors have a huge burden of notes and managing medical information. Um, AI is actually turned out in some examples to be uh very good at diagnosis because it integrates enormous amounts of information and much more than any human doctor can hold in their head. Uh, and it can potentially reduce errors of all types in diagnosis through that. So those are all examples of applications in healthcare. In uh cybersecurity, it's actually helping uh in threat detection and threat management and mining logs. When you have a very large number of log entries happening, how do you identify that? Um, I think, of course, coding, and I'm sure we'll talk about it uh more later. Uh in every type of coding, whether it be user interfaces, uh backend coding, diagnosis of errors, uh, log mining, all of those, data science, all of those uh is being transformed by AI. Uh, and of course, in defense, uh, everything that we can imagine in terms of drones or automated control of various types are all going to be affected by this revolution in AI as well.
SPEAKER_02:So that sort of leads me to this question, right? The governments world over are um somewhere between concerned, paranoid, and scared, right? Some combination of them. And you uh at your time at both Microsoft and Amazon, you dealt with a lot of world leaders, big and small, doesn't matter. And they all have the same level of paranoia. On the one hand, they're worried what's going to happen to their own citizens because they could be replaced, like Karna said, hey, I've replaced 700 people at just flip of a switch, that it takes a while to replace them. While on the other hand, they're worried about defense and are we being threatened and taken over, etc. Right? Um how does how does uh governments react or even interact with uh with AI, which is moving at a pace that has never been seen in the history of humans? Because this is the first technology that was democratized on the day the light switch went on. It's not like hey, we had an iPhone 2007, okay, by the time say India got it was 2020. No, India got it on the same day. You and I could write a poem on on Chat GPT. I don't mean to pick an idea, you know what I mean. So the governments play a very important role. Uh and are they uh worried beyond uh anything? Uh are they running scared, or where what's your take on it?
SPEAKER_01:Yeah, I think it's a great question. I think uh governments are caught unawares. The one technology I can think of similar is the internet, and there was a lot of uh lot of thinking in the earlier days of the internet and worldwide web. How would governments own uh control over the internet? Or how would they uh you know manage the internet in their country when the technology is invented somewhere else and that is actually dominated by other countries? And I think that becomes a really uh pronounced issue now with large language models in that they have to run on a cloud. And they can't, they're not open source software, they're not things that you just download and use, and training it is very hard. Most countries don't have the ability to train large language models, and so also it is now dominated by English, and the few companies in the world now completely dominate large language models, right? That's uh Microsoft, OpenAI, Google, uh Facebook. Um, I mean, these are all companies, by the way. If you think about it as a country uh outside of the US, some set of companies in Silicon Valley, I mean, completely dominate some of the most important technologies of the next 10 years. And I think that's a uh a real concern. So uh countries are uh investing in it, and Jensen is happy to sell them GPUs uh and uh encourage uh sovereign AI. Uh, I think the lack of um specialized talent is uh is a big challenge in that not not everybody can build a very large language model because there is a lot of intricacy to that. Yes, so I think that's where we are. Uh I think there are concerns. I think um interestingly, I see people being very opportunistic in the Gulf countries, UAE, etc., are now leading efforts to create data centers right next to oil fields with very cheap energy. So you can actually pipe intelligence out instead of piping oil out. And it's a really interesting thing. Uh, right? Like, you know, if you think there's actually a really interesting uh analogy here. You know, until now, the primary uh currency of civilization was energy. Uh and with almost no cap. Essentially, you could say into a civilization, pour more cheap energy, and the economy will move. Okay? And there's almost as though it's almost as though humans can use as much energy as you give to them. Okay, of course, that's the economic capacity. There's never been anything like that since the time of energy. Okay. So if you were to create more CPUs, I'm just taking examples as a technology. I mean, only so many CPUs the world uh needs and the world can take. But the world can take unlimited energy, right? But like we're getting to the point where the world can actually take unlimited intelligence. I mean, like you know, anybody can get more intelligence, meaning, you know, like chat GPD or you can use it for every single thing. It can be healthcare, it can be science, it can be engineering, maybe, you know, uh art, art, movies, music. So there's this new technology where intelligence becomes like energy in that it's a commodity that can be now built, piped, and consumed. Um, and so some of that is now happening uh with uh countries with oil uh resources or cheap energy, uh, taking this back to fund massive data centers that are now powered by cheap energy. Instead of taking oil out of the ground, refining it into all sorts of things, shipping it all over the world and these you know, moving atoms. Now they move oil atoms or molecules out of the ground, convert them to electrons in a data center that represent intelligence and pipe it out through the internet.
SPEAKER_02:That's uh that's probably the first time I've heard how uh how atoms move to electrons in a very different fashion than in my earlier physics lessons.
SPEAKER_01:Yeah. And electrons that are now much more useful than uh you know atoms and molecules and immediately useable. Yeah, so I think it's a very different type of economy that we are looking at in the next uh two decades.
SPEAKER_02:You started uh to build on that, you started your this interview with using a very interesting uh phrase, authentic uh you know, uh generative.
SPEAKER_01:Generative intelligence, generative interaction. AGI, but yeah with the player.
SPEAKER_02:Yeah. So let's sort of dig into that, right? Because uh the reason the governments are worried is that the intelligence that this model has can be exploited by a few to gain uh enormous uh you know ability to change the way the world thinks, etc., and so on and so forth. But you have now taken on the responsibility to say, hey, this actually can be harnessed for an authentic uh interaction. A, how did you come up with it and then elaborate on what led to the startup from that little thread of idea that you had?
SPEAKER_01:Yeah, great question. I felt that generative AI is going to create so much uh commoditization of what is seen as human creativity. In new images, you know, yeah, you can't sometimes in at first glance tell whether an article is written by AI or a human. And at that point, I felt that the creator actually matters. So people will start uh uh really trusting the author they like, whose wisdom they have read before. And a human being standing behind the AI comes across as more authentic, even if they use AI, because they are now standing behind it and are providing this trust factor, which is even if the AI is generating something, that's something that I approve. And if it is not generating the right thing, I'm gonna curate it. And so you can trust in the output that you're getting from my AI, right? I thought that was really important. I think I felt that the humans are more differentiated in the world of highly democratized AI generation. So I wanted to make that happen. Uh, and uh so that generative uh interactions uh are going to be the currency of the future, by the way. Meaning like everything that we do on a computer is going to have AI in the middle. So the interactions you're having, right? All the conversations are going to be mediated by AI in one way or the other. And so now AI becomes, in my mind, a new format, just like video became a format, right? Streaming video created YouTube, and people now created YouTube videos, which are definitely designed for YouTube, very very different from movie making in the past. And so now that AI becomes a format, I believe now creators have the opportunity to create outputs that take advantage of all this power of Gen AI. So it can now extrapolate the creator's thoughts in a way that the creator likes and is blessed. Um, the creator can in themselves expand their horizons. I was speaking to yesterday to a very prolific Substack author who said Gen AI has really hundredxed his productivity because not only you know he goes for a walk in the morning or run in the morning, he has you know four ideas at that time. He speaks them out aloud, and it gets captured, translated by an AI into notes, and then he takes those notes and puts that into uh Chat GPT and asks it to brainstorm and expand on those ideas, and he just gets so much creative expansion of his thought that he can now be extremely productive. How do you bring all of that power on a platform? How do you platformize that? That was really my thinking. So bring, first of all, authenticity of the creator, allow the creator to create new universes powered by AI, and then communicate in an effective way where the audience now gets 24 by 7 ability to interact with the author's thoughts and representations and feels that sense of authenticity and in any language, right? And that's what I ended up uh imagining. And and hopefully we'll launch in the you know small number of months uh in front of us.
SPEAKER_02:So of all the ideas that you could have pursued, Joseph in Gen AI, right, you pick this one because you must have felt a sense of commitment and mission to this idea, knowing fully well that you're taking on your previous employers who could essentially do something like this if they choose to. Because they have for the sake of the discussion, they have unlimited resources as you mentioned. Why this mission? And why did you say that I gotta start with creators? You could have started with many others, you could have started with GitHub developers. So uh I mean developers who are on GitHub. But you picked very specific mission with a very specific uh value proposition, and what led to that uh specifically. You could have done anything at the uh when you left. Yeah.
SPEAKER_01:It's a great question. Yeah, it it to it was that sense of uh mission and purpose in that I believe the human imagination is incredibly powerful. Now, certainly software developers are imagining things, so and that's a now it feels like a well-served market. But I feel that authors, writers, and scientists, and I'll be happy to serve them at the right time and the platform is mature. The people who have original thoughts, I think amplifying them is one of the most important missions that we can go after. This is not about ads, uh, this is not about just generic TikTok like entertainment. I'm really going after people who have original thoughts and people who curate original thoughts and who shape the very fabric of our civilization. And I think that matters. Um, and uh let me allow me to expand on it a little bit. Uh, the famous author, you all know Ahara, talks about how stories are the foundation of our civilization. Completely. Everything that we believe in, we operate in, our legal system is a story, um, religion is a story. Everything that we believe, uh, the founding principles of the country, the constitution, every single thing is a story. We created, we human beings came up with a story to live by. And those stories are these original thoughts that define civilization itself. And the humans who come up with those original thoughts, I think they can be amplified by AI like no other. And those thoughts matter, and those thoughts can be not just amplified in terms of the production, but communicated to people who want to get that original thinking in very powerful ways with AI. And I felt that was just an amazing mission. It is like the early mission of Google to organize the world's knowledge. At that time, maybe people didn't fully understand how deep that can actually be. And I felt a sense of purpose with this mission, and I felt there is tremendous depth in this mission, and I think tremendous room for innovation in that mission. And so I felt really, really compelled to go pursue that. And although it's not obvious in day one that there is a business model that is easy or anything around it, right? The enterprise business models at Microsoft and Amazon are very easy to understand, and there's a way to grow. But I'm not going after making uh a quick business come together, but after creating uh but I'm going after creating a specific direction of innovation that I think is well worth pursuing.
SPEAKER_02:So the the um tool sets that you will initially come out with for the creators, etc., right? Similar uh tool sets are being created by others, like there's a GitHub co-pilot or a wing co-pilot, maybe it's branded differently. But they've all had mixed reviews. What is it that you could share with our audience that that you could see coming out with that will elevate you or differentiate you from all the others? Because uh the reference point right now is co-pilot, right? For good or bad, it is it is what it is. But now you and there's a ton of other tools, especially in image creation, etc. There's lots of tools. Uh where where do you see fitting in into that uh schema, so to speak?
SPEAKER_01:Yeah, great question. And I'll I'll tease a few things, although I can't share a lot, right? I think perhaps the best analogy in my version 1.0 is to GPT store in Chat GPT. So in a GPT store, you uh anyone who wants to create a GPT can come in and upload some content and create a GPT that has a specific kind of behavior and you use it within Chat GPT, right? And it has some knowledge that you can attach and so on. So imagine that that concept, but then you platformize it uh so that any author, uh, for example, Sharish, could come in and upload his blogs and his books and his writings and his wisdom into that platform and then generate an AI in his image, so to speak, right? Which represents him and his thinking as closely as possible. And it feels authentic to what Shirish would write, uh authentic to his originality. And you know, people would through the interaction feel that sense uh from Shrij, as opposed to going to Chat GPT and getting a generic uh outcome. You go to a co-pilot, you get a generic outcome. And what LLMs do, uh Gauri, is they average across everything they're pre-trained on. See, they're trained on the world's knowledge. And there may be a thousand experts in any particular field. It sort of averages all of that together and generates an output. Whereas here, the outcome is a specific sharp point of view. And Shirich, uh, to take an example, may disagree with what ChatGPT generates. He would say that is not how marketplaces work, for example. Okay, and say the most important things, in his opinion, are X, Y, and Z. And the AI should express X, Y, and Z, not the generic answer from Chat GPD, right? It's not the world-understood answer from Chat GPT. I think that matters, right? Every creator, every original thinker, originality is different from the average. That's how original thinkers stand out. It's not everybody's known wisdom average together, but they're thinking different. And so this is the part of the authentic generative interaction, which is the ability to think different, but the AI to represent something different and authentic to that creator that they want to communicate. And that, by the way, is not indexed at all today in any co-pilot. No co-pilot, no chat GPT, nobody says that their goal is to enable someone to be different. Their goal actually is perhaps opposite. Their goal is to be aligned, okay, meaning aligned with what human preferences are, what people would like to hear.
SPEAKER_02:Not definitely so does this open up opportunities, say, for enterprises? Uh let's say they have an earnings call, they publish a ton of um performance indicators in form of uh, you know, um either government mandated or their own uh everyday uh releases to the audience, right? Other than the few who are very interested in, say, that enterprise's uh performance, in this case and up uh their earnings, the rest just kind of listen to it and move on. Now, are you seeing that the narrative in this case Sherish uh or in another case an enterprise, or in another case, let's say a doctor, could be authentic about their narrative or what they intended, as opposed to the LLM saying this is the most likely in intention of the author.
SPEAKER_01:Yeah, very good question. I think there are innumerable use cases like that. So this is why I said AI is a format. Okay. So today, investor relations on public companies use PDF as the format. And it's a static presentation and a static set of numbers you can browse through. Uh now, what if instead of PDF as a format, you put out an AI that is actually very curated by that company? So it answers the investor questions in a very accessible way. And where they don't want to answer, it's politely declines, right? And politely steers away from the type of questions that they are not prepared to answer, also, uh, which are legitimate use cases, right? And so this is um again the the thing that I want to accomplish is an authentic AI that the creator feels represents them authentically. It can the creator can be a company, the creator can be the marketing department of a company, the come with the creator can be an author, a movie. Um it can be a doctor. In fact, I have a small pilot with the doctor uh in process right now. And it's it's a very, very interesting um and and remember this thing about being authentic means it being allowing yourself to be different from what an LLM produces. And I think I've not heard anyone say that.
SPEAKER_02:I would completely agree. I don't think authentic. I mean, Gen AI has conjured up the idea that it is the least authentic, whether it's right or wrong, it's given that yeah, it's uh so it's the least authentic.
SPEAKER_01:Yeah, right. It allows you to dream, right? I mean, it hallucinates and dream. I mean, I was somebody was saying that all outputs of LLM are hallucinations.
SPEAKER_02:Very well said, very well said. So this sort of leads to this question, right? Everybody is fearful that AI is going to eat their jobs for breakfast. Then what do we do for the rest of the day for uh our livelihood? You have spent enormous amounts of time in your career uh, you know, uh in large companies where optimization of of uh jobs or optimization of productivity has been a core of their existence. And do you see this uh as a real thing that people have to worry about to people say, I don't want to be a developer, I I can just get Gen AI to write the code? Or how does how does the fear of every new technology has its fear, right? I'm sure when the wheel came about, people worried about it and the car came about, people worried about so on and so forth, you can trace it all the way back. How do you, as the leader who has spent that kind of time you've spent, uh go out and tell the public, no, this will actually make your lives better, not just take your job. I'm sure there's some uh jobs that are going to be lost, but that aside.
SPEAKER_01:I think uh no technology has been democratized as fast as AI, also. So Chat GPT is a great example. I think uh I find that people who are not very tech-savvy. And actually some of the deeper adopters of Chat GPD, small businesses, people who feel constrained, people who feel they can't afford uh you know certain services, and so many things, right? So I do think that there is an incredible democratization happening of intelligence, democratization of intelligence uh and access to it. Uh, and that I think is very, very positive in the short term. Now it causes displacement, right? Uh displacement doesn't mean elimination of jobs, it just changes everything, right? It's everybody now ends up being forced to adapt very, very quickly. Now we went through this with the internet era at some point. This is just happening at an accelerated pace. It's 10x faster than internet. So um I think I would essentially uh say everyone who wants to be entrepreneurial and entrepreneur in the broader sense, not about starting companies, just you know, uh, yeah, people who have this curiosity, the ability to engage with new technology and use it and all of that. I think they're going to be so much more productive and uh they're going to really amplify the economy. I think in the next five years, we will see uh uh an expansion of the pie um in in incredibly powerful ways that I think will lift all the boats. And then after that, it will be about how we deal with the displacement and changes that are happening as a civilization and as a society. So, what happens in the world is when lots of people feel just anxious, it leads to political change, unfortunately, populism, all kinds of things. I think like this is going to be some of the biggest challenges the world will have to confront because a very, very large number of people will just feel uncomfortable because of the change. There was a book I remember reading in my teenage years by Alvin Toffler called Future Shock. And this is future shock happening in spades, uh, especially in the next five years. And so that is the more disruptive uh outcome, and it's less.
SPEAKER_02:I haven't had anybody recall Alvin Toffler, just puts me in in an age category that that that makes me feel a little old. That is right. Democratizing intelligence. That I think is the is uh our uh you know headline for this podcast. Joseph, thank you very, very much for this very insightful, very, very insightful uh interview. And I think uh we may have to have you back at the end of the year so that we capture everything you said and then go back and say, where do you think this is going? And then and hopefully when you launch, you would come back on our podcast and announce the launch on our podcast. So look forward to it. Thank you very much, Joseph, for this uh great uh great hour we spent together. Yeah.
unknown:Yeah.
SPEAKER_03:Thank you very much. This was an amazing session and look forward to speaking with you again.
SPEAKER_02:Thank you for listening to our podcast from Startup to Exit brought to you by DICE Apple. Assisting in production today are Isha Gen and Mini Verbal. Please subscribe to our podcast and rate our podcast wherever you listen to them. Hope you enjoyed it.
