This week’s podcast is about two interesting parts of the new GenAI Tech Stack.
You can listen to this podcast here, which has the slides and graphics mentioned. Also available at iTunes and Google Podcasts.
Here is the link to the TechMoat Consulting.
——-
I write, speak and consult about how to win (and not lose) in digital strategy and transformation.
I am the founder of TechMoat Consulting, a boutique consulting firm that helps retailers, brands, and technology companies exploit digital change to grow faster, innovate better and build digital moats. Get in touch here.
My book series Moats and Marathons is one-of-a-kind framework for building and measuring competitive advantages in digital businesses.
This content (articles, podcasts, website info) is not investment, legal or tax advice. The information and opinions from me and any guests may be incorrect. The numbers and information may be wrong. The views expressed may no longer be relevant or accurate. This is not investment advice. Investing is risky. Do your own research.
This week’s podcast is about two interesting parts of the new GenAI Tech Stack.
You can listen to this podcast here, which has the slides and graphics mentioned. Also available at iTunes and Google Podcasts.
Here is the link to the TechMoat Consulting.
——-
I write, speak and consult about how to win (and not lose) in digital strategy and transformation.
I am the founder of TechMoat Consulting, a boutique consulting firm that helps retailers, brands, and technology companies exploit digital change to grow faster, innovate better and build digital moats. Get in touch here.
My book series Moats and Marathons is one-of-a-kind framework for building and measuring competitive advantages in digital businesses.
This content (articles, podcasts, website info) is not investment, legal or tax advice. The information and opinions from me and any guests may be incorrect. The numbers and information may be wrong. The views expressed may no longer be relevant or accurate. This is not investment advice. Investing is risky. Do your own research.
Welcome, welcome everybody. My name is Jeff Towson and this is the Tech Strategy podcast from TechMoat Consulting. And the topic for today, watch out for Huawei in Gen AI data architecture and watch for Baidu doing agent builders. So yes, another sort of overly long title, but really I just wanted to talk about two things. And this really follows on the last podcast. And if you're a subscriber, I sent you some really sort of in-depth emails this week about the generative AI tech stack, specifically the one at Huawei. I've talked about others, but Huawei really has great detail. And I mean, I really sent you a long email, and there must have been 10 slides there with the architecture for it. I know it was a lot, but it's kind of what I use as my little go-to reference. So I thought I would do that. And I was-- I kind of wanted to pull that up a little bit and say, look, okay, the generative AI tech stack, very important. Let's talk about two aspects of it that are very different that we don't see in other tech stacks. And I think it's stuff that's not really being talked about very much. And one of them is the data architecture layer, which I think Huawei is arguably the leader. And then it's this idea of agent builders and other new types of toolkits for developers, which Baidu has been talking about really in the last month at their big conference. So I want to kind of touch on those as examples and review a little bit the the tech stack because it is kind of important. Okay, and let's see standard disclaimer, nothing in this podcast from my writing website is investment advice, the numbers and information from me and any guests may be incorrect. If you use an opinion to express may no longer be relevant or accurate. Overall investing is risky. This is not investment legal or tax advice. Do your own research. And let's get into the topic. Okay, there's no real concept for today. Just the AI tech stack, which is an important subject, but not a digital concept per se. Really, we're talking about is generative AI. I mean, you kind of say AI tech stack for short, but really, you know, we're talking about generative AI, which is a different animal and very different than sort of traditional predictive AI, which has been going on for 20 years. Now, this is, I'd just say AI tech stack, but it's generative AI. Okay, so, now the Huawei version of this, which I think is really the best one. It's really three things. It's Pangu, which is their foundation models. It's a lot of hardware that sits underneath that. And then there's a lot of partnerships that sort of sit on top of it. That's businesses customizing its developers making their own apps that run on top of this and it's all of that. So that's kind of how I think about it is you know the hardware is really where Huawei is coming from. I mean they are a hardware company they're an equipment manufacturer at heart they've always been that and they started with the connectivity piece, 3G, 4G, 5G. They moved downstream into edge computing. So anything smart devices, smartphones, IoT devices, all of that, that's actually where their operating system came out of, Harmony OS, which is now, you should really pay attention to HarmonyOS. It has basically finally reached critical mass as a mobile, a smartphone operating system. That is the first serious rival to Android. I mean, it's got enough usage in China, 15 to 20%, something like that, that, yeah, it seems to have broken into their duopoly. But that didn't start out as a smartphone operating system. That started out as an IoT operating system for cameras and sensors and things because they were working on edge computing and they were sort of doing it for there. And then they adapted it to smartphones after they got hit with the entity list band. OK, so they did connectivity for G5G hardware. Overwhelmingly, they moved downstream into edge devices, again, mostly hardware, although they did Lanch Harmony OS. And then they have moved upstream into cloud computing. And that's really the solution they provide is an integrated end-to-end solution for telecommunications companies, but also for enterprises. They have a big enterprise business, but they're really offering an end-to-end solution that others don't offer, end-to-end solution that others don't offer, which is edge device connectivity and cloud. However, within that they have always been mostly hardware. So when they're talking about cloud services, if you actually look at what they've been doing, especially across Asia, outside of China, it was a lot of data centers, a lot of data center projects. So a lot of hardware stuff. And they do sell these massive data centers that they can put into place like places like Thailand, where they've built two, I believe, one for sure, maybe two. So hardware more focused, but they've been moving into becoming a software company over time. And that's really when they talk about their AI tech stack. Usually they start by talking about, okay, the base layer of the AI tech stack is gonna be compute power, right? Because generative AI takes a huge amount of computational power and it takes different chips. If you want better efficiencies, you know, go from CPUs to GPUs to NPUs, really different data centers. So they talk a lot about that and they have these big servers. They're announcing all the time. Fine. That's kind of your base layer is compute power. And then the next level up is you get to the foundation models. Okay. And the next level up is you get to the foundation models. Okay, for Huawei, that's Pangu, lots of different foundation models, generative image, large language, multi-modal, and those keep expanding. Open source, which is important. Who's building the big foundation models? That's kind of turned into an interesting question where, you know, most of the China players, especially Huawei, are open source. Although, by do, I thought they were, but Robin Lee has come out and basically said, it looks like most of the value creation is going to have to be proprietary, not open source foundation models for them to work well and get the efficiencies you need. They're going to have to be closed, which was interesting because I thought they were going the other direction. Open AI is closed, although even it's called open AI, it's all proprietary. Facebook meta has turned into the big source of open AI, llama and all that. So Facebook has turned into what open AI was supposed to be before they changed their approach, but kept the name. That's kind of interesting in Google. So it's an interesting mix of the major players all doing that open versus closed. Okay, Huawei, we're talking about Pangu. And they call that their L0 model and then they move up to L1 models which are customized for industry. So they have a transportation one, they have a finance one, they have a healthcare one. Baidu has pretty much the same approach. And then you move up to scenario specific, which is L2. That's when you're making it for specific situations like a chat bot, customer service, processing insurance claims, all of that. So L0, L1, L2, and then lots of lots of apps being built on that by developers, by companies, and so on. Okay, that's pretty much what you hear from Baidu as well. It's pretty much what you hear from Alibaba Cloud as well. However, and I'll get to the point now. Huawei is doing something very interesting at the data infrastructure layer. They've basically argued, and I believe it, that there's actually another level in this tech stack, which is the data architecture, which has to be for generative AI to work, it has to be different than the data architecture everyone is used to. And they actually wrote a fairly fantastic paper about a year ago they published it, 60 pages or so, laying out what next generation data architecture is going to look like. Very interesting, and I'm totally convinced this is a separate layer in the tech stack. And actually when you listen to OpenAI, Sam Altman and them, you can hear them increasingly shifting their discussion from we've got to have more and more compute power. We've got to have more and more compute power. We've got to have more and more parameters in our foundation model to, it's about getting the right data. And the argument is basically like, if you wanted to make your foundation model smarter than another foundation model, the big lever you would pull was more compute power. We have more NVIDIA chips. Our GPT-4 is run on far more parameters than GPT-3. The way you got smarter was to increase the raw computing power of your model. And it worked. That did actually make a big difference. But there is at least an argument coming from people who build these things that that's only going to take you a little bit further. Where you're going to start seeing diminishing returns by increasing the raw computing power in terms of intelligence. And the next wave of intelligence that's going to jump up is going to become from the scope and the quality of the data you can access in real time and in training. So it becomes a shift from CPU power to data in terms of the next generation of models. I don't know if that's true, but the people I think really know this stuff, that's what they seem to be saying. So we'll see. So let me talk to you about and this will be, I'm going to make two points for today. This will be point number one. How does Huawei sort of talk about the next generation of data architecture? Now, a year ago, Huawei wrote a paper and talked about, they had a big conference. It was really great in Shanghai. And they talked about the challenges of AI, generative AI. And they said, you know, computing power was a big one. Algorithms were a big one that you had to go from general foundation models to industry specific models to get better and better. And then they talked about application deployment. For this to work, we need people to start deploying these things into industry because that's going to generate the data that creates a feedback loop and makes the next generation of models smarter. And I talked about this recently with the Generative AI flywheel, which is kind of Baidu's primary strategy. You can look that up if you're curious, but basically, you know, those are three of the problems they talked about in terms of building AI capabilities. The fourth one was data. Data was sort of going to be the big bottleneck after computing power. It's going to be data. And that can be data from users deploying these things, businesses. It can be from sort of, let's call it, intelligent perception, where we're getting huge amounts of unstructured data flowing in because you've put cameras everywhere now, and you've put cameras everywhere now and you've put sensors everywhere now and you've got digital twins of every factory and every Transportation network for a city and every banking sector all of that is going to throw off a huge amount of mostly unstructured data And the way that data is going to have to flow is very different than when we were talking about traditional AI. Traditional AI, traditional enterprises, you know, when we talk about the data architecture for that, we're mostly talking about companies building ERP systems in their business starting to gather their operational data in their digital core, starting to use that data to run analysis of how well their factories are doing, maybe starting to pull in some data from their supply chain, where their suppliers and vendors are now connected and the data is flowing in, and maybe starting to gather data from their usage of customers by customers. So if you have smart refrigerators that is now sending in data, if you're Geely or higher, that is flowing into your system, maybe if you have smart cars or smartphones phones or anything that's a smart device or a data service you're doing, all that data is flowing in. But traditionally, that is what we're talking about when we're talking about the data layer for enterprise computing, digital cores, digital transformation, and also for traditional AI, where that data, which is mostly internal, is being gathered, it's mostly structured data, it's being put into data warehouses, and we're starting to run algorithms against that, and that would be sort of the traditional data architecture 1995 to 2010. Now 2010 to 2020, we start to see traditional AI really move forward, but we're really talking about the same type of data architecture. A lot of internal data that we're gathering from a company, maybe some targeted external data like from our suppliers, like from a handful of partners, and like from our customers, that would flow in. And now we're starting to do predictive AI on that. And okay, it's not all structured data anymore. It's not all stuff that sits in spreadsheets and, you know, relational databases. Suddenly, okay, maybe it's not unstructured data, but it's sort of quasi or semi structured data. Where yeah, we're getting some stuff. And that's when people stop talking about data warehouses, which are very some stuff and that's when people stop talking about data warehouses Which are very organized and they start talking about data lakes Okay, now we've got two systems running in Glee or higher Circuit 2015 we've got our traditional data warehouses and we've got some data lakes and We're running different algorithms against both. Now that kind of gets you to 2015. Okay, but then we move to generative AI, and the data architecture's different. We are doing not just unstructured data, video, images, text, messy data. It's not just unstructured, but it is flowing in massive amounts from all over the place, not just from the enterprise at all. It is flowing from the entire ecosystem. And we have to be able to access, categorize, and process that data in almost real time for the algorithms to make predictions in real time about we think the Tesla should go, we think it should change lanes right now. Well, for the Tesla to make that decision going down the highway at 60 miles an hour, that we should change lanes and get off, you have to have a massive amount of data flowing into the system in real time. Most of that's gonna be in the car, which would be your edge device. Some of it's gonna be be in the cloud and we're going to have to have 5G at least connecting the car to the cloud to make a real-time decision to change lanes. So that data architecture looks completely different than what we're talking about. And Huawei talks about this as what they call intelligent computing, intelligent perception, and intelligent connectivity. Okay, they kind of put intelligent in front of everything. But the idea that we're going to have real-time perception coming from sensors, from inputs that is going to flow in real-time up to the cloud that requires the intelligent connectivity part. And then we're going to make real time decisions somewhat in the cloud, somewhat in the car. And then it's going to flow down, and you're going to make real time decisions. That's a totally different data architecture. And it's really different in two ways. Number one, it is no longer just about data that is mostly inside the enterprise. It is going to have to flow across the industry. We're going to have to see transportation data in Shanghai, flowing between the streets, the cameras, the traffic lights, the parking lots, all the vehicles on the street, all of it's going to have to flow in real time. And Shanghai, the municipal government, has actually set up a data bureau for the city that is in charge of gathering and setting the rules and standards so that data can flow throughout the city in certain areas, including transportation. And everyone who plugs into that data layer can access all that in real time. So your car, if you're driving and being in Shanghai plugs into the layer, it can see everything in the city like Godlike vision in real time. And it's the government bureau that sends that basically sets the standard for how the data will be structured, how it will flow, what is the security, who can have access, who can't have access. That whole data layer looks very, very different in terms of the fact that it's mostly outside of an enterprise. And the other way it's different is it's just generative AI takes a massive amount of data to work. Both for training and for ongoing inference. It's just huge. So again, totally different type of data architecture that's required. That's kind of point number one. Now, for those of you who are gluttons for punishment, I can give you the Huawei paper on data architecture. And it's like 80 pages of just technical graphs. It's really, for me, it was a pretty, it was a bear to get through it. I went through like two cups of coffee, but it was really helpful. If you're interested, send me a note. I'll send you the paper. But let me at least read you a couple quotes from it. Okay, so here's from their paper on all of this. Quote, "Typically, with every application transformation comes a corresponding evolution in the architecture of data infrastructure. Reliable, performant, and shared data storage is the optimal data infrastructure for databases such as Oracle Database." However, as new intelligent enterprise applications continue to push their limits, a new data paradigm is forming. Quote, "To take AI to the next level, AI foundation models require more efficient collection and pre-processing of massive amounts of raw data, higher performance training data loading and model data storage. Let me read that again. AI foundation models require more efficient collection and pre-processing of massive amounts of raw data, comma. It also requires higher performance training data loading and model data storage and it requires that enterprises shift your digital transformation focus from application innovation to collaborative innovation of both applications and data infrastructure. Okay, that's kind of wordy, but basically in somewhat technical language, they're talking about the problem of collecting massive amounts of data efficiently when it is overwhelmingly unstructured and chaotic. I mean, think of all the data flowing in from every factory and street in person, all that flowing in. So there's a collection problem and how you share that across multiple users. So collection goes with sort of access. And then the next issue is pre-processing. It's gonna have to be put into somewhat of standardized format so everyone can use it. And then the other problem is how to use that for training and for inference in an efficient way. How to do it. I mean you can't store all the data so all this data flows through the algorithm. It has to make a decision on the fly, but it can't store all of it. It is just way too much data to store. So it has to almost like to sort of position itself in the river, have the data flowing, and then making decisions on the fly. And now some of those learnings will put into the knowledge map. But from collection to pre-processing, to model training, to model sort of inference and then to storage, it's just a completely different picture. And if you want the actual solution for how to do this, I can send you the paper, they basically lay out the structure. What you're going to use flash memory in this part, you're going to have to use collaboration here, you're going to have to use these types of, they lay out the actual hardware, if you're curious. Anyways, it's worth reading. Okay, I'll finish up here in about five minutes. But that was kind of point number one. I wanted to talk about, you know, as we look at these generative models, that data layer is a huge thing. And I actually think when it comes down to competitive strength and who's going to win, which is what I focus on, I think a lot of that's going to come from the data layer. I don't think it's going to come so much from the foundation models. I think it's going to come from who gets more usage and market share. And I think who has superior sort of a data layer to it. That's my guess. I think that's where it's going to play out, but I don't know for sure. That's why I'm trying to get smarter about it. Okay. Other point for today, which will be a quick one. Now the other point was model builders, agent builders, and app builders. Now, a couple weeks ago, Robyn Lee had a big conference by the CEO who I'm literally following everything this guy says. I think he's on top of everything. They were talking about sort of, okay, Baidu has released these foundation models. They're pretty far much out in the lead for China. Doing really, really well. I think their strategy for their AI cloud is outstanding. I've written a lot about that. I mean, a crazy amount about it, like four articles, and I got one more coming. But the other part of this tech stack I think is interesting is the toolkit. Okay, there's the data layer, the standard layers, The other part of this tech stack I think is interesting is the toolkit. Okay, there's the data layer, the standard layers, but the other one is the toolkits. I mean, to win in this game, you have to get customers to start using your tools and adapting them to their enterprises. You also have to get all the developers to start building stuff with your system. Now, traditionally, when we talk about getting developers to write with your system, whether you are the Apple iOS, you want application developers to write apps that run on iPhones. You want Android developers to write apps that run on Android. When people talk about fighting for developers, and this is where Huawei's Harmony OS is doing quite well, because they have gotten to critical mass in terms of developers writing apps that run on Harmony OS. They've really hit the numbers. When we talk about developers, we're usually talking about developers writing apps. They're writing apps that run on Microsoft Windows. They're writing apps that run on smartphones. Game developers, okay, game developers, they're running, you know, they're writing games that run on Android or, you know, epic games or whatever. But what Robin was talking about when he talked about the new toolkits that run on Android or Epic Games or whatever. But what Robin was talking about when he talked about the new toolkits that they're releasing for developers, he basically said, we are developing toolkits for AI developers. And he talked about three different toolkits that I thought were really just cool as hell. The first one is app builder. That's their name. It's called app builder. It's for developers and it's so developers can write apps that build on their foundation models and other things. Okay, that's pretty standard. But then he talked about two others and he talked about model builder and agent builder. Now, what is model builder? Well, that's the idea that, look, we are gonna provide you with the foundation models, Ernie, basically. You can then use this, our toolkit, model builder, to build customized models based on our Ernie bots. So you can take our foundation model and L0 and build your own L1 as a developer. That is specific, so this would be a GPT-like model that is customized for running a call center, for creating marketing content, for creating short videos that are personalized for marketing, for creating a GPT-like model that is very customized for running a supermarket. So they want developers to create their own customized models that run on Ernie. You know, that would be and the toolkit they are providing is called Model Builder. Just like App Builder would be, you can build an app. Model Builder, you can build a customized foundation model for whatever you want. And they want people to do this, which if you look at the numbers for Ernie, for Baidu, they're up in the 200 million users right now. I mean, it is a massively used foundation model. And they have, I think, hundreds of thousands, tens of thousands, but I think it's now hundreds of thousands of customized models that have been built based on their foundation models. So they have app builder, which is cool. They have model builder, which I'm paying a lot of attention to. But then they have a third one, which really got my attention, which they called agent builder. And I've talked about this before, which is what happens when we start building AI agents that basically can kind of act autonomously. They can perform a task like order me a pizza and have it delivered and have it paid for. So in that case, my AI agent is like my personal assistant. I can have an A.I. agent who's a lawyer who reviews all my contracts and makes edits and checks them against the laws of California or New York or whatever. So you can have lawyer A.I. agents. You can have A.I. agents that are programmers, And that's been very effective. They can basically write code for you. But there's this whole idea of a non-human workforce that's being created. People are excited about it. I've talked about it a lot. It's pretty crazy to think about. Anyways, one of the developer tool kits that Baidu released a couple weeks ago was called Agent Builder, where you can start as an AI developer. You can start to build agents that then you can sell to people. And I've been following this, and it's pretty unbelievable. Like I heard of one which basically does sort of payment processing where there's a company is called Hercules AI, which basically if you don't want to hire your own employees and you don't want to build your own AI agents, you can hire one of their AI agents like as a contract employee but in this case it would be a contract AI agent employee not a human being and you can deploy that employee into your business to do your payment processing and the cost of this employee is a couple hundred thousand dollars per year and you would think, that's not good. Like, I thought this was going to save me money." That sounds a lot more expensive than me hiring a regular employee to do payment processing, which it is. It's more expensive until you realize that this one AI agent that you're hiring is a contract employee from Hercules AI can do the work of 500 regular humans. Oh well then it suddenly it doesn't sound so expensive. It sounds really cheap. So yeah there are companies that are starting to basically create AI agents that you can you can rent as employees in your company or you can build your own internally to have basically a non-human workforce do stuff at a tremendous productivity rate that's pretty stunning. You're not going to hire one AI agent to replace one marketing employee. You're going to hire one AI agent from a company like Hercules or whatever. That can do the work of a hundred humans. Right now you can hire it from an outside firm or you can develop it internally. But either way, it's probably going to be built by some sort of agent builder building this on top of a company like Baidu. Very few companies are going to build the whole foundation model. They're going to build this on top of one of these major models like GPT or LAMA or Ernie. So anyways, if you look at the Baidu press releases from a couple couple weeks ago, they've got these pretty great toolkits app builder model builder agent builder. I thought that was another sort of layer to the tech stock that was really important and and really kind of awesome that and the data layer kind of blow my mind. So I've been thinking about those a lot in terms of competitive strategy and what that means when supermarkets or payment processing firms start to replace certain parts of their workforce with that. Anyways, very interesting. You can look it up, it's pretty cool. Anyways, that's kind of the two sort of so what's and I'm thinking about right now and yeah, that was that long title for this podcast was, you know, keep an eye out for Huawei in terms of data architecture and keep an eye out for Baidu in terms of agent building. I think that's going to be a major thing. Not in five years, like in two months, I think this is really going to hit. So we'll see. Anyways, that is the content for today. I came in at 34 minutes. So that's better. I'm not at 50 minutes anymore. I'm keeping it to 30, 35 minutes. I'm trying to get it shorter. But yeah, anyways, that is the content for today. As for me, it's been a pretty great week. I've been, I've been traveling around the Philippines, a lot of time in Manila, which is not awesome. It's kind of crazy and dysfunctional, but they've got great restaurants. Well, great cafes anyways. Lots of fun stuff happening. But then I went to Sebu, which I like much more. A quiet city, but you know, I rented a scooter and I went up into the mountains. I mean, Sebu is right on the water, but it's also right up against the mountains. So you can basically hop on a scooter and you'll be pretty much up into the province in about 20 minutes. So you know for those of you familiar with Cebo I basically went straight up the mountain past the temple of Leia something like that and up into those mountains where everyone rides their motorcycles and scooters and just had a great time. I really love it up there. I didn't have too many days. Otherwise, I would have sort of gone all the way down to the south and, you know, done the whale sharks and all the swimming and the beaches and the mountains. I really do like that whole area. Like Sebu is a lot of fun as an island. Not so much as a city. It's pretty... Not a lot to do there. Just a couple of miles. But the island's great and then you can bounce over to Bohol and you can sort of go the other way up into Ilo Ilo. Gimmeres, for those of you who are familiar, Gimmeres is like the mango capital of Thailand, of the Philippines. It might be the mango capital of the world. It's a little island right next to Ilo Ilo and it's just all mangoes. Like you go down the street and it's a little island right next to Ilo Ilo and it's just all mangoes. Like you go down the street and it's just stalls set up all along the street with just mangoes. It's really fantastic. So I try to go there. And as I'm going to go back in a couple weeks to a sebu and I'm going to sort of explore and probably try and get over to Gheemra's that's my low plan. Anyways yeah that's a fun part of the Philippines. I really like it there. So there's not a huge amount to do, but it's a great place to sort of chill out and go swimming and my girlfriend is kind of obsessed with swimming. Like I don't totally understand it, but it's like every day. Like what are you gonna do today? Swimming, like again, yeah. So every, so the Philippines is a big hit because you can go swimming every single day. Like, you know, wherever you are in the Philippines, you can basically get into the water every single day. So yeah, that's kind of part of it. A lot of swimming and there gonna be a lot of hiking. We're gonna hike the volcanoes in Java and then bounce over to the Philippines. I'm looking for good hikes in the Philippines. I kind of know Indonesia hiking pretty well. We're looking for good hikes in the Philippines. If anyone knows of any, let me know, I'd appreciate it 'cause we looked online. It's kind of hard to tell which ones are good and which ones, you know, I didn't get a good read so any of us that'll be the next plan is to do some hiking in Indonesia and then a lot of swimming and hiking in the Philippines so anyways that's my big plan for the summer. Anyways that is it for me I hope this is helpful and I will talk to you next week. Bye bye.