Democratizing Analytics with Gen AI: AnswerRocket's Innovative Solutions, LLM Agnosticism, and Transformative Impact in Healthcare and Retail

Unlock the potential of Gen AI-powered analytics with insights from Michael, the CTO of AnswerRocket. Discover how this pioneering company, founded in 2013, is transforming database interactions by enabling natural language conversations. At the forefront of innovation, AnswerRocket has adopted an agentic approach to model decision-making, steering clear of the pitfalls of pre-trained data inaccuracies. Michael discusses the significance of being LLM agnostic, which empowers users with flexibility and cost efficiency. As AI token costs fall, this adaptability becomes even more impactful, making AI solutions more accessible. Listen to captivating stories from healthcare and retail, where AnswerRocket's technology turns raw data into actionable insights and streamlines business operations.

Explore how AnswerRocket democratizes complex analytics for non-technical users across industries. By employing a semantic layer, they ensure industry-specific data interpretations that are crucial in sectors with diverse terminologies, like retail and pharmaceuticals. This empowers "citizen developers" to leverage AI for deeper insights, resulting in improved ROI. Additionally, Michael shares strategies for using consumer survey data as a digital twin to understand real-world behaviors, aided by consulting services that guide enterprises in adopting Generative AI. With a strong focus on measurable goals and iterative demonstrations, businesses are equipped to harness the full power of AI, ensuring ongoing growth and adaptation in today's dynamic enterprise landscape.

Transcript

Speaker 1 0:01

Hey everybody, Fascinating chat. Today we're diving into the world of Gen AI powered analytics with AnswerRocket. Michael, how are you?

Speaker 2 0:12

I'm doing well, Evan. How are you?

Speaker 1 0:14

I'm doing well. Thanks so much for being here, really intrigued by your mission and vision at AnswerRocket. Let's do some introductions to AnswerRocket and yourself and a little bit about your background as CTO, michael.

Speaker 2 0:26

Absolutely so. Yeah, so AnswerRocket was actually founded in 2013. So we've been at this for some time. Our mission and this is coming out of a previous exit from a business that we built our mission is to democratize databases, and we do that through natural language conversations databases, and we do that through natural language conversations. Now, obviously, we had a number of years before Gen AI when we were building out these capabilities, working with some fantastic companies, and we reinvented ourselves in 2022 with the advent of large models, and now our product, max, is really bringing that ultimate mission of the company to life for a lot of fantastic customers.

Speaker 1 1:06

Well, that's fascinating. So talk about your notion of AI-powered insights. How do you provide?

Speaker 2 1:13

those unique insights and what's the approach there.

Speaker 2 1:22

Right, right, right. Well, look, everybody has the experience of jumping on chat, GPT and asking it to do something, and we all have our own personal miracles, right, that we've experienced there with this fabulous technology. What AnswerRocket does is a little bit different than that. So we use that same language model capability to interact and have a conversation, but we try to stop it from ever using anything that's in its pre-training, right? So the PNGPT, the pre-training, is really what leads to things like hallucinations. It leads to all the problems with the model saying things that we don't want it to say. So for AnswerRocket, we take a different approach. It's more agentic, and when I say agentic, a lot of people say that they're agentic. So let's clarify what that is.

Speaker 2 2:01

Agentic simply means that the model gets to make some decisions. It has agency. So I'll give you an example. If a user comes along and says to the system hey, what's making my sales go down? What's driving my market share up? Why is there no halo on my promotion? These kinds of very natural questions the model can say I don't know. What sources do I have to consult? I can consult the internet for policy around what it is that drives consumers to buy products. It can consult databases for trends in the actual product evolution. It can look at syndicated data to look at the overall market versus one brand. So that agency is really letting the model. Instead of forcing it to say, well, I better make something up, because that's what I was trained to do, it lets it say, oh well, maybe I need to go research this, maybe I need to use some tools to be able to get that answer. And so that's the fundamental step that AnswerRocket takes, that Max takes, and the reason it's really been adopted.

Speaker 1 3:00

That's fascinating and I understand you're also LLM agnostic. You pride yourself on that, supporting multiple LLMs. How do you do the?

Speaker 2 3:09

fundamental technology we're using is simply the language model technology, and they're all based on the same kind of transformer architectures. In fact, a lot of the same people have moved around between the various companies that are building them, so they're all using very similar overall neural network capabilities. So the fact that we can use them no matter which one, simply by swapping them in and out, means that we are taking advantage of the fundamental capabilities of the technology, not any one proprietary company's sort of secret aspects that they have. And what that means for our customers is that ultimately, they can make these moves from one model to another easily without having to sort of face that lock-in proprietary nature that you see a lot that IT leaders fear. That ultimately consumers, you know, want to shirk off. Right, I want to choose the best because it's the best, not because I don't have any other choice or because it's expensive to choose something else, right? So it was a great quote that I heard.

Speaker 2 4:18

I think it was Andreessen Horowitz that did the analysis. But the cost of a token, right? So the token is the unit of cost. In the AI world, the cost of a token has fallen roughly 200 times 200X not 200% 200X since these things started to be sold commercially. So that just means that you want to be able to move between models. Right, because it's really kind of a Jevons paradox situation where it's fallen so low that the demand is going up. Right, because each token is so valuable in its business purpose and the price is so low that there's just a growing desire for more and more. And if you can stay flexible about which model to go to, then that means you can shop whichever carrier is giving you the best price. And that means that you can get more AI for your buck, whichever carrier is giving you the best price.

Speaker 1 5:04

And that means that you can get more AI for your buck. Yeah well, wouldn't that be fantastic? Let's talk about your customers and the impact you're making. Care to share any you know anecdotes or stories where you know how it's impacting their business or maybe their industry healthcare, retail, finance, others in which you are active.

Speaker 2 5:23

Yeah, absolutely so. You know, some great stories for us are. I'll give you some examples. Again, the mission of the company is to democratize that access to data right, to be able to make it so that this data that customers are investing in becomes a valuable asset in a more broad way, right? So one of our customers had, roughly on a typical syndicated data cycle, they had every 90 days new data dropping, right, and they had humans. They had a room full of humans whose very boring job was to turn that as quickly as possible into a bunch of PowerPoint decks, which took a couple of weeks. Right? So now, every, you know, every 12 weeks, I'm going to take two weeks of additional delay just to get this out there. And then, once they would publish that information, any questions and follow-ups and whatever that happened as a result of that, you know, took more manual time to rework and then within another two weeks, it was stale information and nobody cares, right? And that sort of vicious cycle repeats what they were able to do with AnswerRocket instead is to basically, within hours of that data dropping and I literally mean within a few hours they would have not just those same decks that they made before, but they would have those decks, but across every one of their brands, across every one of their countries, right? And what that means is that it's coming in two weeks earlier, which is exponentially more valuable, right? Because, again, we're on a 12-week cycle, so two weeks is a huge amount of time to save, and that means that those customers, now when they have follow-up questions, there's still time to act on that information, so they're able to look at that and make more questions and because it's automated, they can very quickly get those answers right. So the whole cycle of analysis is compressed down and that means that the time that you can spend observing insights is increased. And that's exactly the way that our mission plays out.

Speaker 2 7:11

That we wanted to play out is to say that getting insights from the data is where you want to spend your time. Actually, I don't know any business user whose goal is to write SQL today. I laugh when I hear a lot of companies saying, oh, we're empowering enterprises by letting them type text and get SQL back. I don't know anybody that wants SQL. They don't want SQL, they want insights from their data.

Speaker 2 7:34

So for AnswerRocket, our approach is to say look, based on the user's question, we are going to go ask the model what it wants. The model is going to say, oh, I need data. What data do you need? Okay, ask the model what it wants. The model is going to say, oh, I need data. What data do you need? Let's go get that data. But that's not the end of it. The user doesn't want data. They want that data turned into insights. I want to be able to take that information, convert it into insights and communicate that back to the user conversationally, the way that a human would, the way that these agents are intended to do, based on how they're trained.

Speaker 1 8:05

Wow, Fascinating. So, as you know, the beauty of these LLMs is their accessibility as a user. You go on, sign up for free or with a credit card and you're off to the races. It's just beautiful. How do you go to market with partners in the enterprise where that's not the typical process for engaging software? How do you deploy and go to market with these customers?

Speaker 2 8:31

Absolutely so. Look, there's two important parts to the relationships that we walk into. Number one they've already got a policy around Gen AI right. They already know how they want to do it, what their limits are for all of the responsible AI stuff like basically avoiding hallucinations, but also avoiding obscenities and filtering the content in lots of ways. So we have to adapt to that, right. That becomes part of how we interact. And then the second part is they already have a tech stack right, so we can't walk in and say, oh, you may have spent 15 years building your enterprise architecture on XYZ, now you're going to switch over to ABC. That doesn't work either, right? So we walk in basically with a working product, but also a toolkit and also a set of skills, and so we'll go to a customer that has, you know, a lot of times, a typical project right.

Speaker 2 9:24

They've spent the first six months of 2023,. They got a RAG solution working. Right, everybody did that. Rag is this retrieval, augmented generation. It's where you take your dark data, your internets and your PowerPoints and your PDFs and your emails and you get all that stuff to where I can have a conversation with it. Everybody had big victories around that, and now they spent the last year trying to make that same technology work with databases and it just doesn't right. So they come to us when they're ready to say look, we have a success working with that RAG technology, with that dark data, but now we need to take this forward.

Speaker 2 9:59

And AnswerRocket comes in and says fantastic, we are going to empower your digital twin. Right, the digital twin, all this data that enterprises have. Right, they've been investing. They've got a model of their entire operation in that database. It knows more than their operations people do, but they can't take advantage of it because it's just numbers, right, numbers and columns in a database. So the language model steps in and says I'm going to be the go-between, the business user with a question like what's driving my sales or why is my market share falling, those kinds of questions.

Speaker 2 10:32

And the database that has that information because it's got a view of the entire world fairly up to date, but doesn't really know how to present it right. And so in between there, answerrocket provides what we call tools or functions in the language model world. For AnswerRocket they're simply branded as skills, and so we provide these skills that allow you to say oh, let me understand trends in consumer behaviors. Let me understand the distribution of a product within a marketplace. Let me break down for you how, let's say, the volume of sales is impacted by price and vice versa. Right, these kinds of analysis that are fundamental. They're reflected in the data because consumer behavior drives them all. But getting the insights out of them takes that middle ground that looks to not just generate SQL but actually get the right data formulated in the right way and under the right kind of analysis, and present that to the language model to make it smart to have a conversation with the user.

Enhancing Analytics Access for Non-Technical Users

Speaker 1 11:37

Fantastic. And how do you think about the unique requirements of certain industries you know in there, whether it's retail or finance healthcare, others that have all their three, four letter acronyms and unique requirements. How do you approach those?

Speaker 2 11:54

This is really where the sort of the eight years of history before language models really helps answer Rocket out. We built our platform from the beginning to be a layer separated from the data, so the semantic layer, you know, whatever your data is, whatever you call things, there's a semantic layer that wraps that. I'll give you a great example. In the pharmaceutical industry, there's an acronym, trx right, and there's TRX in retail as well, right Now. In retail, typically, you're talking about transactions. Right. In the pharmacy industry, you're talking about total Rx or total prescriptions, right. So if you just say to a language model, hey, here's a database, now, how many prescriptions did we do in January? It's going to look at that and say, well, I've got transactions, I don't have prescriptions, so you're out of luck. Answerrocket instead wraps that data with a semantic model that is automatically generated right, because models are really good at things. But there's a human in the loop to make sure it's right to correct it to get it exactly the way it's supposed to be, and so the result of this is that we're really using that language model's natural capability to simply interact with people. That's what it's good at. It's a language model, after all, and we're taking away from it the responsibility for knowing anything about any one industry. Right, we're neutralizing anything that it might know because it's probably partial. Right, For any given business, a language model probably knows something about them, and that's kind of cute, but you can't ask it a question, right? If you ask it a question and you want a reliable answer number one it's not going to be able to quote you a source for it. Number two, you better not trust it if it does, right? So that's where a tool like AnswerRocket comes in and ensures that when the language model tells you something, it's either going to have a citation and a reference or we're not going to show it to you. Right, that's the kind of wrapping that it takes to make something like a language model, which, again, is a miracle technology. Apply to this very tight use case, right?

Speaker 2 13:51

So my analogy is always the horse and the engine. Language models are a little bit like horses, right, you have to have a special place to put them and a good way to feed them, and you have to harness them. Just right, you know they're not an engine, they're going to be an engine, but you need something to really wrap that power to take advantage of it right and you're not going to just put to the, you know, strap it to the front of your spreadsheets and tell it to pull right. You're going to get really bad results if you do that. On the other hand, if you wrap that horse in an engine like AnswerRocket, then you can take advantage of that unique power, right?

Speaker 2 14:23

This modern miracle that we have in a way that still satisfies some of these enterprise workloads that are really thirsting to be brought online. I mean, the fact is, the number of roles in analytics, in the customers we serve, is never enough, like the number of people, like they could just keep hiring forever and they would never have enough people, right, and those jobs are terrible, right, one more time run the report that, right? So we put AI in there. We put AI in that core of sort of repetitive things. We let those people wrap that work in a way that allows them more agency of their own, more of that. That's got a much more satisfying job and the AI is very happy to sit back and do the repetitive work and drill down and spend as many hours as you want it to researching some minutiae that you're after understanding.

Speaker 1 15:21

Brilliant. So you're making complex analytics accessible for decision makers at all levels, particularly non-technical users. We sometimes call them citizen developers. How's that going? And that must be quite gratifying to see amazing, right.

Speaker 2 15:41

It's one of these where, you know, I've built products throughout my career, always bringing new technology into the marketplace, and there's a phenomenon that occurs that I'm starting to see now which is really gratifying, and it's that a customer has a whole lot of complaints to give you and it's kind of shocking like wait, what's going on? You know you don't like it. Oh no, no, no, I'm changing my whole business because of this and I need these things addressed because I can't live without this right. So it's very different to say I'm complaining, I don't want your thing, or, in the middle, I'm simply apathetic, I don't care whether I use your tech or not All the way at that other extreme is, I'm ripping this out of your hands now and you better show me a roadmap of all the cool things you're going to do, because I'm betting on this horse Right, and so that's a really phenomenal feeling, and you know, hearing some of the quotes from our customers how fast, how insightful, how deep it goes. I mean realize that that enterprises are really only applying some of their best human talent against some of their largest brands, against some of their biggest opportunities. Right Now. Imagine being able to have that human talent, train the AI right, that's the human in the loop and then spin the AI off to go down into smaller markets, into smaller products, into lesser opportunities right, and now, all of a sudden, you're driving real ROI for this investment. You're getting actionable insights right.

Speaker 2 17:06

A lot of the times before Gen AI, it was pretty funny. We would have insights coming out of traditional machine learning models with high confidence, but they were completely unactionable, right. There would be things like you know, get people whose names start with P to buy more. Well, that's, you know, the math says that. But that's not useful, right? And a language model understands. No, that's not a thing I need to be able to give you.

Speaker 2 17:31

If you're advertising, it's got to be about brands. You don't advertise manufacturers, right? If there's a supply chain issue, then we're talking about the commodity prices, we're talking about geopolitics, we're talking about geography, right, the models understand these things intuitively, right? So we don't want them to learn facts from their pre-training, but we do want them to learn these fundamentals about how the world works, right, and that's what makes them good chat conversationalists, that's what makes them really good at COT, which is chain of thought.

Speaker 2 17:59

That's the reasoning side of these models, so that when I say to a model something like what's an example I had the other day, oh yeah, it was, show me brands that stopped selling a certain kind of product this year. So think about that. You could never go to a SQL database and say, show me things that stopped selling. What you really mean is find the ones that sold last year, find the ones that didn't sell this year and tell me why, right. Which ones transitioned out? Who replaced them? What were the drivers maybe at a subcategory level or a SKU level that were driving those transitions? What happened, right? Did distribution go down? Did the pricing go up? Did their advertising fall?

Speaker 2 18:35

And so a model is a language model that understands, kind of, how the world works. Right, consumers receive advertising, they buy products, they pay prices, they, you know, have wages. If the model understands all that, then it can help you interpret those results in a way that a traditional, you know, support vector machine or regression or some other you know gradient boost algorithm that you know three years ago, two years ago, would have been vogue right. That would have been how we made really good AI decisions. We're not even in that league anymore. Vogue right, that would have been how we made really good AI decisions. We're not even in that league anymore.

Speaker 1 19:07

We're in a completely different place, brilliant. So you mentioned marketing. You're making waves with Gen AI and marketing making you know marketing decision making easier, especially for non-data scientists. Talk about your partnership with Kantar and what some of the goals are there?

Advancing Analytics Solutions for Enterprises

Speaker 2 19:25

Yeah, absolutely so. The idea of survey data, right, the consumer surveys, and you know this extensive, let's say. I go across hundreds of consumers and I have thousands of questions, but I can't ask those same consumers, those same questions, right? So this respondent level data is actually very complex, right, and statistically, it's really important to understand the data how many respondents are in each category, how many respondents came at each time, and so, from a, from a model perspective, it's a great example of this complex digital twin. These surveys really do amount to a copy of the real world, right, we, we have enough of a sample to really represent the real world, but that data is way too hard for anybody to interact with.

Speaker 2 20:06

I just want to know why consumers didn't like my product launch. I just want to know whether they're going to tolerate a price increase in this one region. I just want to know how the demographics changes that are happening in the movement of people from the Northeast to the Southeast, how that's going to impact me, right, and so what we'll do with a situation like Kantar, where they not only have that survey data but they have a framework for understanding it? Right, they have their meaningful, different and salient framework that really goes deep on understanding exactly why consumers answer questions in which way. They have a lot of history on how certain answers result in certain actions that those users take.

Speaker 2 20:46

So we combine those two, that sort of let's call it that background information, that formal methodology for how to apply the surveys, with that very lively actual survey data, and AnswerRocket, because of these language model skills, can pull both of those together and begin making observations. So you have a problem with this particular brand in this region because it's no longer salient for the occasions and the things that drive salience are these. So now a brand manager has the opportunity to look at that and say, oh, I see why I'm losing power, I see why my price change is failing because I don't have anybody sticking to my brand, because I used to be the number one in this, in this particular kind of question, and now I'm number two, right. So so you can see how this again, this idea of the digital twin, the real world data reflected through the policy and the methodology, can result in and essentially having a conversation. That's with an AI, but it feels like an expert helping you make those decisions.

Speaker 1 21:50

Yeah, just incredible. And you also help enterprises through a consulting capability to kind of realize the full potential of Gen AI. How does that work? And what's a typical engagement, like at the beginning of a project.

Speaker 2 22:03

Yeah, yeah, so, absolutely so.

Speaker 2 22:06

As I mentioned at the beginning, you know we don't get to dictate which model you use and we don't get to dictate what your technology stack is right.

Speaker 2 22:12

So, at the end of the day, if I want to be able to help anybody and again my mission is to democratize access to data it's not to make you know my specific product better, right, it's to democratize access to data. So if I want to do that, then I've got to be able to walk in with nothing but the skills. The skills that are the capabilities of a team that's deep and able to understand what your business problem is, able to know which technologies to apply and able to get that done for you, right. So that's our consulting practice. Now, if we can accelerate that with some of the tools we've already built, well, that's just an advantage. We can make timelines go shorter, we can demonstrate and this is important in the Gen AI era, because everybody's a little disillusioned, right With you know, it was so cool when it talked like a pirate, why won't it you know why, won't it predict the consumer trends?

Speaker 2 22:56

for me, so, if we can very quickly get demonstrations, because some of the tools and infrastructure that we bring, then that helps get budget, that helps get projects moving along and ultimately, if the technology stack we bring is compatible with what the customer already has, then we can just install that side by side. So we have kind of all three of those echelons and we see that play out in engagements where we're typically at this point it's no longer help us get started with Gen AI. Right, Every enterprise out there has gotten started with Gen AI. But then they got RAG working and that looks good. But then they heard about agents and they heard about artifacts, and now there's swarms, and then there's the outer loop and then there's you know like, and then there's strawberry I mean O1, right. So they come to us and they say all right, we see that we could do this, we know we are capable of doing it. We're also going to be 18 months late to the party. How can you help us get started? And that's when the conversations begin. We start with a simple consultancy that says what are you trying to do? Let's try to advise you on your strategy.

Speaker 2 23:58

A lot of times, the strategy is a chatbot, and let me tell you, Evan, chat is not an AI strategy, right? That is incomplete. You need to have a real goal that says what percentage of your workload are you moving on to AI, what functions are you augmenting and how? What is the ROI for that spend going to be? Right? That's a strategy. Chat is a feature, and it's an important feature, but it's just a feature, right, and so we have those consultations and sort of go through the art of the possible, but very quickly get down to All. Right, let's show something in six weeks or less, Right? This is not about stretching out long projects and you know the project will go long but only because at every rung, it's going to prove itself Right. To prove itself right, it's like a SaaS services.

Speaker 2 24:44

When you buy SaaS, you get to pay. Basically, you get to vote yes or no every time you want to pay for that software, right? Well, Gen AI is the same way. You want to see these demonstrations working, you want to see the progress that it's making and you want to decide at every step if you're going to go further and further, and that's ultimately how these companies are going to be able to build these long roadmaps are going to be able to move as fast as the technology does. Because that's another, maybe final important point is, believe it or not, the technology is moving faster than the enterprise sales cycle.

Speaker 2 25:18

So, in other words, by the time you've decided to make a purchase, you're already out of date if you're just buying a piece of tech off the shelf, so you've got to buy services that go with it. You've got to basically buy flexibility. You've got to stay mobile right In the database world. You'd be fine if you picked a database now and five years from now you looked at it again In Gen AI. You would be obsolete. You'd be wasting your money within two years if you tried to pick a technology and go with it. So that's where an answer rocket comes in and really helps you understand what are the challenges, what are the opportunities. And let's make a very granular roadmap that gets you there without you having to believe any snake oil, right, Because there is a lot of snake oil out there. Let's show it. Let's show it working, and that's really where answer rockets come in.

Speaker 1 26:03

Wow, quite a mic drop moment. So, yeah, great, great insight. We're in the busy run up to year end. What are you looking forward to? What's the team and answer rocket up to through the end of the year.

Speaker 2 26:15

Absolutely so. So the, the, the gen in gen, ai, right, the generative is a fantastic opportunity, and so far, what you see out there is the generation, either like on Dali, where you say, hey, draw me a picture of a sheep and whatever. Or a chat bot right, there's the enterprise version of that, which is where you say, hey, I want to build a presentation, I want to build a story. Help me tell a story. I don't just want you to tell me what's driving sales in Alabama, I want you to help me build a story that that says I should raise prices at Halloween, right, or whatever. Whatever that story is and the model interacts with you, right?

Speaker 2 26:55

This is where this is where agents come in. The agent says, oh, let me build a chain of thought, let me let me design a routine that's going to justify that price change or tell you that it's a bad idea. Right, it's not always going to be a yes man or a yes bot. And that agent then is part of a swarm that says, okay, let me pull up the PowerPoint agent. He's going to make us a PowerPoint about this. Let me pull up the deep research agent.

Speaker 2 27:20

It's going to go off and in half an hour or so it's going to have scoured the past to find all the other times we tried to take price at Halloween and tell us whether that's a good idea or not. Let me fire up the long range forecast and see if any competitors are likely to follow suit or if they're going to undercut us and hurt us right. So that's the work that we're doing now. It's beyond the sort of hey, I need to get this answer from the database, and it's moving more into the realm of hey, I'm a fully collaborative co-worker. Let me help you do the strategic things that you're trying to get done, and that's what we're building again in a 2024 timeframe, and it's going to be very exciting.

Speaker 1 27:59

It is indeed what groundbreaking stuff. Do you have any travel plans or events meetups coming up? What are you up to personally? Where can folks meet?

Speaker 2 28:10

you, yeah, yeah. As far as getting out there probably not for the rest of this year I'd need to consult my marketing team to find out for sure where they've got me, but I don't think I've got anything more in 24, but I am anticipating, in 2025, being back at big data, being back at A4I you know a number of different opportunities like that. I'd love to see folks out there and engage.

Speaker 1 28:34

Well, congrats on the amazing work. You're really shaping the future of data-driven decision-making in such a positive way. Congratulations, Michael and Dean.

Speaker 2 28:44

Thank you, thank you, I appreciate you carrying the message.

Speaker 1 28:47

All right. Thanks so much, and thanks everyone for listening, watching, sharing and beyond Take care.