The Dashboard Effect

The Reality of Generative AI in Analytics: Hype vs. Progress

• Brick Thompson, Jon Thompson, Caleb Ochs • Episode 98

Share episode

In this episode, Brick and Caleb revisit the world of generative AI in analytics. They revisit their experience with Microsoft's Power BI Copilot and discuss the current state of AI in generating SQL queries, dashboards, and even predictive analytics with machine learning. Tune in to get their take on the hype vs. reality of AI within data analytics (and other applications like video, voice, biomedical, etc.) and what to watch for in the near future!

Click here to watch this episode on our YouTube channel.

Blue Margin increases enterprise value for PE-backed, mid-market companies by serving as their fractional data team. We advise on, build, and manage data platforms. Our strategy, proven with over 300 companies to-date, expands multiples through data transformation, as presented in our book, The Dashboard Effect.

Subscribe here to get more episodes of The Dashboard Effect podcast on your favorite podcast app.

Visit Blue Margin's library of additional BI resources.

Brick Thompson:

Welcome to The Dashboard Effect podcast. I'm Brick Thompson.

Caleb Ochs:

I'm Caleb Ochs.

Brick Thompson:

Hey, Caleb. So today, I thought we would revisit a topic we talked about a few months ago. And that's generative AI for analytics. We, you and I, attended the Microsoft Build conference back in May. And they showed a pretty mind blowing demo of what they're calling Power BI copilot, where you can use natural language to query your data sources, build visualizations beyond visualizations- build full reports, interactive, cross filterable, all that stuff. And have it give you analysis, sort of generative AI analysis. And we're waiting for that can't wait to see that it's going to be be a big deal. But in the meantime, we've been watching what is coming out just in what other companies are producing in this area. And I thought we just might talk about what we're seeing so far.

Caleb Ochs:

Yeah, sure. That sounds like a good idea. I don't I don't think that this is totally new, even from the Microsoft conference. You know, there were some things out there anyway. But that kind of kicked everything off. And I don't know if the whole thing mapshare GBT and everything getting super popular that, you know, now it's a good time to kind of reset, like, what, what is out there? Is there anything good? And what to maybe keep an eye on going forward?

Brick Thompson:

Yeah. Well, I remember when we started playing with Chat GPT, I think it was probably, I don't know, February or something, you started building some natural language query stuff, using the Chat GPT or the OpenAI APIs to build SQL queries based on natural language inputs to give you some sort of output. And it was pretty cool, but not ready for primetime at that point. Yeah, we're taking a lot of development still.

Caleb Ochs:

Yeah, kind of what it did is it took a set of data and kind of explained what it was like it's a week over week comparison, and kind of give us some examples of what you might consider doing to either correct or continue your good performance or lack thereof. Right. So it did work. Yeah, it gave you some cool stuff and it was really cool. It was really cool at first, and then like, the more you got out or like, is that actually yeah, it's kind of like a new, new and exciting. Now it's advanced data analytics.

Brick Thompson:

yeah, exactly. So I read a bunch of different Yeah, Thank you. So perfect. Yeah. So it's pretty AI newsletters and kind of keep my eyes peeled for new cool. You can give it data, and it will give you visualizations, generative AI around analytics. And what I'm seeing is kind of similar to what you built. I mean, it's definitely further along that we're seeing people where that have tools, I think a lot of them are wrappers just on top of, you know, open API's API or other MLMs. But seeing things where you can upload this is very common, upload a flat file, a CSV or an Excel file, and have it be able to query it give you some information, you have to know what the column names are, and, and sometimes it gives you the wrong answer. But you can do it and you can ask it to build visualizations for you too. And in fact, open AI and their GPT it won't build a whole kind of nice dashboard report for you. four model now has an analytics thing they used to call it. Go to Turker. Yeah, Kurt and Trevor. And now I'm blanking out what the name was. It will also tell you things about your data. It's pretty amazing. And actually, it does other amazing stuff, too. Like in order to not give a wrong answer to a question that involves math or counting that type of thing, which these MLMs are not that great out yet. It'll write its own Python code behind the scenes and execute it to get the right answer, which, which is pretty cool. Yeah. Other things we're seeing are our systems that are saying, Hey, you can connect this to your cloud database, so data warehouse, or whatever, and then start asking questions. You know, I haven't done deep analysis or research on that. But what I'm seeing in the demos and reading about is that it still has some of the same problems. You've got to have really good metadata for the AI to be able to know what you're looking at, even then it's giving wrong answers. Some of these systems want you to help train on the metadata. So we'll take a guess at it and then when you get a wrong answer, you need to tell it what it should have done so it can modify the meta data. You know, you and I, as enthusiasts would probably enjoy using that. But your average business person I'm gonna guess, would not

Caleb Ochs:

Yeah, yeah, I mean, that's gonna be the key, right? He's getting something that's accurate. So that people I mean, people will just stop using it if if it's not like the first wrong answer you get. It's like, Well, I wasn't it's not worth my time. Right. But yeah, I mean, That's kind of what I, what I found, too is, there's one tool that I used a few months ago. That is, it's all about, you know, just being able to have a conversation with your data. And I did a free trial and I stood it up, but you really had to do a lot of backend configuration, you had to set up the relationships, you had to kind of do all the synonyms, set up the tables, like it's basically like building up, you know, data model, like you have to have to do all that stuff in order for this thing to know what it's talking about. Yeah, it responds to you. And it did do some cool things. But it just was like, man, it just feels like you're doing another big data project just to enable them. And I don't think that people are going to be ready to, to bite that off, or at least right away, and especially not in, you know, we're talking about middle market companies, they're kind of just getting off the ground with just pure business intelligence, they're not quite ready to go there yet. And it's gonna be hard to hard to swallow another big data project when you're when you're not quite there yet. So we really need this thing get advanced quite a bit more before it's really compelling for a larger audience. Right. I think the other thing that was really interesting that I that I did, this was at the Microsoft conference, I remember I was showing you breakfast. I took one of the Open, open source, LLM models, right? I remember, yeah, yes. Stuck some data in there. And like, had it read it. So it had that knowledge, you know, in its in its in its model, and then trying to ask the questions about, you know, that set of data and everyone's really having a hard time, maybe we didn't configure, right, maybe there's some tweaks to be had, but it just was just kind of like it's not quite as easy as you'd want it to be. To do something with, right.

Brick Thompson:

Yeah, I think I think it's very clear, it's gonna get there. And, and using these tools, you can see how great is going to be, it's probably going to be a series of steps, though. Like, even when Microsoft launches copilot for Power BI, especially, they're still going to be a lot of configuration to do on the back end, you know, you mentioned synonyms. So you have column names, you know, the LLM may be able to guess, when you're asking for relative example, made is not a great example. But you're saying give me total revenue for the last 90 days compared to the 90 days before that. And the column name is sales. And the LM probably be smart enough to figure that out, but might not be there will be more obscure things where you definitely have to give it what are the synonyms people might use. That will keep getting better, though. And I think it will happen fast. Actually, that kind of brings me to another idea I've been thinking about, which is, you know, the whole hype around MLMs and Chechi. Beatty and AI was crazy this spring, I mean, everybody was talking about it was on every new show every podcast, it just just pervasive. And now it's not quite as much. I mean, it's definitely out there, but it cooled off a lot. But what I'm seeing as someone who's really interested in it, and watching it closely, is that it's actually accelerating. It's, it's crazier than it was then what people are doing with it. And it's I think it's gonna, you know, there's gonna be more of those surprising things to people who aren't watching it closely. One of them is going to be around data analytics, I think.

Caleb Ochs:

Yeah, I'm interested to see when that happens. I think that for me just using chat, GBT, I use it a lot. But you know, the more you use it, it does the kind of the novelty of it starts to wear a little bit and you start being a little bit more critical of Yeah, because it's not so amazing anymore. Right now, you're just expecting it to give you good stuff. Right? And when it does, and you're kind of like, what the hell yeah. So yeah, I can see how like, you know, that hype train has kind of gone down. But you're right. I mean, the development? Sounds like sures sure hasn't stopped?

Brick Thompson:

No, oh, no. Yeah, I don't think it has. I think it's, I think we're going to be surprised by things we see. You know, this, this latest version of Chet GPT that you can upload images and ask questions about them. I uploaded some notes yesterday, from our, from our meeting that we were in earlier in the day, just have it do some summarization and talk to it about that, I found that really useful. It did make some really not useful remarks and some kind of dumb statements and misunderstood stuff, but it's still kind of useful to be able to bounce that off of, you know, a quasi intelligent partner, you know, it late in the evening when all your colleagues are not available. So, yeah, it's pretty amazing. Oh, I know what it was. The other thing I'm seeing with the generative analytics is that people are automating the machine learning aspect of AI. So taking a data set and having the AI do predictions with that data set using machine learning and figuring out which algorithms and and which models which machine learning models to apply to Get good answers. I think that seems to be coming along pretty quickly. Because it can do testing and sort of figure out what the best way to output is. And obviously, with machine learning, you've got a training data set. We know what the answers are. So we can look and see how am I doing against the training dataset? So I think we're gonna see that coming. Fast, too. There's already been a bunch of that. That'll keep coming. Who's doing that? I think several companies are there was one I was thinking about, in particular, I can't think of the name of it. It's sort of obscure company.

Caleb Ochs:

Interesting. But yeah, that's cool.What other cool things are you seen?

Brick Thompson:

I mean, in terms of the generative analytics, that's about it. I mean, frankly, the visualizations that that I've seen kicked out, even the best ones are not great, sort of simple bar charts and column charts and sort of the stuff that you used to produce with JavaScript on your web page using so those those old libraries, so I'm sure that'll keep getting better. But it's not quite ready for primetime yet, in my opinion.

Caleb Ochs:

Yeah. Yeah. Interesting. What about just

Brick Thompson:

Generally, with API's, I mean, the stuff around generally? image, video stuff is unbelievable. I mean, I think people are gonna be making hopefully usable video soon. And we've all seen some well probably seen some of those crazy videos that were produced.

Caleb Ochs:

I think it's freaky.

Brick Thompson:

But you wonder if you're losing your mind. So those are cool. I think the things that people are doing with voice are amazing. So you can get it to clone a voice, just how a voice sounds and then dub whatever they were saying into whatever language you want. I've seen some, some examples with that. There's a podcast I follow called Hard fork. That's a technology podcast and those guys were playing with, there's two of them, playing with putting one guy's words into like a different language that they don't speak, but in his voice, it was amazingly. And now they're their tools to sync up video with that as well. So you can actually delve, Delve voices onto the video, and it looks very real. I don't know if you've seen that at Lex Friedman interview with Mark Zuckerberg, where they're both in the metaverse and it's just creating awesome clips. I mean, it's pretty amazing what it does, it looks like them, and it's mirroring their expressions, or what their mouth is doing. So I mean, there's amazing stuff coming. A lot of that stuff that we that I just talked about is a little scary.

Unknown:

Is this podcast, even real? I mean, this might be generative AI.

Brick Thompson:

Exactly. That's great. I think of the number of phishing attempts I get these days, and they get better and better. I don't think I've fallen for any yet, but I'm worried that they're gonna get so good with the help of AI, it's gonna be tough. I mean, I, there was a, I saw a demo of someone talking to a customer service representative on the phone. I mean, it sounded like a person was a demo, but you know, there are gonna be people to get taken in by that for sure. Yeah, so a lot of good can happen with it. There's a lot, you know, one other thing before we wrap is in the area of biotechnology and medicine, amazing things that AI is being used for, to help create new proteins that might fit a certain shape for a certain purpose, or come up with which molecules might be good drugs for treating certain types of things. And sifting through 1000s of those and picking out the 50 that should probably be tested. And I think we're gonna see really amazing stuff there. Even the image processing, they're being able to look at images, radiology. And so I think there's a, there might be a cancer here, we need to get a person to look at this. Right. And I think some of those are working are more accurate than people at this point, too. So I mean, there's a lot of amazing stuff coming.

Caleb Ochs:

Yeah, that's pretty crazy. I mean, as this is, this is super elementary, but it's like, kind of back down to earth. So I do think it's even just crazy when you're talking when you're doing stuff on chat GPT or some chat bot, right, you're, you type in this big long thing. Sometimes you might just paste in like a transcript. And it'll it turns to that thing so fast. Like, that's amazing.

Brick Thompson:

And so smart.

Caleb Ochs:

Like, you'd think you'd take a little bit of time to like, read it. Yeah. It just goes really fast. Yeah, it's great. Yeah.

Brick Thompson:

So cool stuff. Anyway, the topic for the day was general AI and analytics. If a listener out there knows of some tool that we're just missing that's fantastic. Please send us an email. Yeah, although CW we would love to Yeah, we're watching closely for it. But but my assessment right now is cool stuff fun for easiest to play with not ready for your general business user yet.

Caleb Ochs:

Yeah alright good advice.

Brick Thompson:

Alright talk to you soon. Thanks!