Welcome to The Dashboard Effect. I'm Brick Thompson.
Caleb Ochs:I'm Caleb Ochs.
Brick Thompson:Caleb Two episodes ago, we talked about answering the question, what is Power BI? And I thought we might do the same thing today answering the question about what is fabric, fabric being the new platform that Microsoft launched back in May, into public beta, basically, the new data platform out on Azure. Yeah, yeah, it'll be good. I mean, there are a lot of components to Power BI and kind of similar to Power BI is just one of the components of fabric. So yeah, there's a lot to cover. But I think we can cover some of the high level to give people a broad understanding of, of what they can expect there. And I know you've been doing some training classes, I'm excited to hear what you've been learning. Let's see what happened. Well, I've been doing the fabric for, for Dummies courses out on Microsoft, it's good stuff. So at a very high level fabric is the branding and packaging, around Microsoft data platform. Previous to that, I think, probably synapse would have been what it was. And so it's becoming fabric, and really includes all of the pieces of things that you need to have data environment, a BI environment, things like data, warehouse, data, Lake, house, Power BI data models, all of that stuff, including machine learning, and so on. So within fabric, they've done a really good job of integrating all of these pieces. And it's, it's pretty fascinating. So, so maybe we'll start with data engineering. So what is data engineering and fabric?
Caleb Ochs:So I think the way they do it, you might have to correct me here, since it's a little more fresh for you. So the data engineering piece, is the it's going to be your your data warehouse.
Brick Thompson:Right, right. Yeah, it's the data warehouse or data lake house.
Caleb Ochs:Gotcha. Yeah. So so it's going to be where you're going to do like, you're gonna pull data, you're gonna land it somewhere, you might do some cleansing, you might do some transformations of it. But you're not necessarily doing like machine learning at this point.
Brick Thompson:Right? Yeah. So so the engineering is really about setting up a place where your data is going to live. So it's something called one lake. In fact, I don't want to get too in the weeds here. But Microsoft is moved away from a true traditional SQL Server, even if you're working in SQL, you're going to have all of your data stored in a data lake data lake house. So it's really creating a data lake house, and then all of the tables and the data models and various things that would exist within that. So another section in fabric, or another part of it is called Data Factory. So it has been around for quite quite a long time on on Azure, what is data factory? And how are you going to use that in in fabric?
Caleb Ochs:Sure. So Data Factory is used to move data, you can do a lot of stuff. But typically, if you connect to your source, and then you dump the data somewhere, in this case, it will be in a one like. So data factory, the way that we're seeing it is that you'll use some of that just data and data engineering pieces, like the spark notebooks, and things like that. And then you'll use Data Factory is kind of your job orchestrator and scheduling, and that type of thing. And like making sure things don't fail and all that fun stuff. That's where data factory will fit in.
Brick Thompson:Got it. All right. Another piece is data science, I think of this primarily as machine learning stuff. So building tools that can help with predictive analysis and so on. Predicting which customers might churn, that type of thing is really well integrated into the to the ecosystem. And in fact, it's gotten so easy to do that stuff. It's amazing. Easy, you still gotta start to know what you're doing. But you know, five years ago, six years ago, that machine learning stuff was really you needed a data science degree to even approach it. And now regular people can actually start using it.
Caleb Ochs:Yeah, I mean, there's that's kind of the theme of fabric, right? Just make it easy. And, and machine learning is one of those those pieces. And I think what's exciting about it for down the road is that this AI you know, AI studio is another really cool thing that Microsoft's doing, I could see that being integrated right into fabric. And so you're just, you're answering questions about your data, as you're processing it. That would be pretty sweet.
Brick Thompson:Yeah, I would imagine we're gonna see some really cool stuff over the next few months. And then obviously, another huge piece of this is Power BI. So Power BI is now included as part of fabric, it's part of the fabric ecosystem makes it really easy to create, what used to be data analysis cubes, analysis or versus cubes. Now you're creating data sources within fabric using Power BI to help you do the modeling of the tables and, and creation of DAX, and all of that stuff. Yeah,
Caleb Ochs:I mean, it's directly because what they're calling it from Power BI directly to the, to your one lake, you go through some of the things like the data warehouse, what they're calling the warehouse and the lake house. But you can model it right inside of, of the web UI inside of fabric. You can even write DAX in there. And then the coolest part is that it doesn't actually pull the data into Power BI just stays there, right? In your warehouse, or lake house. And Power BI just transacts against it. And it's lightning fast.
Brick Thompson:It's amazing how fast I don't know quite how they're getting that performance. It's amazing. It's awesome. It really is. Alright, so you're already mentioned another piece data warehouse, how to, I have a little bit of confusion around what they're calling data warehouse. Now in the old days, that would have been sort of a Kimball model SQL Server base something, how are you seeing that in fabric?
Caleb Ochs:Yeah, so the way that they've laid it out is warehouse is going to be your sequel engine. So this is really the only difference. So warehouse, you get to write SQL against it. And you can do some transformations, or creating new tables using SQL, T-SQL. The Lakehouse is going to be spark. So that's going to use PI Spark, or, you know, Python as you as you write things there. And that's really the only difference inside of fabric as your as you're dealing with them, they look pretty much the same, right? Right. It's like, oh, there's your tables, you can create your relationships, you can do all the things that you can, like between the two of them. The difference is the engine that's processing the data.
Brick Thompson:And all of the data is sitting natively and delta parquet files in one lake, but it's just basically how you interact with it. How you model and that
Caleb Ochs:type of thing. Yeah, yeah. And those Delta parquet files are really cool.
Brick Thompson:Yeah, those are really cool. There's a thing, it was called Time Machine, something like that. Where Yeah, we're when changes happen to the tables, that delta parquet files, you can actually now go in and say I want to, I want to see exactly the state of this based on some date and some time in the past. And, you know, we've we've built data warehouses for years that allowed you to do that. But it took a lot of careful modeling and writing of DAX to be able to do that. Well.
Caleb Ochs:Yeah, right. The typical scenario there is, I need to I need to keep data as of like month end or something, I don't want to change that, even though in the SIS transactional system, it might you want to be able to say no, this is what we reported on January 31. That's going to be what we want to report on forever now. Right?
Brick Thompson:So you know, so if you're billing historical report, you might say, Alright, I want every month and and now it's very simple to go back and get that without having to have set it up beforehand to write those out to a table somewhere, right? And the built in, there's so many great features coming with this. And it's still in preview. Do you know, I can't remember the date that they may be going GA on this?
Caleb Ochs:I don't think they've announced it yet. You know, it's it's interesting. I was reading this post a couple of weeks ago about someone saying like, you know, when is this going to be generally available? And I'm having a hard time getting people to buy into it, since it's public preview. And, you know, someone replied, like, well, like, what functionally, does it not do well for it, and they didn't really have anything to say. So that the really where the conversation went was? Well, you know, just because there's a label on it means you, you're not going to use it, even though functionally it meets all your specs. But if it didn't have a label, and functionally it didn't meet all your specs, you'd be able to write Yeah, so it's kind of an interesting thing. It's really just the public preview right now is it is just labeled, there are some some nuances kind of on the edges. I think those are going to be there, even when it's generally available, there's still going to be some bugs and kinks getting worked out. Right, just like there was with Power BI. But public preview right now seems usable.
Brick Thompson:Yeah, I agree. So there may be some of this that I got wrong, or that we got wrong. But that's sort of a high level overview of the pieces of fabric as we see them. There are so many details and cool features and things you can do and things that we'll all be learning about how to optimize and sort of do as best practices in this. I'm really excited about it's cool.
Caleb Ochs:Yeah, I think it's gonna be great. I mean, it just fits in really well with where Microsoft's headed with everything. And the the AI stuff is gonna be there and fabric is gonna enable a lot of it right. And it's gonna be pretty cool.
Brick Thompson:Yeah, yeah. As, as they said during the keynote back in May. Data is the fuel that powers AI. And clearly the company Microsoft is really setting yourself up to provide that sort of that conduit that place to have the data and manage the data so that you can enable your AI really well.
Caleb Ochs:Alright man, it's gonna be sweet
Brick Thompson:Alright. Thanks, Caleb.
Caleb Ochs:Thank you