The Real AI Advantage Isn't What You Think

Everybody says they're data-driven.

But according to a Harvard Business Review study, only 7% of enterprises believe their data is actually ready for AI. Not agents. Not advanced machine learning. Just AI.

Everyone wants the outcome. Far fewer have done the years of work required to make AI useful in the first place.

But Matt Pappas has.

As Chief Data Officer at Kiewit, he's spent more than 15 years building production machine learning systems that influence billion-dollar decisions, risk assessments, and project forecasts. His team deployed its first company-wide production algorithm in 2017, years before most organizations had even formalized a data strategy.

This isn't a conversation about chasing the next breakthrough. It's about what "AI-ready" actually means, why data science remains a craft, how Kiewit uses machine learning to make better decisions, and why Matt believes technology should solve business problems instead of going looking for them.

The companies pulling ahead aren't necessarily the ones with the newest models. They're the ones who started taking data seriously long before everyone else had to.

The AI Proving Ground Podcast leverages the deep AI technical and business expertise from within World Wide Technology's one-of-a-kind AI Proving Ground, which provides unrivaled access to the world's leading AI technologies. This unique lab environment accelerates your ability to learn about, test, train and implement AI solutions. 

Learn more about WWT's AI Proving Ground.

The AI Proving Ground is a composable lab environment that features the latest high-performance infrastructure and reference architectures from the world's leading AI companies, such as NVIDIA, Cisco, Dell, F5, AMD, Intel and others.

Developed within our Advanced Technology Center (ATC), this one-of-a-kind lab environment empowers IT teams to evaluate and test AI infrastructure, software and solutions for efficacy, scalability and flexibility — all under one roof. The AI Proving Ground provides visibility into data flows across the entire development pipeline, enabling more informed decision-making while safeguarding production environments. 

Transcript

Only 7% Think They're Ready

SPEAKER_01 0:00

AI is in a super athlete body, but the brain is not there. It's not thinking through all the steps, right? It's still heavily guided through a scientific process by data scientists.

SPEAKER_00 0:11

7%. That's the share of enterprises in a Harvard Business Review study that believe their data is actually ready for AI, not ready for agentic AI workflows, not ready for advanced machine learning, ready for AI full stop, just 7%. And yet every board deck has an AI strategy. Every CIO has been asked to present a plan. Enterprise AI investment is at an all-time high, but something isn't adding up. My guest today has been building production machine learning systems inside one of the largest construction companies in North America for over 15 years. Matt Papitz is chief data officer for Keywit. His team runs more than 50 production models, models that influence billion-dollar decisions, risk assessments, and project forecasts. They deployed their first company-wide production algorithm in 2017, before most enterprises had formalized a data strategy at all. I wanted to know what separates that 7%? What does the foundation actually look like when you've built it right? And that's where we started. This is the AI Proven Ground podcast from Worldwide Technology. Let's jump in.

Everybody Loves Data. Until They Need It.

SPEAKER_00 1:39

So if you were to kind of take your experience with KiaWit, what what were the learning, what were the lessons that you would impart on others to say, and here's how you actually make a data-driven culture, here's how you make an AI-first organization grow? That's a great question.

SPEAKER_01 1:56

Yeah, I would say everybody says they're data-driven, but uh when you get into the inner workings of the latest application that you just purchased and how bad the data model is that underpins it, and the fact that you've got day-to-day and week-to-week failures of data pipelines that seemingly don't get fixed quickly, it's kind of hard to make the case that on any company is truly data-driven. And like I said, I think Keyword is a company of data freaks. Our leadership is freakishly interested in data. Both the prior CEO and the current CEO called me in to have some statistical conversation and a data conversation. So it's not the tone at the top at all that I think where you see sort of the cross-section where companies proclaim to be data driven, but yet, you know, day-to-day, week to week, there's there's all sorts of evidence that they're not at all. So it's how do you how do you get a top-down culture that's bought in sort of effective at the lower ranks of of the data work? And for me, the way that you get good data collection is you provide value from the tools. I think there's been a lot of efforts in QIT where people will go on a data collection journey without providing evidence that the data collection is valuable. I've never seen any of those finish. Uh, so then that would include even like these long data catalog journeys, right? Where you're gonna make the data more accessible, you're gonna have metadata about data, and there's all these, you know, big words people use to really describe like metadata. And I just I think it's it's all about showing value from your projects as you go, and and we've been able to do that, and it's made people even more interested in an already data nerdy culture, right? And how do we get more analytics? And we've continued to see a doubling of investment every three years here in data analytics.

SPEAKER_00 3:49

I am curious.

The Accidental Data Scientist

SPEAKER_00 3:50

So a little bit in the prep, you started in construction finance, you taught yourself code, you built your own planning tools, and eventually you were buying GPUs to train machine learning models. That's not exactly a job description you see on paper. So I'm curious, when did you realize that you accidentally became a data scientist?

SPEAKER_01 4:08

I was actually a data scientist before you could get a degree in data science, is I started to really hone in on project cost forecasts. So I ended up building a Cisco model to forecast a construction project. And just one of the fun parts of Kiwit is you get put in really high levels of responsibility as the company sees fit, not necessarily based upon your age. So we are a meritocracy. So I was the chief financial analyst, self-titled for a $200 million job in my first year of Kiwit, which was really, really fun. So I was presenting the finances and the operational reporting to board members at Kiwit in my first six months at Kiwit.

SPEAKER_00 4:50

Yeah, thrown into the fire right away.

SPEAKER_01 4:52

Yeah, thrown into the fire right away, and begun to see how much of a company of data nerds Keywit is. And so I really got into it as well, even though we were on at that time, this was 2007, we were on a local enterprise platform. So I started doing a lot of local extractions, wrote a bunch of code to help build reports and charts, and for some reason just kept enjoying it to the point where I then went out and got books. And yeah, I rolled out a stat model across all of our power division, I think in my third year, and it got me really popular as an analyst uh within the company. And that really just opened the door because now you got you know bigger data warehouses.

Clean Data Is Overrated

SPEAKER_00 5:34

How does your background, you know, more in the business, more in finance, how does that help shape how you view data and how it should be used within the enterprise setting?

SPEAKER_01 5:43

The way I view it is if you get into a company, like you're increasingly more valuable and less susceptible to any changes that AI may make if you are the expert on the business process of data collection and you're willing to just kind of step into whatever aspect the company needs. I think what I've seen, a lot of data scientists that come in with an expectation of data cleanliness and an expectation of documentation of where the data is and how it got there. And that's just not that's just never going to happen, or the problem would already be solved, right? So I think fundamentally you've got to approach a business and then try and learn about it and the data, the data that really drives it. I mean, we've been, I would say, data driven as early as I've been into QWIT. Keywit has always just been freakishly interested in data and analytics and now applied AI.

SPEAKER_00 6:36

Yeah, and the way you've characterized that is at least as regards to AI and data. And I'm paraphrasing here a bit, but you got in early and now you're ahead because of that freakish interest in the data. But take us back to a little bit of those early days. What did early days look like for you? What type of shape was data in compared to where it is now? And you know, what were some of the fundamental steps that you had to take that put you on the right track to being quote unquote ahead today?

SPEAKER_01 7:04

Well, back in my day, you had to actually like read books on how to write software eventually, right? So I graduated from running stuff locally with Cell Analytics to running things in a very large database, to trying to run things locally with some aspects in a very large database to writing full stack applications. And I you get there as you try to build bigger and bigger data set analytics. And so I just sort of followed that, I followed that path of trying to scale those analytics and provide more powerful answers. But I didn't, I just didn't get into technology with a lens of I want to be able to do XYZ cool new technology process. I just grew where I saw need and it it offered me a great career and excellent opportunities and a great vantage point on what makes the business run. And I kind of got really into who built the coolest thing and the most complex analytical thing out there, and how can I learn from them? And you know, Keywood hired data scientists in early 2010s, and so I mean it was like you know, two magnets attracted the guys that were in our centralized data science group that we had bought, and myself with all that, I had a lot of field knowledge, so we had something to trade, and I just got really into it. So into it that eventually, after becoming a CFO of a smaller company, I wanted to get back into data science now that I knew what data science could provide an enterprise of Kewitz data size.

AI Ready Is Simpler Than You Think

SPEAKER_00 8:43

I was reading a Harvard Business Review piece of research that said I think it was only 7% of enterprises think that their data is actually enterprise or is ready for AI to consume, which is alarmingly low considering how much activity is happening in the AI space. You're talking about how momentum is helping you um articulate the value of data, which is helping you reinvest. Looking back on that journey, are you are you seeing any kind of stepping stones that you took there beyond just the you know create momentum along the way? How do how do you start to actually put data into making it AI ready?

SPEAKER_01 9:20

Yeah, this AI ready concept, I think in it really is discussing like the speed at which your data is available and the context you provide around your data for AI. I feel like uh there's a lot of jargon out there that's overly it's it's just inflated. The what is truly just context about your data. You know, if you had a great data catalog, I would offer that that that metadata about your data would would do a lot for AI. And if you've got it in a cloud accessible warehouse, which companies of keyword size certainly do, like that's a Databricks or a Snowflake, you have that ability to scale and have massive parallelization. So I think it's I think it's kind of an inflated complexity environment right now. And I think it's gonna get a whole lot more simple. And then you're gonna see a lot more progress come from more companies. I've been able to wade through it, and I think I think we're we're just now getting to differing consumption patterns than just traditional human to a user interface. I think we're we're just now getting into a user interface with the human operates to an agent and truly evolving to where an MCP is beneficial. We're seeing a huge shift in consumption patterns coming from, I would call it more ugenic IDEs, cursor, codecs, claude cowork. And so now we're starting to migrate our patterns. And so I would say the way, if you look at how we build those APIs, they're they're obviously an MCP format, and there's a whole lot more to MCP than just understanding the three, what the three letters stand for, and what to us means is like, how are you going to accomplish a secure connection between one of those coding IDs and a tool that is hyperscalable? So not just a local agent and not just you know an open-to-the-internet connection, but truly enterprise grade security and true scalability of your data so it's fast, access to the latest models in a dynamic fashion, and then it's you know, it's good old-fashioned adoption, which which is always a challenge in every company, right? And we are we have a lot of challenge there because we are divided into business units. You know, keyword is a is a really, really strong contractor because we're so diversified, but it's really challenging to be diversified to the extent we are and roll things out. So, you know, AI ready, I think overly complicated. You know, simple things are okay, consumption patterns are changing. We need to make our data speeds fast. I think most people get there. And then it's just the serving of the data. We're starting to have more conversations around like, are we even going to build an app for this data science model, or is it is it just gonna be MCP? Can people make the transition to only a command line? Or do we see that as the pattern of consumption going forward? And and to me, those are interesting conversations because as we move more to like a command line interface, it takes time off these project cycles and and will speed us up. So uh it's made it way more fun, but with that, you know, kind of embarking on a totally

Why Data Science Isn't Going Anywhere

SPEAKER_01 12:40

different question. It's like the expectations of management are so hyperinflated, right? So we're delivering, I would say, probably 30% more than what we did last year, similar amount of people, and and yet we now feel inept, you know, when we deliver a new product because it took you know four or five weeks and they're like, well, this could have been done in a day. And it's like, well, sure, it could have been done in a day, but you know, it's like asking, you know, somebody who's a master of their craft to do it sloppily. You know, we don't, we're not gonna let slop go out. So yeah, there's a lot, I think there's a lot there to that question.

SPEAKER_00 13:14

Yeah. I'm struck by the fact that you you didn't necessarily say anything about cleansing the data or standardizing the data. I mean, that I feel like that's a lot of where I hear clients of ours or guests that we have on the show talk about what you need to do to get AI ready as it relates to data. Is that something that you think you maybe took for granted because of where Kiwit came from with its strong data foundations, or did you do that on purpose and leave that out to the point where you mentioned earlier, you know, if you talk about building it right from the get-go, you might not finish it all?

SPEAKER_01 13:45

I think it's I in my opinion, what it is is I just have a we've treated data as an asset for a very, very long time. So KewIP deployed its first company-wide production algorithm, something that went out to every business unit and changed the business product process fundamentally from the ground up in 2017. So to get to a point where you're literally changing a business process with an algorithm, you have clean, you have figured out how to clean and manage data. So I would say maybe I just, you know, these days, data being an asset and data pipelines, and I don't know, maybe I just forget about them now because they just happen all the time. It's in our culture.

SPEAKER_00 14:23

You know, you've described your own role uh in as it relates to data science as that of a business, uh, a business process counselor. And I'm curious what you think that means. And does the person who carries domain knowledge within the business, what role do they have in kind of training the model so that the model can actually digest what the data says?

SPEAKER_01 14:46

Yeah, I think this is this is where you're gonna get into people misunderstanding, I think, the what applied AI will do in a positive sense to the company, and that they think, okay, well, what a great data scientist is is somebody who could code and write model.fit. And we really, we're really a group that's very aggressively negative on data scientists that don't understand mathematical principles deeply. And so the way I would describe the business role is obviously we want to know what the business knows about the data. We want to present a model that's been built by a data scientist, which means that the actual AI is being steered by data science, but the user can better explain what they want in terms of like the output of the model, how the model will use in their business plan. They can better validate the model going in. You have kind of a shared, trusted ground truth data set because both people can access it now that coding makes it so now that the coding part is so much easier. So I would say, you know, they still fill the same role that they have. There's gonna be far fewer applications. As I said, like I think people will use more of a command line interface and less of a full web GUI. So I think they're just gonna be really empowered by the applied AI not sort of making these long IT projects any longer. And I'm really excited for that. I had no interest in those Uber long website type build projects. So do I think they'll really like will it like truly democratize data science? I would say no, because you still have to understand the scientific process of building a machine learning model that doesn't harm the business. We have probably an inflated uh term we use, we this like do-no-harm principle. It's like a comes from the medical field, and and we really take that seriously. You know, knock on wood, because I'm I'm humble enough to realize we could make an error because I know all of our data pipelines, but we have, you know, probably 50 big products, big, big products. I actually know, because I was kind of going through our roadmap and of those, you got a lot of models out there. Sure, they could make error, but uh, as we kind of roll the results up every year, there's not a year where I felt like we didn't provide a lot of value with the models. I think that's because we really understand what could go wrong in the data pipelines. And we've had a very consistent leadership group over data science. You know, I was around when we first we had our first data scientists hired, and the data scientists that we hired first is still around. So we have a long lineage of applied machine learning leadership here. And I think that makes a huge difference in understanding the scientific process that results in models of high quality. Where I think we're getting into now is we're gonna have I have business users reach out to me and be like, hey, this is what the model put out for this prediction. And I'm like, great, you'll never be able to validate that. And I know all the pitfalls of our data pipelines that we build and how difficult that is to communicate to data scientists. And typically, like a you know, perfect ACT data scientist, they'll come to QA, and it's a year before I can get any trust in what they're building, is absolutely right. And it's not because we don't write it down, it's because we're asking them to solve problems that aren't solved. So when you do that, right, there's all sorts of questions you have to ask and should ask as you go through those model building steps. And you know, I think there's a world where eventually the business has more power, but I think, like I said, it's going to be more on the ground truth control and the usage of model results. Because I truly believe there's a profession of understanding building a model, a data science model. And I think what's lost on people is that that is a very it's a professional skill set that you're going to have to recruit the best of the best to build the best models in your industry. And that and I think like people get it when you know they say, like, hey, I built an AI agent that does XYZ construction process. You know, we don't credit the frontier model with that. We credit the manager over that operations. And so why would we then credit the frontier model with codifying the model that a data scientist steers? And again, I just think it's because people don't understand the process of building a model and how many pitfalls there can be. And the fact that you just you gotta be looking around the corner in terms of will this model generalize into a new market? And and in our company, you know, we're we're very, very diversified and the markets evolved and shift, and our market mixtures have never stayed stable. And not only are we not, we're very, very proud of the fact that we get into new markets with clients that want to take us into new markets as a as a contractor. And then and I think our goal is to build anything better than anybody else, right? As a company, and that's not like a quoted Kiwit thing. It's just what I would say it looks like if you look at our you know, market revenue mixture over a long period of time. So yeah. That's my take on that. And I think we're a long ways from people getting that answer and the nuances of being a data scientist.

The Business Of Better Decisions

SPEAKER_00 20:10

You've built systems across estimating safety, design, scheduling, project cost controls. Of those, any of those use cases that you think best illustrates how data is changing the way that that KiaWit and your teams make good decisions?

SPEAKER_01 20:27

I love the risk management aspects of machine learning data science. I love the decision science pieces of the puzzle. The automation pieces, at least in construction, are a little bit tougher because you know it's a hard, it's a manual labor industry, right? You have like humans with physical tools, not digital tools, installing work. And we do see a future where it's more robotic. But to date, I've enjoyed the risk management and decision-making aspects better. So I like I've really enjoyed the estimating tools we've made. We're operating with a very limited amount of information. And the part that even offers a greater ability for our data science group to add value is the fact that a lot of these jobs, you can have a billion-dollar job that's expected to be bid, like a full price given to a client in six weeks. So, how on earth does somebody make a good decision with a very, very limited time frame without knowing the full breadth of Kewitz experience? And, you know, one of the things that was famously said to me early in my career, which is absolutely true, is we don't learn many new lessons. We just have new people learning old lessons. And that's profoundly true from what I've seen. So I I can I can fully agree with that. And again, that is an effort manager. I'm just not sure who to credit. I don't know if he was the first that uh maybe even when he retires, it's okay to say he's credited with it. And he's still here who told me that. So so yeah, long long story short, I think long. Long journey still yet to come. We're on the the early days of it.

SPEAKER_00 22:03

Decision science is probably not something that a lot of people hear about a lot. And I I love that framing for AI because it's more practical than talking about, oh, we're driving outcomes, but instead we're driving decisions. So maybe extrapolate a little bit more on how you think about decision science. Is that something that you guys came up with on your own? Is that something that's a common industry term? And why is that a good way to look about AI?

SPEAKER_01 22:27

So I'll kind of get into the nitty-gritty here. I I really like the model fitting process and model scoring process. I think that's very interesting. It's interesting to see where the algorithm doesn't get the final answer right and the humans don't get the answer right and try to figure out well, is this a data problem, right? Is it unknowable by a model? Do we have the right statistical model selected that we're fitting? Is there nuances in the data that we need to be filtering out because they're just not helpful data points? Did something happen that just never happens again? So I think that the whole model selection, model scoring is incredibly more complicated than just you know, fit a you know, decision tree model. And uh we've evolved a long ways from that aspect. And I think we kind of went through these going to more complex models, and then there was a time where we kind of went back to more simplistic models, and now we're just like kind of all over the board. It's just been so many years of data science. So when we talk about how effective is the model, uh that's always an aspect of whether it should be used more or less. So uh we have a model that tells a project whether or not the project's gonna lose. I'm not gonna say the percentage, but a percentage of their margin and profitability, which isn't just a key web problem, that's an owner problem too, because if you're not making your original estimate, you're probably not behind, you're probably not making your schedule. You're probably, you know, short on time, I guess, as a whole. So it can lead to other issues. And used the model, user group B didn't use the model, and here's how much better user group A did. And so we do that every year. So I would call it like impact. And the part that I really enjoy is when people come to us with novel problems. They're like, hey, Kiba got offered this opportunity to build a job with 50 million hours of pipe. What would the model think with a quantity of 50 million? And so you're able to kind of step back and think through the coefficients of a model or parameters of a model. And that's what I really that's uh I enjoy those conversations a lot where it's like, hey, we're gonna take a very large amount of money and put it on the table, and it's at risk if we don't do this well. We also really want to build this project because when you build novel projects, you as long as your company continues, and all not all contractors keep continuing as they take on work that's way sort of over their skis. But as you as I can help make more risk-adjusted decisions with decision with data science models, I love that part. Um and I've get I get sort of thrown the other side too, where people are like, you know, can you help us in this very niche operation that we've done one other time in the history of QA? Where there's like really not enough data to have a data scientist work on it. And I love that too, because I'm like, all right, well, what other data could we have could we have used? And one of our couple of our data scientists, and again, I'm not, I don't think I'm making this term up either. They say, like, any you don't have to be a very good data scientist to make a model with a million data points, but when you have you know five data points and you want to build something that has any utility, how do you do that? That's when you that's when you truly are a more pedigreed data scientist, because it's how do I apply all this extreme mathematics into my business, right? The math plan is what I call it. So as far as you know, decision making and and how we do that, I kind of view those three trances, you know, model scoring for what what's the effect of the tool, effectiveness, impact on the business, users, non-users, and then I love the hey, what would your model say has no data in this area? Or if it's you know something we've just never seen before, how how how do you think you know we should, I'll call it mathematically bid this job without anybody's feelings involved? And I get to do all of it, and I I like that part of the job a lot.

Agents Are Here. Now What?

SPEAKER_00 26:54

We're pretty consistently seeing validation that we're moving into this uh agentic era, agents popping up all over the place. Certainly, you know, maybe last year it wasn't as effective this year, it squarely surely is. When I talked to you about kind of the current state of AI and agentic, you were saying it's a little bit of the Wild West. And you know, more people now have more tools at their disposal to do all these things. So, how do you think about the future with Kiwit as it relates to agents, having users creating agents all over the place? What does that do from a risk standpoint or a data standpoint, um, so on and so forth?

SPEAKER_01 27:30

Well, yeah, I mean, as a Keywid employee, I'm fearful for the future in terms of the risk that's being created. The attack plane is undoubtedly larger. People have ill they have ill intent to basically get hold of our data, personate users. I think we're seeing a rise in cyber attacks unlike any other time in my career. So that part of it was a real negative for me in terms of how will the applied AI agentic era look and feel for businesses like ours. The part that I love is like I was talking about earlier, the delivery time frames are expediting because you're able to code with AI. And I'm not talking about AI slop. Yes, we have lots of those new employees that do that, and you got to look out for it. But you know, we have a lot of we have a pretty tenured staff here, so we're not sort of like gonna start kicking out inferior code because it's you know, whatever, cheaper. But and that's in a centralized capacity. And so what I what I am seeing though that is gonna be outside of a faster delivery that's really powerful is I think there will be less workflow software. I think it's like the the SaaS pocalypse. I don't think it'll be a full SaaS pocalypse. I think people will sort of shift to the fact that like it's easier to just sort of have an agent operate like a software package that like you trust and has checks in it and it's ultra deterministic than to like just sort of hope AI creates the storage schema and manages the data well. To me, that's that's pretty pie in the sky for someone to expect, at least in the short term. And I might be willing to say you always need somebody to manage the storage and think through how these complex systems sort of tie in together for data storage. But I it is gonna be a rocket. And the part that I like really get excited about is like I think more of the tools that I was dreaming of building in 10 years are gonna pull forward to you know one to two years because it's faster at the whole on the whole cycle, right? Adoption is faster because they have these coding tools, builds are faster because we have the coding tools, and then I think it's also it gives us like a list of things to try. The whole like generative and thinking part of it is is a lot more fun. So I feel like it's non-confrontational to say to a data scientist, hey, here are the other 10 things that AI thinks you can try. What are your thoughts? And we can both like sort of laugh at the seven out of ten, but then like you know, the first top three, you're like, well, those aren't all that dumb, you know. What did why didn't we do that? And I think for the younger data science crowd, that's that's a powerful thing for them to come in and learn. The other thing that's really cool is you can sort of point AI at someone else's work and it can train them on how it was built. And that's really cool. I think that that's a really fun, fun thing. So I just think like we're in this rive of rise of ephemeral software building, and it's way more fun if your goal in life is not like, yeah, I really I don't really want to not really sort of tied or hitched to any one technology. Kind of like I said, when I started my career, I just kind of went where the technology, where I needed to to get to where I wanted to go with data analytics, and I'm still in the same boat of, you know, so things that make it faster are kind of fun for me, minus the hyperinflated expectation of you know, it not harming the company with the end of that process, right? It's like I agree with you, this goes really fast. I mean, in a lot of cases, AI is in a super athlete body, but the brain is is not there, it's not thinking through all the steps, right? It's still heavily guided through a scientific process by data scientists. So, but overall, overall

AI Learns Faster Than We Do

SPEAKER_01 31:25

positive.

SPEAKER_00 31:25

So expand on that point. What do you mean by pointing an AI at somebody's work? What can we learn from that?

SPEAKER_01 31:32

So I think within 100 lines of code, we have some of our most sophisticated models, right? And so what you can do with AI is you can go through the experimentation cycle. And I know there are I'm I'm just trying to keep the words simple because I think way too many people are making things more complicated than they truly are. But if you go through an experimentation cycle and you do a good job of documenting out your work, and I really would credit like Wes McKinney with my viewpoint on that, because Wes is very big on like how do we build good data science modeling platforms. You know, he's committed, he's contributed a lot to Positron and and we're a Python shop predominantly. We also have R. And I think that if you can document the experiments you've done that landed you on a certain type of model, and then you can have somebody ask AI to take them through the journey of model building, and then they can learn about all the data and they can say things like, why did they remove you know this sample from the model? And the thing I really want to do is I want to take all of our models, all their training populations, and I want to compare them and say, like, AI, what should this project have learned from this training set and this other project? Because, you know, we're a project-based company, and so there are samples that share a lot of common traits, to say the least. And I think that would offer our data scientists absolute huge dose of humility because we have so many models, right? And there's just no way to communicate out all of the population changes through every one of those models. But every every time we retrain a model, every time we try to train a different type of algorithm on a model, we learn something. And so it's like, how do you how do you scale those learnings across the entire, you know, I would call it analytical enterprise? And we have way more data scientists also, right? Like I said, it takes a year to get one up to speed. And so, you know, the more of these, I would call it data science foundations you can build, and AI can sort of help help purify your training set, help you understand why why certain samples are in or out, help you understand the type of model that was selected and what other models were tried. I think as we can make more of that data accessible, it's going to be a way more powerful amount of work getting built. Because you'll be able to take a strong, mathematically talented employee and put them right to work. That's how I look at it.

The Next Generation Of Data Teams

SPEAKER_00 34:10

Yeah, I mean, recognizing the fact that employees have more and more access to data, to tooling, to models than ever before. What role, if any, do you see data scientists or data teams having in governing all of this to make sure that that rocket doesn't get off kilter and just go somewhere else?

SPEAKER_01 34:27

Well, so I would interpret it for the modern enterprise as the people that can just code, right? And they're terrible communicators, they're they're just it's not gonna be a pretty future. I think that's the part that's really gonna come down. Because like I said earlier, I think is you got somebody with a lot of domain expertise and operating a business unit even better, they're gonna, they're gonna want to have insight into the data itself. And as they get smarter about the data, which will happen because you can just ask the question instead of having to know how to write software, it does the software piece of it for you and it brings it back right into your command line, right? Or into your user interface. And so I think I think the roles are gonna change for the better. Because I never liked that aspect of it anyway, where I had to hire somebody that wasn't a good teammate, and then find out like they are the only ones that know this process, and then like try to cross-train them, and then try to replace them with them. I'm like, hey, this is just way simpler to manage because seasoned data scientists can wear more hats. The business unit has a greater purview to what the data scientist does, and you just don't need as many of these what I call like code-only employees. And so I just I think it's gonna you're gonna get more to a foundational explanation, expectation of data science and analytical staff and data teams where they understand the business. Like technology is a means to an analytical ends instead of technology trying to find a problem. And so I think we a lot of a lot of technologists try to use technology to find a problem instead of you know, use technology to solve a problem.

The Companies Pulling Ahead

SPEAKER_00 36:10

Well, Matt, this was an insightful conversation. Um, very much appreciate you taking the time out of your busy schedule to uh to share your wisdom and insight and just kind of generally where the uh where the industry is going. So thank you for joining today. Okay, thanks to Matt for joining us today. The thing that stayed with me from our conversation, Hewitt didn't get ahead of AI by moving fast when the pressure hit. They got ahead because 15 years ago, when nobody was calling it AI readiness, they decided data was worth caring about, worth measuring, worth building a culture around. Matt described it as treating data like an asset. And that's it. That's the whole playbook. The companies that understand that early are the ones writing the rules now. Everyone else is trying to catch up. This episode of the AI Proving Ground podcast was co-produced by Nas Baker and Kara Kuhn. Our audio and video engineer is John Knoblock. My name is Brian Felt. Thanks for listening. See you next time.