AI Governance Needs a Dungeon Master

David Wendt spent 30 years building models for Fortune 100 companies before stepping away from the keyboard entirely to focus on AI governance. He also happens to be a Dungeon Master, and it turns out those two things have more in common than you'd think. 

Mo and David get into red teaming, what good metrics look like in a probabilistic system, why you should probably stop measuring AI and start measuring outcomes, and what we actually owe the workers whose roles are being quietly restructured. 

🔗 Podcast: https://alice.io/podcast

Follow the show so you don’t miss the next episode.
New episodes every two weeks. Stay curious.

Transcript

SPEAKER_01 0:00

We as a data science technologist community, we have a belief that if we can predict X better, that will lead to more sales, whatever it is. If we can provide a better layout for the warehouses, if we can tell them when they're going to have to restock the front bins, that's going to lead to more sales or reduce costs or whatever. It's sometimes drawing the connection and drawing the line between the metrics and the dollars that gets hairy.

SPEAKER_00 0:29

If AI has ever made you stop and think, wait, what is happening? You're not alone. I'm Mo and I'm a security researcher asking the same questions. On Curiouser and Curiouser, we're having open conversations with experts, researchers, and leaders working at the edge of this space, talking through how AI is taking shape, what's shifting, and how people inside the work are thinking about it as it happens. So join us and listen in as the conversation takes shape. Hello, hello, and welcome back to Curiouser and Curiouser. I'm Mo, and today we have David Wendt with a silent D from Sherwin Williams. He's currently there leading AI governance and innovation. However, I'm more excited about his history as a Dungeons and Dragons dungeon master. I will let him uh introduce himself, uh, but I just had to throw that out there. So you didn't forget to tell us more about that.

SPEAKER_01 1:26

Yeah, thank you, Mo. Um, yeah, so I have been thinking about the math that is AI governance for about 40 years. That led to me getting a PhD in statistics. That led to me spending 30 years on keyboard as a lone wolf data scientist, a mad wolf data scientist, mad scientist, data scientist, uh, director. And but recently, in the last two years, I've I've stepped entirely away from the keyboard to really focus on uh the space of generative AI and specifically the governance of it. Uh it is a new, different way of thinking about things. And um I find, you know, an incredible amount of enjoyment from it. And uh as Mo said, along the way, during that 30 years, I've also been a dungeon master. I've written for a number of gaming books. And as I look back over both my career uh as a data scientist and as a dungeon master, I think there's a lot both can learn from one another.

What D&D and AI Governance Have in Common

SPEAKER_00 2:34

Yeah. Um, I'd actually like to hear a little bit more about that. I know that there's uh a bunch of different um, like in education, right? Um, and um psychiatry, there's or psychology, there's this like correlation between like using stories and being able to actually use the tabletop RPG to kind of work through trauma. Um that said, I don't think as much as some people would say our field is traumatic, I don't know, I don't see too many places where we can kind of use um Dungeons and Dragons to kind of draw analogies. However, I do know that you're working on a DD governance framework. So I think that's actually really interesting. Um, because when you think about it, I I play Call of Cthulhu, but I think uh Dungeon Master to Dungeon Keeper, we can kind of be real. At the end of the day, we have final say and final judgment over um like the rules. So if we need to justify a decision that we're making um in the game in the moment for the player's experience, we do that. Uh in I guess when you look at it from a security program or actually, you know, implementing policy, uh, it feels like we have a little bit of the same kind of final say, like flexibility that we need to play around with. Um so it's pretty interesting, um, especially in a field where we need to have deterministic proof when we make these decisions. So I would love to hear more about how you've applied DD to your governance framework.

SPEAKER_01 4:03

You got the right starting point, right? I grew up in a household, we played games, we played card games, we played board games. And yes, cards and dice and board games have randomness to them. You pretty much, when you played, say, Euchre or pinnacle with with regular players, you knew how things were going to play out. There was not a whole lot of randomness unless you were like me and you're like, well, what'll happen if I try this? But then discovered Dungeons and Dragons. And it shifted the paradigm from deterministic or largely deterministic to largely non-deterministic, um, leaving a lot of randomness, a lot of interpretation on the table. Um and I think that's the first parallel to generative AI and AI governance is we've had to shift the paradigm. We can put rules down, we can talk about process, but we're inherently in a non-deterministic world now. And so those rules and processes have to be more like a guideline than uh strict rulings. Um, the second thing I would say is really around, I've learned so much about communication. Um, you'd spent 30 years uh as a as a dungeon master, and periodically you get this idea, I'll have this great secret and I'll surprise my players with it. And more often than not, that collapses, falls flat on its face because they're not prepared to be in that surprise with you. It's the same space, right? It's the same thinking. Must communicate constantly uh with across the organization, not so much that uh they are gonna form the rules with us just so much that they understand the rules and they feel that we've been talking to them. Um and that's been realizing that coming into this role has been has been huge. Um and then really I think the third one, and you touched on this a little bit, is uh, you know, role-playing games, Dungeons Dragons, call Cthulhu, there are books after books after books printed, and you can have a pile of books as tall as I am, um, all kinds of rules and all kinds of stats and all kinds of numbers and all kinds of whatever. In the end, at least at this point in my my game master career, I don't look at those books anymore at the table. I might check something afterwards beforehand, but those are the guidelines. Those are the rules that are written down. So if we have a disagreement at the table, if if we're not quite sure how to play something out, we have something to reference. And that ultimately is what you said is you know, I really talk internally about we we record our processes, we record our rules so we can be intentional when we break them, right? In in DD, at least at my tables, there's something called the rule of cool. And and basically that is okay, you as a player have just decided, described something so cool that I'm gonna give you advantage because I don't want you to fail. I want that that coolness to us to to emerge in play. Absolutely the same thing here. Uh, we're having conversations right now about Replet and who do we release that to? And it's it's this balance of like, well, if we give it to too many people, it'll get crazy. I'm like, yeah. But think of the amazing things they can create that we wouldn't be thinking of by ourselves.

SPEAKER_00 7:48

Yeah. So just for like folks that don't know what Replit is, they actually started as just a like a small, smaller company that was kind of just like doing a lot of like code development, right? It was just how to deploy software and how to um kind of kind of manage it, right? Um, and they've kind of expanded those offerings to like full-on application and site development and deployment um with AI, which is the really, really cool part. Actually, there's there's one thing that you mentioned that was the rule of cool, right? Um, I feel like that has like, again, you said it before, it has like so much good application here. Um, and then you again, you you did mention something about surprise. Um, to tie all of that together, I think professionally, and even even as a DM, I think we can both relate to this. We don't, well, my players would say that I am out there to make sure that they lose horribly. And it is the the coolest, horrific ending to their character as possible. But that's the actual opposite, right? It's really to build an experience that is true to the world, but also feels good for them. So what I try to do is make sure that they understand

When Security Feels Like an Inhibitor

SPEAKER_00 9:03

the constraints of the world and the limits of the magic system and all of these kinds of things so that when they go to do something, they understand the real implications of it. And in a way, that's like making them smarter and making them more aware and giving them the information that they need. Um, I think we're trying to do the same thing when it comes to AI governance, where we're trying to keep everyone um like inform them of the uncomfortable reality and some of these like real limitations. Um, but at the same time, like I think we've lost this like piece of um awareness that has kind of made our users feel a little bit um, I don't want to say dumber, but I feel like less informed about the decisions they make when it comes to AI because of how fast and easy it is to use. Um so I guess on all of that, like again, you can't always predict what these systems are doing or understand all the you know the processes that take place in the in the black box. You're only there for like the the ending piece. So how do you feel like um I guess how does that translation process now work? Or how has it fundamentally changed now that you know our partners are totally different?

SPEAKER_01 10:18

Yeah, they're coming from a different perspective. It's deterministic to non-deterministic, right? I think part of the challenges we face is, and it may be because because they don't feel informed, a lot of our partners are looking to the media, looking to marketing to say, oh, that's a cool thing we could do. Can we do that here? And that's okay. Um, but it also causes us some challenge because marketers tend to talk about things before things are done or ready. Uh commercials don't talk about all the data structure that needs to go in underneath. You know, a lot of the things that have always been true are just as true as before. Um so that's a challenge. But, you know, I think there's a truism, and it's what I try to teach, you know, junior data scientists that I work with. Our partners know we're smart. Our partners know what we know what we're doing. So it is not our job to go into a room or get on a video call with them and prove how smart we are. Our role is to help them feel smart. And that's the really hard part. That's where the translation comes in. Can I, and again, we were talking about it earlier today. Uh uh, you know, people want rag for everything, and and maybe sometimes they just want data extraction from a document. You know, how do we start to shift the conversation from words, some of which they've heard us using, but words they've heard elsewhere, to what's the real use case? What is why do you want to pull data out of this? Is it to put it into a database? Is it to have a conversation with your documents? Is it, you know, and then let the folks who are professional at it figure out the best solution for you or a good solution for you. Um but that's tough. That's tough. You know, a lot of egos have to get out of the way to have that conversation. First of all, the the ego of the technologist, but also the ego, you're often talking to senior leaders. And to say, well, you're using that wrong is not always a career help. You know, sometimes that's a little bit of an inhibitor. So, you know, really building relationships, having that communication, and translating from the technical how it works to the uh more touchy-feely, what do you need from it?

SPEAKER_00 12:50

Yeah. Um, you said uh you said uh something along the lines of um like security kind of being an inhibitor, right? Like kind of slowing things down, kind of stalling. And I have a lot of friends who would say they feel the same exact pressure around that, right? Um, but I think it's because of how reactive security teams have been, um, just overall, especially like I would say, especially GRC teams. I don't think that there's ever been a time where like I've worked with a GRC professional who wasn't really great at their job. Like ironically, like I would love to say that someone I've worked with was horrible. I've worked with some great people and I've been fortunate to. Um, but always, um, it always feels like they are one step behind because of the stuff that they have to work with. So I'm wondering, like, you know, it's it's always responding to regulation or incidents or pressure, right? So I wish there was some way to create some space for um GRC to kind of, or just anyone in security, really, like having more space to kind of make these decisions. And in some cases, uh let's just imagine having like a really well-funded security program, right? Like, what do you think could be built in like one of these kind of environments that would enable that space to be um had?

SPEAKER_01 14:23

The one thing I'm working on on putting in place is uh generative AI dedicated red team. Um, red teaming being, you know, essentially people you bring in to try and break your stuff before bad actors do. Um and I think that's really important. The research I've done says uh something along the lines of a single breach can cost six million dollars. And you know, for a tenth of that, you could put in a pretty solid team. You know, the the the

Making Space for GRC and Security Teams

SPEAKER_01 14:59

challenge you're facing, and and I want to dive into the the time it takes, is not new, right? And it it goes into it almost strikes me back to Dungeons and Dragons and really the origins of the of the game. You know, Dungeons Dragons started as a war game, started as a simulation of historical battles. Well, those historical battles are now fought on the digital battlefield. And what happens is somebody gets a new weapon, people respond to build a better defensive wall, which drives people to build a better weapon. And so this is just happening faster and faster. And if you then mix replit or things like it into the game, you know, it can happen even faster than than before. So what you need is a combination of, you know, and and this is this is the truth going to be for throughout AI generation for generative AI for a long time, you know, an AI tool that is running sort of constantly looking for abnormalities, um, and a human team, preferably, that is actively pushing the boundaries and trying to break other things. Um and that's that's where the investment, I mean, it it can't be one or the other, it has to be both. But you said something, you know, about giving our our our risk teams, and and I would extend this to a lot of our technology teams, giving them space. Um, you know, the early one of the early uses for for generative AI was efficiency, right? Right, write emails faster, write papers faster now, code faster. Um, and I you know, I had a lot of leaders who were like, yeah, but that's not savings for us. We don't want them going home early. And for a while it's like, okay, that's a little close-minded. But I agree. Actually, they shouldn't go home early. Do those things, do them an accelerated way. And that time you've saved, now take the breather you need to take to think about what's next and how to be prepared for the next thing, rather than running like a bull through multiple China shops, you know, just trying to kick things out as you go.

SPEAKER_00 17:21

You know, it's it's not a position that I envy. Um, although like as a builder as well, it feels like if we slow down at all, um, we're already behind, right? So it's almost like you either you're either the bull in the the China shop or you're like you're not even in a storefront, like you're not even there.

SPEAKER_01 17:41

There's a couple of response to that. First of all, I think we all think we're further behind than we are. Um, and every time I I talk to folks outside, you know, and I compare what we're doing to other folks, at least in our peer set, we're we're right right up in the front front grouping, right? But it doesn't feel like it. And I think that's the bigger part. It's the feeling. You know, things are changing so fast. How could we ever be at the forefront? And so, you know, I I don't think we should. I, you know, maybe there are some companies that should be, but companies in manufacturing, companies in sales, companies, so many companies, they don't need to crack the bleeding edge. They need to say, hey, this function in the company uh needs help. You know, we need to distribute our our products better, or we need to speed up our time to failure in our research areas, or uh had this conversation this week. You know, we just need someone to act as the intern as we did review our tax law stuff. Have something, someone go and review that. That's a great use for generative AI. Um, ironically, our legal department said they would never use generative AI, and now they're asking for it. But we'll we'll that's for another another discussion. But uh yeah, it's I I think honestly, we have to give ourselves grace. I think we have to say, I am confident enough in my ability and in my teams to take the time to breathe to do it right. You know, or in other words, you know, and I heard this early in my career slow down to go fast. And I thought that was ridiculous at the time, and now I think it is absolutely brilliant. Um, I have shifted from bouncing from task to task to task to task to stopping working with my my generative AI to identify what's the next most valuable thing I can be working on. You know, and and I think maybe not directly, but I think that can apply to a lot of folks. What is the next most important thing I can be doing?

SPEAKER_00 20:12

Yeah. And you touched on something really important. Um when you mentioned kind of having an intern 24-7, right? In the same kind of idea frame as um as that, you take the what you mentioned about, you know, looking for a red teaming kind of uh like uh activity in your organization. I think that is actually like one of the killer use cases for testing, which is like awesome, right? It's like having being able to do continuous testing, continuous governance, um, 24-7 pen testing to like an extent, right? Like, for example, I can imagine like even like a chat bot, right? Imagine a chatbot is just sitting in a test environment and like all day, every day, 24-7, when your red tech, when your red team, your human red team is sleeping,

What Green, Yellow, Red Looks Like in a Probabilistic System

SPEAKER_00 21:01

it's consist consistently getting harassed by like an an AI on the other side that is just constantly giving you a flywheel of um attack response. Uh, you get to go and look at this over, you look at the telemetry, you see how things are changing. Um, you kind of make it a self-improvement mechanism, even, right? Um, I would love to know kind of what, like as you're looking at things like continuous security solutions, including red teaming, what that kind of looks like for you and how you see implementing it into your security strategy.

SPEAKER_01 21:37

That is a really powerful question. Um, and it's one of those where I know it will work for me. I don't know if it works for anyone else, right? I despite being 30 years in hands on keyboard, statistician, data scientist, I'm an incredibly visual person. And so what I would be looking for, and it's it's how I describe It over two years ago when I took this role. I want a dashboard that I can come in on Monday morning or whenever, turn it on and see where things are yellow, or heaven forbid, where things are red, and being able to immediately reach out to the people who are responsible for those areas to make sure they know that's there. You know, to be able to reach out to that team and say, you know, did we change something or are we facing a new set of attacks? That becomes powerful. Back to communication, relationship, et cetera. Now, the thing, and this is the crazy part, right? I don't want that just for me. I want that for not only me and my team, but all the way up. I want my CEO to have the confidence that they can go in and look at any point, whether it's a specific project or just in general and know what's going on. Now, I say that knowing that no executive will ever do that more than once, but I want that there for them. I want them to have that confidence that if they had to, if for some reason I got hit by a bus, they could still get the same information.

SPEAKER_00 23:23

And the good thing is the build versus buy argument has only gotten better. So, like when you say, like, oh, you know, I know it works for me, but it might not work for someone else. Well, guess what? You're currently in an era where like more teams are actually building than buying. And you can now build something that is really well tailored for your security program. Now, let's take it up a notch, right? Um, you mentioned that it's really difficult to um to figure you like you just want to see green, yellow, red, right? You want really easy signals. But when we've got non-determinism in the process, right? Like it complicates testing, debugging is difficult, evaluation's hard, right? Um, what if you can't reproduce the prior model behavior? What if you can't find the results that you got like five minutes ago? Um, you know, like what do you do, right? So, like, how do you know that your AI systems over time are actually getting better or worse? What does improvement even mean now, right? Like all of these different things I think are really challenging to like kind of understand. So, from your perspective, what does green, yellow, red even look like?

SPEAKER_01 24:32

You know, you're you need to take it out of the moment. Um, because at any given moment, right, ultimately generative AI models are probability models, full stop. They're really fancy ones, but in the end, they're probability models. If you take that into account, then if I'm looking at any given property, whether it's accuracy, whether it's latency, whether it's whatever, if I see a consistent behavior uh that is positive, that's easy, right? That's green. Then you have this dimension thing, right? You might see a spike that's bad once, but if it's only bad once, it's probably not a red, it might be a yellow, depending on the the importance of that system. But if I see spikes that are happening regularly, even moderate spikes, that definitely is pushing me up into yellow. And if I start to see consistent, and I'm waving my hands like a madman over here, you know, high-level spikes, that becomes red. And so it's it's to me, you're relying on the laws of probability, which says if it's a fluke, you're not going to see a repetition right away. The other part of that, then, is how conservative do you want to be? I have spent most of my career in very conservative companies. So anytime there is a spike, anytime it the chatbot is giving a slightly wrong answer, we want to be aware of it. So that's definitely a yellow, if not a red. Um, there are other companies not as conservative. They may be able to run it with the wing it, go, oh, we got some spikes. But overall, yeah, it's not too bad. It's getting what's us what we want. You know, you know, it's making sales per, maybe not sales per, it's predicting who might buy from us. If it hits 80% of the time, that's better than we get with cold calling. That's okay, right? And so none of these, none of these metrics, and it's been true forever. No metric exists in a back in a vacuum. The number of people who said, well, is that is is 57% good? You know, well, was is it better than you used to do? Did you used to do 50%? That's good. If you used to do 60%, not so good. And and so it everything's got to be dealt with in context, and especially when we get to non-deterministic, because we're not in raw chaos. As much as some people want to say, we're not in raw chaos, everything is still predictable. Um it's just a lot more variable than people have been used to.

SPEAKER_00 27:30

Yeah, I was gonna say, like, um, you know, even if you look at like some standards like salsa for like open source and like um you look at MITRE, right? And you look at the MITRE, especially the MITRE AI maturity one, right? That is all based in even NIST. I mean, you could even apply NIST to this, right? Um, but like you they're all maturity frameworks in a sense, where you can kind of set where you are and then see exactly what steps you need to get to the next level of maturity. And it doesn't mean that like if you move from like a level one to a level two or right, it doesn't mean that like, oh, you're bad at level one. It just means, oh, this is where your organization's at today. This is like the bare minimum. And like that's if that's sufficient today, then like this is how you move forward, right? And you know, like depending on the organization, you actually have different priorities. But let's let's like go and look at yours, for example, right? There's a lot of product development and customer engagement, um, supply chain, sustainability, you have all these different priorities that exist through throughout um, you know, just product and um the offerings that you've got, right? So, what does that look like in terms of uh when you need to actually apply some of this stuff? Because these frameworks were not meant to cover um, I think a multifaceted organization that's like, oh, you do very heavy manufacturing and supply chain, oh, but you also have this like really strong technology customer experience piece. Like, do are you applying it separately and for different organizations, or are you more trying to find a one shoe fits all good enough and then let you know figure it out uh on the nuances for everyone?

SPEAKER_01 29:17

It is very easy at the top of the house. We sell paint. The more dollars we bring in and the more gallons of paint we sell, the happier our CEO and and the street are. That's very, very easy. At the bottom, at the most detailed, it's pretty easy. Um, you know, I I've sat down with our data scientists and said, look, I need stuff in these seven categories. I need you to start logging in. And I got less grumbling than I expected, which was lovely. Um but you know, the response is, well, but this accuracy measure doesn't work for me and the things I do, um, can I use a different one? Yes, right? There are categories of things. It's the middle ground that is really, really difficult. You know, we we as a data science technologist community, we have a belief that if we can predict X better, that will lead to more sales. Whatever it is, right? If we can provide a better layout for the warehouses, if we can tell them when they're gonna have to restock the the front bins, that's gonna lead to more sales or reduce costs or whatever. It's sometimes drawing the connection and drawing the line between the metrics and the dollars that gets hairy. Um and it's also hairy because uh is it the model that made it better? Is it the people who reacted to the model that made it better? Were there other features or things changed at the time? Everyone wants a piece of that improvement, so that's a real challenge. Um so I haven't really answered your question yet, but I'm getting there. The the realization I've had over the last couple of months is this we should not be measuring AI. Full stop. We should be measuring the use cases that we are changing that happen to leverage AI. So if our big focus is raw material pricing, that's where we're thinking we're gonna make our bang for our buck this year, making stuff up here. Um then I don't care what I do. You use traditional AI, you use generative AI, you use a hybrid approach, you throw darts at the board. I'm looking for more effective pricing. Simple to simplify, say, I'm looking for lower prices on our raw materials, which then lead to better margin, et cetera, down the line. But nothing in there is AI specific. And so that that is

Stop Measuring AI. Measure the Use Case.

SPEAKER_01 32:13

the shift that I'm trying to bring for that middle section is I I don't I don't really care what model you're using. I care if you're getting the results you are after. Because at that point, the they the the the client, the customer, the partner, they can own what's next, right? We provided you a model, we got our our our our raw material prices down by 10% and announced that up. If you're a good partner, they're gonna take you with you. You don't need to say, we own this half of that. And so that's really where I'm gonna be pushing um you know, my teammates, my peers, the younger data scientists to really be thinking not about what we do. Again, back to what I said earlier. I know you're smart. I know you can make good models. Your job is to provide the clients what they need and make them look good.

SPEAKER_00 33:15

The problem that you're describing traditionally is a data literacy problem, right? Because now we're saying, hey, we shouldn't be judging these models just to say that we're using AI, right? Instead, we have this outcome. And how much of this outcome can we attribute to the changes that we've made, like with AI, or at least like how have we used AI to kind of improve this outcome, right? So has it made us 10% better? Has it made us 20% better? You know? And then kind of we go down from there, like, oh, this is the exact use case that we've used versus like this other use case, which we just threw threw into production, right? Like use case A is actually better than B. Um, and then you know, you do all the analysis because there's also a cost to AI, right? Which is a whole other episode. Um, but it's kind of like, oh, like how much does it actually cost to run these models 24-7, right? Um, are we running it in infrastructure we own? Yada, yada, yada, right? Um so I'm wondering if data literacy or at least literacy is the right metaphor anymore, right? Um, because again, I think literacy, when I think about it, I think about a book, right? But books don't change words every time you look at the same page. Um, in this case, are we trying to teach people to be really literate about something that's fundamentally changing all the time? And in that case, maybe harder to really um like the same thing you knew on day one is now different on day two, and the metrics may actually be describing something entirely different.

SPEAKER_01 34:52

So the funny thing is, is uh I have in the back of my mind uh to leverage technology to write a book where the words do change from on the page from time from read to read. Um so everything's possible, but but your your point is is very well taken. I have a number of conversations, regular conversations. Uh we're fortunate to have an AI literacy team dedicated to this space. Um and I have a lot of conversations with them as a former educator, as a former professor. I, you know, I have thoughts. And I I just I constantly have to urge them to think outside of the traditional teaching methods, right? We can't, if this isn't Excel, this isn't if you put equals if parentheses, this thing, you get the same answer every time, right? That's not where we are. We're we're in a space, as you say, that is constantly changing, that the answer you get will be different than the answer I get if we put the same thing into the same chat engine. Um and so we can't do that. And so, you know, the things I'm urging, and I and I both urge our literacy folks, and I've had the opportunity to talk to uh to high school students and high school faculty, you know, I want you to start thinking about critical thinking. I want you to start thinking about problem solving, puzzle solving, I want you to think about these broader, more intelligence driven, creative-driven skills. Um intelligence isn't the right word, but cerebral, that's the word I was looking for. Because if I am thinking about the problem holistically, then I will bring in uh concerns about cost. I will bring in concerns about this metric, um which is another reason to not really worry about measuring the the AI, but measuring the thing you're trying to do with the AI. Um but that's tough. We're we're not set up in this country to teach people how to think critically, how to um, you know, solve puzzles. Uh and it's it's really unfortunate. Uh and it's gonna be a challenge. Now the good news is uh at least in the greater greater Cincinnati, Cleveland area, we've got some great teachers and some fantastic students who are learning fast and adapting quickly. Uh so I actually have very high hopes. I think it's going to be my, you know, people of my generation and and and the generation or two following that are probably really going to struggle the most here. Um not unlike, not unlike, and I'm I'm shifting a little bit, I understand, when robotics came to the auto industry, right? Yes, some jobs were lost, but mostly jobs were transformed. Um and they had to think differently instead of, you know, I can do this and I can pull that and I can do the other thing. I now had to think, need to think about how to make the robotic piece do what I needed to do so that the next guy is ready for that piece. Um But yeah, how do we how do come and bring it back full circle? How do we teach people about this stuff? I I think we teach, I think we teach with story, I think we teach with allegory, um, really going back to the beginning. I think we teach with what ifs, you know, role-playing. What if? I'm I'm noodling, it's not quite done, but I'm hoping by the end of the summer uh to have essentially a role-playing experience that I can take to conferences with me and have a half dozen people sit around a table and basically play through a governance issue. Um, hopefully in a little more fun than than the real thing, but but to have that experience uh available to them, I think would be helpful.

SPEAKER_00 39:19

I actually want to maybe push you to go a little bit deeper too, right? Um, because you spent most of your your career helping people make sense of data and technology, right? And like you said, the people who are gonna struggle the most are actually not the younger generation. I think the younger generation is struggling right now because let's be frank, we've created a world where they don't have opportunity. And that's because we've decided that entry-level jobs, um, we're gonna replace it with AI, or we're gonna replace it with a couple of agents. And as many, as many times or as much as we'd like to say, we're not doing this, we're not replacing humans, you are, we are going to replace humans. We are taking away tasks that are these like bundle of tasks that would make our interns, these bundle of tasks that make your level ones, um your IC ones, uh, whatever your organization calls them. On every task that they do, you we're automating in some way, shape, or form. Review these these pull requests that are, you know, that are really basic. Cool, we can do that with AI now. Guess what? We're not gonna pay an intern another 10 hours to go over pull requests and PRs and bug tests, right? So we are slowly but surely removing the opportunities for them to learn. And some organizations have caught on and said, you know what, we're eventually going to need people because of turnover, right? So yeah, we need to keep some of these roles. But I think on a larger scale, we're kind of we're not really preparing. Um, we're not really preparing for that. So I'm wondering what how do we really build resilience for the like, you know, for this older generation, in my opinion, because they're gonna need it the most. Um, because I think this younger generation, they're gonna completely uh just take the ball with it and say, you know what? Um, in this space where our opportunity was previously gonna be for these massive companies, we're gonna build our own thing. We're gonna do our own things. And like I think earlier last year, Carta had released a study that showed um there have there's been a massive increase in the amount of single-founder companies with no funding because of AI, right? I will attribute most of it to AI, but sure there are a bunch of causes. So I think we're actually we need to create resiliency in the older generations if we're not going to give younger generations the opportunity.

SPEAKER_01 41:43

Yes. So uh, and I I apologize, I can't remember the name of the book right now. Um, but uh I read a book or a speaker who talked to this very thing, and and one of the things that he said really stuck away with me. He looked at dozens, dozens of professions, and every time automation of some sort came in, uh the next cadre were not as good. Every every field except one. The one that was not, it was actually an improvement, it was bomb disposal. And it was bomb disposal because before automation, the gentleman, the senior would be out there with the bomb. Sure, they'd have a mic on, but they're more focused on not blowing themselves up than explaining what they're doing. The junior's back in the truck, uh a good distance away, can't really see, can only rely. Once you put this flip on it, once you do the robotics, now the junior is driving the joysticks, driving the robot, and the senior's over his shoulder, you know, guiding and and explaining. Yeah, my parents told me video games weren't going to be good for anything. I I I agree I disagree. I also don't want to do bomb disposal for a living. But but um so yeah, there's there's a real concern. I do think um, I've had the opportunity to work with with some ethics groups at some universities. I do think this is an ethical piece that we are missing, right? Should people cheat? Yeah, could people cheat? Could do it. No, it's what are we doing for the next generation? Because you're spot on, right? I do all sorts of things every day now that five years ago, ten years ago in the same position, I would have passed down to to my next level to to do and figure out. Um what do we do for our seniors? That's so for the junior folks, I think we are ethically bound as as organizations and and carekeepers of our world um to make sure we continue to bring in. Junior folks and continue to give them the opportunity to learn on real things. And maybe that's not quite as cost effective as going pure AI, but to your point, we're gonna need it. Folks my age and older, well, and even a bit younger, I I share your concern, right? They're you know, I came from a generation where you were a jock or you were unpopular. Full stop, right? Um and we have taken that, we've turned it around and said, hey, look, we tech guys, we kind of know what we're doing, and and and we've been fairly successful for that. But there is still a lot of socially driven roles in a company, sales organizations, um, marketing, that uh now they're concerned and rightfully so, right? Um, salesperson, I throughout my career, even before for generative AI, they don't want to give up their secret sauce. I'm like, but if you I know you know these people's favorite sports teams. If you give us to those to us, if we can put them in our database, then marketing can send customized materials to them based on their spit, their their sports ball teams. Um they don't want to give it up. This is worse. This is worse. You're gonna come in, I'm gonna come in and say, here, here's your list of of AI approved uh people to call. They don't want that. They know who to call. And I believe they know who to call. But the junior folks don't. And the junior folks aren't gonna learn in the same way that the the senior folks do. I did uh an analysis probably 10, 15 years ago. You could see generally generationally how business changed. The oldest folks, salespeople in the company I was in at the time most had face-to-face visits. That's where most of their dollars were coming from. The next generation down,

What We Owe the Next Generation

SPEAKER_01 46:16

it was phone calls. Next generation down, it was electronic communication. And what was really fascinating when I split it by generate gender, it was offset by a generation. So women of the of the second oldest generation were behaving like men of the oldest generation. They were echoing what they saw their mentors being, and and it then carried down that way. Um I think that's there, there are a lot of reasons that's not great, but what it does illustrate is the younger generations, to your point, will adapt. The older ones, I think we just have to frame things, communicate things, build the relationships. Um, like I said, I won't go into a whole lot. At our sales meeting last week or two weeks ago, we announced our our internal sales-focused chatbot. It's really a product uh-driven chatbot. And it really was targeted towards our younger folks in the stores. We have a problem with turnover with those younger sales folks. Um, and I believe that it's I don't feel smart. I come in, someone asks me what stain I should put on my deck. I don't know. And I don't want to ask my boss every time. This should help with that. That should increase improve retention, improves profitability all the way down the line. And when you're in a company like mine, where the CEO literally started in the stores, a lot of C-suite literally started in the stores. That's important. And so I have no great answer other than build the relationships, let them know you're not here to take their jobs, you're here to help them do what they do better and get the stuff that they don't do well out of their way.

SPEAKER_00 48:10

That was like really well put, a lot better than I could have put it, or like I think some of the conversation, like some of the conversations that I've had around it have really just been kind of um skeptical and kind of pessimistic, like, and to be frank, realistic too, right? But there's also that other side of reality that's like, okay, well, like what if we focused on enablement? And the other piece is like, well, we haven't really talked about what it's actually doing to like senior engineers and principal engineers, right? And that's like it's it's hard, right? Because now it's like you've got these people up top with like less support, but more responsibility. And then if they get burnt out and tired, like who did who does that get passed on to? So it's it's pretty, it's pretty, it's a pretty interesting um dilemma we've gotten ourselves into, but you're right. The way out of it is really by enabling. And you said something really interesting about like kind of echoing um like one generation echoing another, right? Um, I know recently, like I I've switched my my um my job so many times. And you know, I've been really fortunate to be in organizations that have just been like, yeah, Mo, go run with it, go go and try this thing out, right? And kind of give me the flexibility to come go and like learn and build. And recently I like worked with someone on sales, right? And I know nothing about sales. All my friends would say that I could probably sell them on one of the the many hobbies that I have, but like I've never made a dollar from any sticker I've ever tried to sell. I just can't sell anything, right? Um, but I think the the interesting thing in working with the sales team is that they are really forward-looking. And, you know, when let's just say we look at competitive analysis or competitive, just doing that. Like this is something that I had to do recently. And it's kind of like, well, like, how do I enable these people that don't necessarily have all the insight into this particular um expertise or field in security engineering, the ability to go and use this at like a higher level, right? It's part like me writing something down and then telling them, and then also using AI to like help generate like a report that is more in a sales language, which is something I don't know, right? So it's honestly been a great glue between fields. And it's like if they and the the uh there's another whole layer to this where like there's a bunch of inaccuracies introduced by AI. So like when I write something, I have to go and review it. On the other side, when I translate it into sales language, they have to review it, right? And there's this back and forth that we have where it's like, oh, this works, this doesn't work, right? And we get better and better at doing this. But it's one way where it's kind of like uh now that jobs are kind of like we can't really echo each other as much, right? It's like how do we inform each other and kind of get to know each other, especially because I believe in the this next generation, these lines between jobs and specialties, um, they're really disappearing. Um, where folks who are focused or like really laser focused in a single area are now performing in other areas where we originally wanted people to become really deep specialists. Um turns out the jack of all trades has started to shine, especially when AI comes in and enables them to be level up mastery across all these different areas. So I think it's so interesting how the battlefield has kind of changed, both in terms of seniority and in terms of specialized specialization and literacy and education and just everything. I don't think everything AI is bad. I don't think everything AI is good, but it most certainly has been dynamic, um, to if that's to say the least. The one place where I think AI will never take over, and I know it's already been done, is creating a really good story for DM. Nothing, nothing will change that. And it goes back to the sales analogy where you're like, you know, your the favorite teams of the people that you sell to. Well, you know your players the best, and you know, you know exactly what makes them tick. I'm sure one day it'll be easier to generate scenarios that are like that, but don't think we'll exactly get there. Um, it kind of makes me think of the book Death of a Salesman, too, um, where it was just the the salesman who was unable to adapt to new techniques and was finding that the customers were just not the same and the people weren't the same. And um he, even though he was the same, um, was no longer suitable for the world because of how fast it was changing. And I don't think we're seeing the death of literal salesmen right now, but I think we're seeing this kind of um death of a salesman theme take place across industries where we're finding these folks that are locked in traditional values kind of um being forced out of their comfort zones. And I don't think that there's ever been a time where someone's been pushed out of their comfort zone, if they stepped up to the plate and they really embraced it, that they were left behind. So I don't think we'll see many people left behind. I think we'll see people become displaced for a while and understand what comfort now looks like. But all that said, I know we are just about at time. And, you know, do you have like anything that you would kind of encourage folks to do? Or I don't know. What what are your final thoughts on on just this topic?

SPEAKER_01 54:08

I think we have a responsibility as human beings to embrace change and accept humility. Um if you can do those two things, uh, I think you can adapt to almost anything. Um, and I think that was part of part of why I'm in this role now, right? You may say going from you know, hands-on keyboard data scientists to governance, it seems insane. But yeah, but I was I was open to the change. I was looking for change, and I was not going in like I knew everything. I I was looking for partners and and looking for getting cohesion and and cooperation. Um and when I look back on my career, those two things seem to be where my successes have come, rather than uh trying to blindly charge forward the way I've always done it.

SPEAKER_00 55:11

Where can people find you on the socials? Do you have anything coming up? You got any new books, any open game sessions?

SPEAKER_01 55:18

Probably the easiest place to find me is is LinkedIn. Um I I try to do at least one post a week on Mondays. Just really my my ramblings and my thinkings around governance and culture and change and and all those things. Um I believe you guys have my speaker page link uh so you can share that. It is it is unfortunately not a quick one to rattle off. Um and uh yeah. And what do I have coming up? Not a whole lot right now. I'm I'm waiting for the freeze to come out of Cleveland. I think the thing to watch for is find me at a conference, talk to me about the game I'm designing, and uh maybe we can uh throw some dice.

SPEAKER_00 56:02

Thanks so much and see y'all next time. If this episode helped cut through the noise, like or subscribe so you don't miss what's next. Thanks for spending time with us. Until next time, stay curious.