AIAW Podcast
AIAW Podcast
E154 - King´s AI Enablement Toolbox - Carl Carlheim-Gyllensköld & Patrick Juhl
Join us for Episode 154 of the AIAW Podcast as we explore the transformative power of the AI Enablement Toolbox with Carl Carlheim Gyllensköld, AI Enablement Manager, and Patrick Juhl, AI Program Director at King. In this insightful conversation, our guests break down King's innovative approach—from a human-centric focus and hands-on workshops to pilot projects and the empowering role of CAKE Champions—that has revolutionized how teams adopt AI tools and drive productivity. Discover how these practical, data-driven strategies are making AI accessible and effective, fueling change and continuous learning across the organization .
Follow us on youtube: https://www.youtube.com/@aiawpodcast
Well, we're just in the inception stage. Really, we just started, but the idea is to give vibe coding capabilities to all our developers and allow them to vibe code as part of their professional activities.
Speaker 2:And have you even figured out an internal king definition of what you mean with vibe coding? I mean, I go with. What is it not?
Speaker 1:Carpatis definition.
Speaker 2:Carpatis, yeah, yeah.
Speaker 1:That's the one I go to. Okay, what's Carpatis? What?
Speaker 2:is it not Carpathes definition? Yeah, that's the one I go to. Okay, what's?
Speaker 1:Carpathes. That's the best one, I think so it's basically the act of coding and forgetting about the code. As you're doing it right, you just let go, you forget about the code. The code even exists. Yeah, you've probably been testing code before.
Speaker 4:Scary right.
Speaker 1:Yeah, but that's exciting because it's a paradigm shift. I think in the profession it's really different from what people have tried before. It's going to put a lot of pressure on the development process and process of development much more than on the code production itself, because it's moving away from that.
Speaker 2:So I'm really excited to understand that better basically, and with the current understanding, why is it different? Why is it a paradigm shift? What are you excited about? But what are you also concerned about when you go into vibe coding?
Speaker 1:Well, I guess the big concern is like I think vibe coding has proven itself in low stakes environments. Right, you can see it, you want a new website with something pretty simple, up and running right away. You can do it, it works. It works right away. But now you put it in a high stakes environment. What happens then? And that's exactly what we were trying to find out. Right, I mean, we're pretty confident works on one end, but what happens on the other end? And where does it put pressure on the development?
Speaker 2:uh, efforts, right, and can we already now potentially think about new types of skills or new types of thinkings and frameworks?
Speaker 1:we need to do safe, smooth, scalable vibe coding I mean I, first and foremost, is going to be going back to first principles, like I mean, it has to be right, it has to be what can we rely on in order to make this work? And I think there is well. Do we have the infrastructure in place for testing what we're producing? Do we have the processes to ensure that what goes out is safe and it just works? And all these things are going to be important questions. And then, further down the line is like how does it evolve with time? And what other parts of the process can we include in vibe coding, not just, you know, the coding itself?
Speaker 2:To me, in order to unleash vibe coding, you need to hide or simplify. For the vibe coding to exist, fundamental production grade stuff probably needs to happen anyway. So we get to another user experience. We get to we need to. Then, in order to vibe coding to work, the cognitive load of some stuff needs to go somewhere else. It doesn't disappear, the complexity doesn't go away, that's true. So that's kind of. Then we need different frameworks, we need different types of decoupling of what we do, so someone else is taking care of it, platform-wise, or something like this. I don't know. I've been going in this direction when I think about vibe coding at scale or production grade.
Speaker 5:I personally find it like you have to shift your focus. It's like you can do vibe coding only for so long, and then there's an error and the vibe doesn't work anymore. Then you have to zoom out a bit and say, okay, where did I start? Where did I end up? How do I deal with this situation now? Sometimes it's even better, in my experience, to just scratch it and go back from start again, but after some time that's not an option anymore and then, it's like you have to be ready in the vibe moment, when you're in the vibe, to always leave the vibe.
Speaker 5:That's my experience. Do you agree with that?
Speaker 1:I mean pretty much right, I think this is what's going to happen. I mean, first of all, the name is so apt I want to point that out because it's exactly what we mean right, you're in the vibe and then suddenly you have to step out and stop yourself from from what you're doing, take a step back and and try to understand what has happened in that time, kind of thing. And and yes, we need tools, we need frameworks, we need things to make sense of it, and I think the tooling is catching up fairly quickly. We see it, there's. There's some obvious like commands you have that that are helpful right away. Right, you want to undo something, just undo it, right, you, you went too far. That's take a step back. So it's possible. But but you're right, where does the cognitive load go and how do you handle the?
Speaker 2:whole thing, because you get into the flow in the vibe and you want to use converse and go forward, but at the same time, when you want to do production-grade coding, you have the 360 view of you know, is it documented right?
Speaker 4:Do you have the metadata right?
Speaker 2:Is it interoperable with other stuff Exactly? And so the tricky one is then you know, simon Wardley is the guy behind Wordly mapping. He's been talking about conversational programming for years. Let's call it vibe coding, right, and he's. Some of his posts is very to the point, right. Where does all this complexity go and how do we handle it? So I think that's interesting, but tell us more about you know what is your project all about in Vibe coding. What is it called? Is it called Vibe?
Speaker 1:It is called Project Vibe.
Speaker 5:I love it.
Speaker 1:We've been very good at acronyms in the past, but people get tired of them, so we stopped that. We just call it Project.
Speaker 4:Vibe, so no more cake.
Speaker 1:No, no, stop that. We just call the project no more cake. No, no, no, let's cake the vibe. Let's cake the vibe. We'll figure out a way to put these two together for sure. Um, but yeah, the project is similar to what we did with cake, but this time it's more narrow in scope, so to say it's more focused on developers and practice of development and, of course, enabled by the tool starter available today. Right, so it's being on top of the wave, as it happens, and make sure that we take advantage of it.
Speaker 2:And I want to hear your opinion. Who are the target audience for Vibe coding? Is it the old engineers who actually could do the backend properly? Who is properly elite? Data engineers, software developers, or is it this kind of pseudo coders who are drag and drop coders today, who are sort of closer to the business? Have you thought about who are the vibe coders in a company like King I?
Speaker 1:mean I'll get back to it later but it definitely enables people right, and I think this is a broader story about llm. It allows access to this language which hasn't been accessible to most people, and the impression before was like sure, now I can code like I used to be able to code. I just haven't done it for a while, but now I can do it again and it's exciting. But I have a technical background so I I can do it. Here. We're a step further out of that. That's what I'm thinking, too right.
Speaker 2:I'm going back to the classical team topologies platform team, enablement team, value stream teams and all of a sudden you unleashed a lot of productivity in the value stream team. Potentially and this is the whole rebalancing If I go to a place volvo or scania or vattenfall or whatever where we have very few engineers who can code and a lot of people who are great engineers or whatever in the workflows in the business. This is about the rebalancing of some of the business roles, in my opinion.
Speaker 4:I don't know. What do you think? I think for me? I agree with patrick, of course, but and I think from a like a personal point of view me not working at King, outside of King I have a problem solver. I love to solve problems and I have, and everybody who loves to solve problems you have a new tool and I can solve really hard software problems. I can do stuff. So I have a friend who owns a restaurant. He always complains about stuff and he makes so many new things with his hands and I said this year you're going to be able to solve all those things because you have the ideas and you can solve them.
Speaker 5:Yeah, I think that's super interesting. Oh sorry.
Speaker 1:No, I was just about to say because I like the bridging analogy. So you have different domains right, domains of technical domains. You have different domains right Domains of technical domains. You have application domains, and they're pretty distant from each other.
Speaker 1:And it just takes me back to the Pilterian days and working on the very sort of technical type solution. We build a platform for training deep learning models right, and that's pretty foreign from people who work in their industries, in their verticals, like what they do there, and we were trying to bridge, getting closer to them, like we could train the models right and then we could do things. That that could do a lot of things and we had. We saw the potential but convincing them was really hard because how, how would they relate to?
Speaker 5:this.
Speaker 1:Yeah, it must be fantastic for you yeah, exactly, but now we're talking about a world in which they're bridging from their side. Right, they're getting the tools that allows them to bridge into this technical space, and that's incredible. So that's a really empowering part of it.
Speaker 4:People have a lot of ideas on how to solve stuff and that we have been tapping into that across King that people really feel that, oh, I can use these AI tools that you have at our fingertips to actually solve my day-to-day problems. That's when we have found that we've seen the spark in their eyes.
Speaker 2:Isn't this one of the big topics right as long as you have you know. Now I'm referring to more traditional companies, where you have a quite clear divide between tech roles and IT whatever you want to call it and business, right.
Speaker 2:And we hire business people. I remember back in IT whatever you want to call it and business right, and we hire business people. I remember back in 2016,. We were working on getting business control, more data-driven, and we made a simple statement to say we're not going to hire a junior business controller who cannot do a join, who cannot talk about SQL.
Speaker 2:So it's about this new normal in this Venn diagram you know, the tech roles and the domain roles and now we need to meet in the middle, and AI is unleashing a lot of stuff, but it's a new normal of living in the intersect of this Venn diagram and that means some guys need to be real hardcore tech platform engineers to enable the Vibe engineers, which is, in my opinion, the new school business people.
Speaker 5:And have you started to identify different roles within vibing. Is it like everybody does everything? Or is it like the tech people do the more advanced stuff? Or maybe you're specialized in back-end, front-end, Maybe you're an application owner and you do the PRDs. Or is it everybody does everything? What do you think?
Speaker 1:So you can join our pilot because we aim to understand that.
Speaker 1:No we have barely started to be honest, but this is the kind of things we expect to find. Like I talked about this dimension of stakes low stakes versus high stakes we assume there's going to be something about that, right, and we assume there's going to be something about the different types of roles and where they sit and what kind of people we enable and the context in which they do it, and there's a lot of these factors we need to extract from this pilot. This is what we're trying to do, right, we're diving deep and we're trying to find out as much as possible.
Speaker 2:This is the perfect answer, and also the segue why we're here because you know what we are. We need to experiment, to learn to frame these things. They're not going to use drop down in the lamp of us, for sure. But with that, yes, I would like to invite to the podcast formerly Carl and Patrick or Kalle or Patrick and I'd like you to in. You know so excited, excited for the theme today, which is going to be about one year with ChatGPT at King.
Speaker 4:Yes.
Speaker 2:But, more importantly, it's a story about AI enablement. It's a story about how do we adopt and embrace Gen AI in a place like King Fantastic story, so why don't we start there? And I will go. A place like King yeah, fantastic story, yeah, so why don't we start there? And I will go to Kalle first. Yeah, and I'm not even going to look into your detailed title, but you can tell us your title and your background and your story and what you do, and then we do the same with Patrick.
Speaker 4:Yeah, I have a very non-internet friend named Carl Karl-Emil Jönsfeld aka Kalle which no other than Swedes can pronounce. So, but I'm the AI enablement manager at King, which means that my job is to enable as many of the Kingsters that are, the people who work at King with AI as possible.
Speaker 2:And then very quickly, you used at King as possible and then very quickly use the 30-second spiel who is King and where do they belong to, and who owns King and all that? What is King?
Speaker 4:Yeah, so King is a game company that creates games. The biggest one is Candy Crush Saga, Then we have Candy Crush Soga Saga and Farm Heroes Saga and then some other games. So in context we have 20 million monthly average uses 200.
Speaker 1:200 million, 200 million, that's an order of magnitude.
Speaker 2:Sorry. I got the two right.
Speaker 4:So we have about 18,000 levels in Candy Crush Saga.
Speaker 1:Alone, alone.
Speaker 2:And you were King started on its own and then was bought by was it Blizzard Activision? Yes, and then Blizzard Activision was bought up by Microsoft. Yes, so you are a Microsoft employee.
Speaker 4:We are a Microsoft employee. We have a Microsoft email address. Part of Xbox.
Speaker 2:Part of.
Speaker 4:Xbox. Yes, so that's, and previously we and we both me and Patrick we came with a Peltarion acquisition, so we worked previously at Peltarion as an.
Speaker 2:AI company. And this is the backdrop to inviting Jesper here as my co-host today, because typically Anders, formerly head of research at Peltarion, is the real co-host today. Because typically Anders, formerly head of research at Veltorion, is the real co-host me and Anders is driving, and then Jesper you've been on a couple of shows now, so thank you, jesper, for joining and Jesper is from Volvo Cars.
Speaker 5:Yes, working with the AI, you know quickly yeah, so I'm AI engineer lead at Volvo Cars and I lead a small team that is focused primarily on gen ai. I've been doing so for one and a half years and, as you were saying, I I was also pulled into volvo cars from a from a volvo owned company, so I know all about what it's like to get into the bigger mother.
Speaker 2:yeah, yeah, yeah, yeah.
Speaker 5:It would be interesting to talk more about our experiences, maybe later.
Speaker 2:Yeah, but that was the first rabbit hole. And then back to Kalle. You know a little bit who you are and what your role is all about and how you ended up there.
Speaker 4:Yeah, so I'm an engineer as a background, but I'm pretty much as good an engineer as anyone else, but I'm much better at explaining how tech works than most people. So I ended up doing that as a profession. And then at one late night I was drinking beer with the former CEO of Pelotron and he asked me hey, the things that you talk about even when you drink beer, about explaining tech, do you want to come to Peltaren and do that? And I said yes.
Speaker 2:Was that Luca, or was it the other guy, elias?
Speaker 5:Elias, yeah.
Speaker 4:So then I ended up at Peltaren and I explained how the Peltaren platform worked for five years and now I try to explain how AI works to the rest of King. So that's my background and also I'm a keen maker, so I love to make stuff, so I have a million ideas in my head, so I love vibe coding because now I can not only make stuff with my hands but also can make stuff software, because I'm not a coder but now I can code.
Speaker 2:But let's start also introducing and letting Patrick do the background and then we can come back to together what your organization team is all about.
Speaker 1:But Patrick, who are you All right? Well, I'm an ex-product manager from Peltarian. I joined through that acquisition as well. I have a background there in working with the operational part of the platforms looking at hosting models, serving models and making sure they stay up to models, serving models and making sure they stay up to date, monitoring models and things like that. I have a somewhat technical background but since I've been a product for a long time, I've lost those skills long ago, so you're a product manager owner more than an engineer.
Speaker 1:Exactly, you can definitely say that at this point. And at King, my role is program director. Sorry for the AI productivity team, so it's a sister team of Catalyst. We sit as close as you can get in the office and we work very much together on a lot of our initiatives, and our role here at King is really to enable, but also get the best out of the tools and make sense of uh, the adoption of the tools and and deliver impact with it so that's what we do and what I've been focused on for the past good two years now but?
Speaker 2:but let's go straight into it and first of all, I think it's quite important to understand the team and the team position and structure. What is your internal capabilities in the team? How do you fit into the King ecosystem? Could you elaborate on that together?
Speaker 1:For sure. I mean, first of all, we are a bigger, bigger than just a team or an organization. We're called ace for the ai center of excellence, and our, our organization does the whole, uh, transformation of king really, when it comes to ai could you define that transformation as a mission or something or objective?
Speaker 2:oh man, I didn't. I didn't learn it by heart.
Speaker 1:I see it every week, you know, but I don't actually remember the the words, but know it by heart. The mission is about transformation right. Taking advantage of the technology that is here effectively and getting the best out of it, but bringing people along with it right.
Speaker 4:I mean, it's also not only I mean today we're talking mostly about Gen AI and that kind of AI, but it's also about more traditional ML and that kind of stuff as well.
Speaker 5:But you're not alone in this. It's like you're part of a virtual team that spans across.
Speaker 2:How does it work? That's a good question. How does it really work?
Speaker 4:So we have about 100 Kingsters who work with ML and. Mlcraft, but we in the central team we call ACE. We have four teams. So we in the central team we call ACE. We have four teams. So we have the research team called AI Labs, and then we have MLSB Machine Learning Special Projects who work with specific machine learning projects, and then we have the air productivity team, which is Patrick's team, and then we have the AML team.
Speaker 2:So we're working towards different target groups basically, and I think it's cool I I want to, you know, please subscribe and have a look at the old episodes, because we had luca here several times who was, uh, you know, founder of pelotarian and then, ultimately, you know, he's still working his way around microsoft. And we had Sahar and I think she's leading up. She's been on maternity leave, I should say, but she's in the research lab right, oh yeah.
Speaker 2:Yeah, so there's been some very distinguished people from King here already. You're doing great work. I just want to point that out.
Speaker 1:I mean, I want to say we're very lucky right. We have these people in the team and King has invested. It's really a token of you know token on the investment. Exactly.
Speaker 2:So if we now look at this context, we have several teams under the ace umbrella. Could we go one step deeper into what you guys do in this context? A little bit into your teams. You start first.
Speaker 1:Oh, it's going to be a long story. I have to warn you because, I cannot tell my team without going back to where it comes from and why we even started this team in the first place. Love it.
Speaker 1:I will start there. So, if you remember if you remember you probably do 2023, um, long after COVID or not very long after we uh well, chat GPT exploded right November, just before, and it was still the the, the third generation models, but then GPT-4 came out and that that was like the effective revolution in my mind. That's really when we saw the real potential and I think Luca, since we know him in this podcast, was the first one to react on that. Essentially, he spent, I believe, an evening, a weekend at most, and produced three prototypes of applications that allowed to interface with these models, and he demonstrated that the next Monday morning brought it back to work and said look, this is what I could do with the tools, and these prototypes were you could think of it as a chat GPT type interface, a Chrome plugin.
Speaker 1:You also had these two IDE integrated solutions, one in Visual Studio and another one in JetBrains and really showed that, first of all, it was possible and, second, like something big was happening, because if he could do that over a weekend and put it in the hands of people, then there was really something. So for that we derived the need for running a pilot and right away tried to understand what was happening and really sort of experiment with this and put it in the hands of Kingsters. So we very quickly put a team together and that's where we created this CAKE initiative. So CAKE stands for Cognitive AI, king Enablement.
Speaker 4:And it's also allowed us to buy a lot of cake for all our meetings. Oh yeah, we did that a lot.
Speaker 1:Good man, you can believe it. There's a lot of candy going on at King. Anything that's sweet works, so our acronyms have to fit in there. So, basically, this pilot program was based on these three prototypes and the idea was put it in the hands of Kingsters and as quickly as possible, trying to understand what would people make of it. How would they react? Like what was it? We had to consider Legal guidelines, security guidelines, like everything was new, and we had to figure it all out as we went.
Speaker 1:But putting it in the hands of people that do the job was the most important part, right, and involving them in that journey. So it was meant to be a one month initiative. It was like we start tomorrow and we take, you know, a hundred people one month. We started the next day, but we took 300 people and it lasted for three months instead, and it's pretty broad scale, right.
Speaker 1:But the smart thing we did, I think, and it was to essentially say the data is really important, we need to collect all these conversations and we, of course, told the people that were participating that this was happening, because this really allowed us to sort of gather the interactions that people were having with these new models and we could later on try to understand what was going on here. Was it providing any value? What could we make out of this? And we also run this as a research initiative. We had UX research people in the team that together helped us really understand people. So we ran a ton of surveys. We had software usability surveys, we have technology acceptance type surveys, so a lot of data. We collected as much as possible, including diary studies. We were like everyday, pounding people with surveys, trying to understand.
Speaker 4:This is also told before they joined that we're going to be died by survey.
Speaker 2:No so so just to summarize the first step here yeah, when luca showed this, it led into a focus to do some sort of pilot. Yes, and the pilot had the a couple of objectives. One was to pilot stuff to see that it works, but actually, more importantly, maybe, to collect data around the experiences and how does this work and what you know? What do we need to do in order to make this stick?
Speaker 1:yes, and that was definitely the objectives. What we did not see is how we would do that, like how, how do you find that out from that kind of data? Right, I mean, sure, if you get. We saw a shift in the data, in user usability and in technology acceptance where people got more used to it as it went. The confidence they had in the tool increased over time. So clearly there was something like people were a bit skeptical in the start but they were like interested in trying and then once they moved forward in the pilot, they got more comfortable with it.
Speaker 2:So let's just stand back up the tape a little bit and say what was the ingredients of the pilot, what were they actually piloting? Did you give them an enterprise license to chat GPT?
Speaker 1:What was?
Speaker 2:it all about. What was the pilot?
Speaker 1:It was the three prototypes I mentioned. Okay, so it was a Chrome plugin that effectively looks like roughly like the chat GPT interface still today, right? So a chat-based interface that people had in their browser and could use at any time For any specifics or any type of work.
Speaker 1:Yeah, for anything, they could use it for anything. And people who signed up at first we thought we had to control who came in, but then we just opened it up and said, well, whoever shows interest and is going to engage with this, come in. And we were actually surprised very early on by who actually showed up.
Speaker 2:It was not your. You could not guess that list. Not at all, not at all, not at all.
Speaker 1:Like. The biggest surprise came from a team called Globalization. They have copywriters, they focus on the written content inside our games and you may, in hindsight, it may look like, oh, this is obvious, right, they write text, they work with text every day, but you can also think of it. At the time, people were freaking out, right, these tools were going to take everybody's jobs, and if anybody had to be worried at the time, it was these teams. But no, they were interested, they were open and they wanted to try it out and they wanted to understand how the world would look like when they had these tools in their hands. So they were the first to come.
Speaker 5:Can I ask um, I'm sure that a lot of people out there are, uh, wishing that they have done this earlier. Maybe, or maybe they're sitting right now and thinking I have this awesome thing I want to try out and I'm working in maybe a medium-sized company, maybe even a bit larger than medium-sized. What were the key ingredients to make this pilot stick, to get a lot of people involved and find all this good stuff? Like you said, this was not expected, that this would be one of the main beneficiaries. What were the key ingredients organizationally and as an initiative? What needs to be in place to get this to work out?
Speaker 1:Well, leadership buy-in, because without that you don't even have a pilot and it doesn't even start, but that also helps allowing people to go in there and try it out Right. So it's quite important. And that it came from leadership and they said, yes, this is important, we're investing in this and we have to do this Right. So the recognition there was essential Communication, constant communication about it. Building a community very early on became it was quite natural because we needed a channel. We needed to talk about it. Early on became it was quite natural because we needed a channel, we needed to talk about it. But it became much more than just a channel to talk about. It became a community where people were sharing their experiences and there was a lot going on.
Speaker 4:So I think that also that the focus that we had a lot was the but with the human centric, and we look focused on the people and the things that they want to do. We didn't say, oh, you're going to do this and you're going to do this, and you're going to do this, try this out and see what's. And we got people who figured things out for themselves. And I also think that it's the nature of our teams that we're not IT teams. We are two teams that are put in place to make people get help and enable with these tools. So I think that's also what we found that there's so many people working in a company and these kinds of tools. They have so very specific use cases and those use cases someone from outside of those teams or even in a team and there's so many different crafts I mean someone a QA in a team and a developer in a QA they won't use it in the same way.
Speaker 4:So, but they will use it within the craft, and that is something so we learned and we tried to do it as well to have a human-centric approach.
Speaker 2:These humans are going to use this and nobody can tell them how they're going to use it, because it's so but now, now we we're talking about uh, you're starting, so you're starting a movement, I would argue, and, and and the core story now is the core movement around how to take um, chat, gpt as an invention and then create a movement that leads to the new normal of king.
Speaker 2:And you started with a pilot and you did that in order to try three different prototypes or approaches, and then you collected a lot of data and then we now know in hindsight that you kind of took the pilot into a framework you call CAKE. That is actually very, very interesting. So could you you know, if you were to tell this story from the beginning, a little bit like how you moved into the pilot, you started there, and then how you flipped from the? What was the process or what happened that took you from pilot mode into framework mode, into arriving at cake and what that is all about? So if you take it in your story, how would you tell the story?
Speaker 1:I'll tell one more bit about the pilot and then I'll go right into that, because we're reaching the end of the pilot. It segues into each other, right?
Speaker 4:I will answer for sure it is a linear story.
Speaker 2:It is pretty much.
Speaker 1:There's a logical Tell it linear One of the last things we did in the context of the pilot. We felt we needed some hard data to convince the business as well that this was a smart investment. And this hard data. We thought about it for a while and leadership had actually been pushing for setting up an experiment and we felt like, okay, yes, we would love to do that, but we need a context for that experiment. And that's when we decided and honed in on a code competition just to demonstrate what these tools could do. So, just to give you a little bit sort of the setup of this coding competition, we designed our own coding problems that people had to solve and we basically created groups or pools of people in which they would allow, they would compete for a prize and in solving those problems, some of them would get the tools, some of them would not, and some of them would get the tools that had been part of our pilot, so we could also study the difference.
Speaker 2:Because three study you get three groups in a research study. Exactly yes.
Speaker 1:And that was just a one-off event. But we really wanted to study that and of course we were careful in the crafting of the problems. We did not want problems that potentially had been seen by the LLM, and this again were early days of GBT-4. We didn't quite know what these models could do in that regard, but we had a sense that they were very much enabling. Now the study has a lot of limitations. It's very narrow in scope, it's very narrow in tasks that we were giving people and we knew it was much more broadened than that. But it could give us an indication of the impact that it has.
Speaker 1:So that was another set of data we collected through this competition. I mean, the outcome, without too much surprise, was something like 13 to 1. 13 to 1. In resolving the problem, and we had been very careful in crafting problems that were hard for language models to solve. Right, they worked with letters and things like that that clearly LLMs really were terrible at dealing with, but still people got so much better and actually one of the best performing person is sitting in this room.
Speaker 4:And Kalle himself did really, really well, and I don't know how to code.
Speaker 3:I came in third. I didn't win, I came in third.
Speaker 4:But that was also really interesting. That was one of my flip moments. Wow, this technology.
Speaker 5:That's the real enabler. It's like, if you have these ideas but you don't know exactly how to execute them, then you have the superpower with ChatGPT that can just tell you how to execute it. Yes, exactly, that's beautiful.
Speaker 1:And so now we're sitting on all this data. We're at the end of the pilot, we've gotten the code, competition, we have collected all these interactions people have been having and I'm sitting on this drove of data and trying to figure out sort of I needed to answer questions that came from the business, from stakeholders, and now I have to introduce one of them key stakeholders that was legal, legal had massive concerns with this.
Speaker 1:I mean they didn't know what to make of it no, and we had to we had to agree very early on the rules of engagement like what would it look like? We had guidelines to build and craft with them. It's a lot of backing to take legal key advice here involve and co-create with legal.
Speaker 2:Don't leave them to last. No, that's as a creator.
Speaker 1:That is a bad idea For sure, and involving them also allowed them to be part of this journey, right.
Speaker 1:They're also potentially going to be using these tools right and they need to figure out what it means for their profession. So some of them were more open than others and allowed us to essentially explore and helped us understand what to think about, what to consider, but they were always asking these questions. The same questions came up what are people using it for? What are people using it for? And I was getting these questions and I mean I'm sitting on this data. I have three months of 300 people interfacing the model. That's a lot of conversations and they're long. I'm a product manager. I love to go through reviews from users and things like that.
Speaker 2:But these are chat conversations, yes.
Speaker 1:And they're long right. So I don't really know what to do with it. Until a moment it struck me like this big aha moment. It's like, of course I have the tool to analyze this data. It's in my hands. That's what I'm doing.
Speaker 2:So awesome, flip it. I have the tool to analyze this data. It's in my hands, that's what I'm doing. So ask to flip it in token window big enough.
Speaker 1:I have to talk about the budget of this project afterwards.
Speaker 1:But anyways, what happened effectively is that I bring in all this data and essentially spin it through the GPT models and extract out of it actionable data, something that I can look at, I can plot, I can start building pie charts to understand sort of what is it used for. And I had demographic data, of course, of our users. And suddenly I put these two things together and I started looking at teams and what they're using it for and I see like a picture that doesn't make too much sense. There was very little correlation between the teams and their usage. I was like okay, like maybe my method is wrong, but then it struck me is like what would people use it for? Well, for the work that they effectively do. And then I flipped the question and then turn it over to well, maybe we should look at their craft and not about their teams. I mean, of course, king, like a lot of modern tech companies, have very cross-functional teams.
Speaker 1:So I went horizontal instead and looked at the crafts, and when I did that, the picture was so bright and visible, right Suddenly, it was so clear that people were using it exactly for what they're doing the work for. So it was like, oh yeah, obviously they are, but I mean, it wasn't the data. So the method was a little bit proven. I was like, okay, now I understand something about this data. What can I get out of it? Is it valuable, is it not? And what can I do with this? And so this is going to lead to the next part of the story, but I will leave it to later. But it informed something about the craft.
Speaker 1:There was something really important about the activity and the work that people are doing, and the other side of this coin was the legal question. Legal, we're asking for what people are doing. But really what they were asking is like what is the risks that people are taking? And once I understood that that was their question, I was like but again, I have the tool to assess this. I mean obvious, maybe, but take the guidelines that we had written with legal, take the guidelines that we had written with legal. Use that to assess the risk of all these conversations and suddenly I could plot a risk profile of all the activity that people have been having with this new large language model and these two things put together like it was so rich in information, but also it completely changed the conversation we were having with these stakeholders.
Speaker 1:Like I could talk about data, I could put a number on it, I could say, yes, there's blatant infringement to the rules, but at this rate, and we can look at them because it's only a very few number of them and we can talk about these cases and we can go talk to people and understand it so effectively. This was an AE-driven audit of the pilot and it allows us to derive sort of what to make of this, how to understand it, and it was very, very powerful as an insight also because suddenly I could code this stuff, like the LLM had allowed me to do this, and this was really, really powerful. Now back to the craft, and this will lead into Kali's work and a lot of things that you do.
Speaker 5:Can I ask just were there very different risk profiles from different crafts?
Speaker 1:Yeah, you know, do you?
Speaker 5:have any interesting insights into, like how people are using it differently and what are the risks and what are the hardest problems to solve?
Speaker 1:So the biggest offenders were security themselves. Oh, wow. And the reason is that they were pentesting, they were producing things that looked like malicious usage of the LLM to actually perform actions that have ill intent, and that also confirmed the approach as well. Like, okay, sure, that's not actually a problem, but it is good that we're detecting it, because we should detect this right. So it was demonstrating the method work, but also like, of course, this is their job, people are doing it.
Speaker 5:It's a finding in itself.
Speaker 4:Exactly.
Speaker 1:Exactly so.
Speaker 1:This is the kind of things we started seeing in the data when looking closely at it, and it was very rich, like I keep on saying.
Speaker 1:But it was so informative about how to approach it later on, because we had a lot of conversation internally with these users and when we talked about what they would use it for, it was almost always like riddled with noise, like there was a lot of noise in terms of oh yeah, you know, I'm going to use it to summarize my emails. It's awesome for that and I'm going to use it for this. And I was like this is not very useful, right? I mean, of course, you're going to summarize your emails, but is that the big use case in your space, in your domain? And then, with the data I was talking about before, it's like well, obviously we need to talk not just to this person and this team, but rather to the craft through this person, and that's where we really sort of got this moment of aha. If we were ever going to roll out this, we need to be able to talk to the craft and and through these people.
Speaker 4:And and maybe you should say I think it's also quite interesting with the pilot also is that? So here we have 300 people that signed up actively signed up to be part of this rollout or this pilot, and then everybody didn't use it. So we had some super users from all different crafts that were really, really using it and they were learning so much and understood how to use it. But we had I don't know how many, but quite a lot that got access to ChatGPT before everybody else and didn't use it, and that was also learning. So how can we make these people that we know will have usage for this, but they have the tools at the fingertips but still they won't use it. So how can we solve that? At the fingertips, but still they won't use it. So how can we solve that? Because if you just give, and then we realize, okay, if you just give tools to your employees and with no guidance, with not empowering the super users, then they won't use it. It's hard to learn a new tool.
Speaker 4:It's hard to go over the thresholds, but if you do, then you understand. Oh, this is how I'm going to do it, so you need to have. So that's when we found these super users that we then called Cake Champions. So they changed champions, basically.
Speaker 1:Before you go there, can I add something? Yes, because to me it's a bit like a blank page type problem. Yeah, because these models, these, these capabilities, these tools are so general purpose.
Speaker 1:Yeah, that a lot of people. What they were telling us is like you know what, I don't know what to use it for. Yeah, I have no idea what to use it for, and they can come up with that. And then, naturally so it's like you're facing a blank page of paper. You have a pencil in your hand, you could do anything with it, but you don't know and and and so like back to the, the craft. Things like in these champions like this would allow them to relate much better if you have the right people in their field that understands their field, their problems, what they face every day, where, where the pains are, they can speak for that craft and what they can use it for.
Speaker 2:So they really helped us bridge into the craft but because what you look, what you're discussing now, is that you have you had a pivotal moment when you flip the data to look at crafts. Yes, and then from the craft, the data started to pop. That is useful, but then you could also see like, okay, within this craft, who is using, who's not using it. And all of a sudden, now all this is leading you down a path to look at sort of beyond the data, into the symptoms and into root issues. That then leads you down to a framework like cake.
Speaker 4:Yes.
Speaker 2:So you are now starting to even talk about some of the core ingredients in cake, which is a core consequence of what your insights were from the data. Yeah, so like so. So could we sort of summarize some of the key insights. That then leads to some other key patterns in the cake framework, like human-centric community, uh, super user, you know, whatever, whatever it is.
Speaker 1:I mean, you, you've started to list them, right, and they're there. It's like, well, engagement was super important, right? So we needed people that represented that engagement, these super users, these thought leaders right.
Speaker 2:So engagement is thought leaders. Is someone that sort of is the evangelist approach within your craft? Exactly, Exactly.
Speaker 4:Exactly so. We call them cake champions Cake champions, but they were craft-oriented, right? Yes, we had 40 different very granular crafts. I mean the developer is too broad. So we had different kinds of backend and different kinds of data. Yeah, exactly, and also legal and different kinds of UX.
Speaker 1:And that's important, right. It has to be granular Because you different kinds of UX, and that's important, right. It has to be granular because you could talk about UX.
Speaker 4:There's a lot of stuff in UX, right.
Speaker 1:You could talk about UX design. You could talk about UX writing. I didn't even know this when I started, but when I started looking into it and talking to people, I realized, okay, these are more specific and they need to be specific. So we need people from these specific crafts. And when you talk about crafts high level, you probably have five or six in a typical company, but here we had like 40 this like?
Speaker 2:what is driving this sort of fragmentation or this, this granularity? Is it the tool base? Is it the different? Because I would argue I, I'm a front-end developer, blah, blah, blah. And then you scratch the surface this is the figma guy working with the prototyping in Figma, and this is the React guy and I'm doing the visualizations in React and I'm doing the backend of React, which is still front-end. Is that what we're talking about here? Why do we get to such granularity, do you think?
Speaker 1:I mean it's specialization. I think at work, right, and if you think about UX, I mean UX as a whole. Again is a lot of things I mean in development. I mean of course it applies there too, and there's these more specific areas. But I think it's more stark in other places, like if you look at UX where somebody that does UX research versus somebody that does UX design. That's very different.
Speaker 5:Yeah, exactly.
Speaker 1:And they face different problems. They have different challenges every day, so that was the important part of the granularity in the crafts when we approached it and what else tell us.
Speaker 4:So then we have the cake champions. I drove it and we had like human centric focus what is human centric? Basically that we wanted to. The tools are for the users and not for the tools itself. So, hey, here's the fantastic cool new chat apathy and look at what cool stuff you can do Rather. Okay, so we have this tool. How can we make sure that people understand how to use it? So it's more like focusing on the usage and not the tech.
Speaker 2:So let me see if I can frame that in a way that you should check for understanding. Way back 10 years ago, 2016-17, we worked with installing self-service BI in the business controller community and we were very adamant to move away from bi or power bi training, which was the tool, into business control training in the way vattenfall does business control in power bi. So this is the user-centric, this is understanding the business controller role first and what is your guidelines and procedures and governance, how you do it in your company and, in that context, the tool yes, it comes last in that story, not first.
Speaker 4:Exactly, is that human centric? Yes, that is. And also, when all these new cool tools come, as we have now, it's very easy to be, tech focused rather than human focused, but I think that in this transformational shift that we have right now, it's super important to be human focused, because if you just give someone a tool that's really what we learned Most people won't use it because it's hard to go over that threshold.
Speaker 2:It's a white paper connecting the dots to. How does it apply to what I'm doing, my job?
Speaker 5:to be done. Yes, yeah. And how? Oh sorry.
Speaker 1:The product manager in me wants to say, like we really want to avoid the situation where we have a solution looking for a problem kind of thing the trash can solution no, but really it was quite important, right. It needed to come from people that understood the problem space, right. That was super important.
Speaker 2:That's why these things fall into place. This is one of the big, profound aha moments when we get invention centered or invention focused, because the hype, the FOMO, all this stuff is really driving us to jump knee-jerk reaction to the invention, rather than starting with the fundamental job to be done and then scout from that angle. It's different. You were on to something.
Speaker 5:I was thinking about the problem you went into when you saw that usage was low for some people that ended up with this blank slate of paper and didn't know what to do with it. Was the solution to that the champions, or was that part of the solution? I'm thinking? The typical thing in my world is you typically want people to go through a training. Yes, is that also part of this story? Yeah, it is.
Speaker 2:So you have the champion. The cake has several facets, so the champion was one. Yes, and I guess we're getting down the list now yeah, exactly.
Speaker 4:So I mean, if you go to continue linear, then we come up to the winter of 23, right? Yeah?
Speaker 1:yeah, pretty much, but I want to underscore something in that part that we're talking about. You know, human centric and all that. I mean you can think of it as a grassroots movement as well. Right, it did not come from the top. This initial pilot, yes, was sponsored by the executive that wanted this, but effectively, this was about people using it for their jobs and it became much more of a grassroots movement. They chose how they adopted it right Through this program as well.
Speaker 2:But so linear then. So I guess the champion. If you look at how Cake was installed, one of the first topics was the champion approach. I guess Continue linearly. How did it evolve?
Speaker 1:So the champions and the crafts two key ingredients, right. And then there was the enablement efforts, and I think Kalle is the best position to talk about that.
Speaker 4:So linearly, we got the license and decided that we're going to have licenses for all King enterprise licenses in December in 23. All right, so this is when you went to all in on ChatGPT enterprise license for every person Exactly.
Speaker 2:Tell me a little bit about how you procured that or how that works.
Speaker 4:I mean, that's the path You're selecting the tool, so the enablement comes soon. Yeah.
Speaker 2:But this is a good segue because a lot of companies and I'm quite involved with Scania, who's been through that journey quite recently yeah, this year or end of last year, I guess. But you were doing this end of 23.
Speaker 1:End of 23. Yes, so all this data I was talking about this trove, was the convincing factor. Right, bring that to executives and show them. Show them the data and show them people asking for the tool. Like we had 300 people and every single person in there said yes, we need to keep this right and that's telling People are like this is good, I want it, and just that helped convince the business that there was potential.
Speaker 2:So you go from 300 to I don't know how big King is now.
Speaker 1:Roughly 2,000. 2,000. Yeah, and we decide that everybody gets it right. We could have done a limited rollout. We could have focused on specific crafts.
Speaker 2:But here it was like this is a general purpose tool. Everybody should be allowed to use it. Yeah, but this is like a segue. But we're stopping right here Because I think there's many companies out there who are thinking about. You know what does it mean? You know we have the shadow AI approach. Everybody brings their own AI to work versus the enterprise. So what's the rationale or what is the difference when you say, oh, we're going to have an enterprise license with ChatGPT, I'm not sure that is widely known or understood. So what were you looking at? How did you approach that? What is it?
Speaker 1:So an enterprise agreement is a trust relationship. I mean it really is about that, right. I mean we're engaging here in a new territory with a trust relationship. I mean it really is about that, right. I mean we're engaging here in a new territory with a new technology. It's a SaaS provider. We need to trust this provider. An enterprise agreement is about building trust. There's, of course, features. That comes along with it, because you need to have it, otherwise the IT teams in your organization are just going to push back and say, no, we can't use this tool.
Speaker 2:So what is enterprise features that makes it more enterprise scalable?
Speaker 1:The most obvious one that comes to mind, and it's unfortunate that it's an enterprise feature. As a product manager, I would never recommend doing that, but it's SSO and integrations and your authentication.
Speaker 2:SSO meaning Single sign-on, Single sign-on authentication stuff, access stuff.
Speaker 1:Yeah, and control right, control Centralized control.
Speaker 2:Yeah exactly A proper sysadmin governance setup Governance.
Speaker 5:Was it always clear that OpenAI was the tool to use?
Speaker 1:Yeah, that's a good question. I mean, we're a Microsoft company, right?
Speaker 4:And it wasn't that clear.
Speaker 1:No, it wasn't that obvious, but it was not too hard to make a case that they were ahead of the curve and they clearly were there when we needed them.
Speaker 2:You would have fought harder to bring in Gemini into a Microsoft company.
Speaker 1:Don't mention it. I love what they do.
Speaker 2:So there is a step here of choosing your provider here, but of course, it looks a little bit different when you are owned by Microsoft. But it's interesting because even if you're part of Microsoft, it's still a fundamental topic to go to enterprise licensing, which I think is very telling how important this topic is.
Speaker 5:Yes, and I'm also living in Microsoft world at Volvo, and it's definitely not clear for us that we should go to ChatGPT or OpenAI, because we already have an agreement with Microsoft. So then it becomes more like the GitHub co-pilot and the regular co-pilot, which is more like a chativity clone. It's not the same, but that was never an option, right.
Speaker 1:Well, no, like you were pointing out, it's not the same. They were just not there, but I have to point out that this was very early days for even OpenAI. I mean, their enterprise offering came in late October 2023, and we were pushing a business case in November.
Speaker 2:You are really one of the first enterprise proper licenses.
Speaker 4:Exactly.
Speaker 2:That's cool.
Speaker 4:Which was also really fun because they were really happy to help us out and join our workshops in spring last year as well. So we got a lot of help from them.
Speaker 2:All right. So now we're a little bit through the procurement or selecting the tool topic and it was a little bit like okay, it's a different story here to some who is completely not you know, and not the Microsoft company. But then this leads then the segue into okay, now we have licenses for all our company. Let's now really talk about AI enablement.
Speaker 4:Yeah.
Speaker 2:So when is your function AI enablement taking shape? Is it earlier Is?
Speaker 4:it right now, so the team already existed. So it is just now. We made a complete flip and focused completely on this, so what was the?
Speaker 2:so do you? There's a slight focus in Pivot, so to speak, for the team. The teams come from somewhere and now this is well. This is the main thing, yeah totally.
Speaker 4:And then so what we did? We launched it 15th of February last year, and so what? We decided that every Kingston needed to complete a course, an intro course, to do that, to get access, which meant that we needed to produce this course and make sure that it was all the legal, that everybody has looked at the legal requirements to use this tool, so there was a baseline here. Right, yeah, exactly, and also that also comes from the pilot that we felt that this course it was one of the most important things was, of course, to make sure that everybody stood the legal boundaries of what you can and can't do.
Speaker 2:Give us an example to just get a flavor of what you mean with legal boundaries.
Speaker 1:I can say something like we don't produce any imagery through these models that go into our products so some of the stuff that we want to stay away from in some cases and that's because we put a lot of importance into what we produce as a company.
Speaker 1:There is space for AI and there's plenty of space for it without having to do that, and, as a matter of fact, the tools weren't really there. But even if they were, it was like would you want that and what does that mean? Like there's still a lot of open questions.
Speaker 2:So you have some policy topics, you have some principal topics that you wanted to convey now to a large company in order to use it for good or use it properly.
Speaker 4:Yeah, the way the brand wants it.
Speaker 4:Yeah, and also understand, from a responsible AI point of view, understand how these models are trained, the bias in the data so you don't want to propagate that bias and also, from a security point of view, how you can do's and don'ts. But it was also basically introducing the concept of prompting. What is prompting anyway? Just a word, and just get people just across the first little little threshold to understand what you can and can't do and also, from the management level, say, hey, this is a tool that we want you to use because it's the top of the line best tool you can use right now. And we put a lot of effort to make sure that you are as productive and have as much fun at work as possible. And this was all baked into this intro course that everybody and when they have taken, complete this course. After one day they got access to ChatGPT and then they were. They can just start to use it.
Speaker 2:But I'm a little bit naive now because, king come on, you are a tech native. Do you need prompting cores? Come on. Compared to Scania or Volvo or an analog.
Speaker 4:I mean 2000 employees and everybody's not developers. This is the point, right.
Speaker 2:So when you look at the crafts down to 40, 50, 60 different levels, of course there are people who are not coders in cake. They're legal and they have the same legal background as anyone else in legal background.
Speaker 5:It was of course the same for all crafts. Yes, it was.
Speaker 2:So this is the point, right, because you want to build a lingo, you want to build a foundational topic.
Speaker 4:It cannot be too advanced, but it's to give a solid community ground. Yeah, is that fair? Yeah, and also one yeah. So well, it wasn't only this course. That was not enough. That's what we also did. So we are 2000 employees. What we did during last spring, we did. It had in-person workshops for 1 300 kingsters, kingsters across all our offices. We have offices in Barcelona, london, malmö, berlin and Stockholm.
Speaker 2:So now we're building a Gen I excellence program Call it a cake, call it whatever you want. It starts with a foundational course. You used to switch on the tool, which gives a basic toolbox and a basic lingo, maybe, and then now we're adding on workshop. What's the difference? What happens in the workshop then?
Speaker 4:and a basic lingo maybe. And then we're adding on workshop. What's the difference? What happens in the workshop then? Yeah, so they're in person and the thing with those is make sure that everybody come in person and test the tools. And here we also had all the different cake champions coming and helping out, so they were super users from the different crafts. So when someone from the craft, say, human resources were in this workshop, there was someone from that craft that was a super user that could help them out, because, oh, here are the use cases and we also gave them some prompting tips on use context and all those things. But also really important here is to have this psychological safety that everybody wasn't new to this, so everybody was on the same level and you can really just play, because that's the best way to learn, just use it yourself.
Speaker 2:But let's slow down, because I think this is way bigger. I just want to follow you now. Yeah, first of all, challenging when you do a workshop and you have 50 different crafts is that you can't really do one size fits all. Then it becomes really watered down. So you solve that by basically mapping who is coming for this workshop. Then we need to have the evangelist for this craft. Yes, mapping it yes, exactly so.
Speaker 2:Then you figured out some sort of workshop methodology which was like a generic frame, yes, but then they worked on the use cases for their craft with their coaches, so to speak, as evangelists.
Speaker 4:Exactly.
Speaker 2:So this is quite brainy, right? So how do you do something that scales, that has a 50 craft mutation? This is what you did. This is pretty cool stuff. I mean, it didn't get it right.
Speaker 4:Yeah, it didn't get it right and I agree with you, it worked and we're super happy. We're super proud that it worked and especially super proud of all these champions. So they're 100 kingsters. They don't have that. This in their, in their profile, can you see?
Speaker 2:how easy it is to cheat this process when you don't have the data.
Speaker 2:And we don't do it for the real reasons, but you do it as a tick box exercise training yeah, we get an e-learning down and we get some fucking idiots standing there and doing a one-size-fits-all and it's fucking shit compared to doing the homework, doing it real, and saying, yeah, yeah, we need learning, we need excellence program. But what, how? And I see this, you know? Hint, hint, there's a lot of people going out buying this snake oil now when they haven't really done their homework, what they really need. This is a big difference. To do a one size fits all, to do 50 crafts in a structured format that scales for 2000 people. I just want to point that out. How that you don't get there.
Speaker 4:But without the data. I feel that you see that as well, because we see that as well.
Speaker 2:No, that you don't get there without the data. I feel that you see that as well, because we see that as well.
Speaker 5:No, but you don't get that by let's tick the box and get a training in.
Speaker 4:And how much work did it take just that one workshop? It took a lot of statistics and making sure that everybody so I mean, it's still something that's still together in a week.
Speaker 4:No, it's not, but we did one format so it doesn't have to. If it's too complex the format, then it becomes too much work. So we did a very simple that basically had an intro of how you prompt and how you should do stuff, and then the next one and then you basically use it yourself with the help of, and then just not someone that's standing in front of you and just telling them you do that and then you do that, but just let loose and test yourself and get help from your colleagues which you trust. And it's not going top down, it's rather bottom up.
Speaker 2:The community, the ambassador, we're building a super user community. We're building cohorts, whatever you want to call it. So this is the shaping and design. So the complexity is in the in that to fit the different jobs to be done. So you need, in a way, a very simple framework that is reproducible, but then you need to, but each individual needs to have their relevant job to be done.
Speaker 4:So, since we are really, really want to point out again that, if you're listening, all the cake champions here you are amazing, you are amazing so what does it take to become a cake champion?
Speaker 2:How is that working? How do you find the ambassadors and evangelists?
Speaker 4:Yeah. So I mean, some of them were in the pilot and then we mapped those to the crafts that we found and then we realized, hey, here we have some holes and here we have too many. And then we just checked that, found the super users around and asked do you want to join? And also, very importantly, we wanted the different champions in different offices, so in Barcelona, in London, in Berlin, in Stockholm and also across the games. So we just tried it and also it was since the A&M team it's my team, so me and my employee Begum, we were taking care of these champions and it really became a personal thing Because it's really for them to feel to build this community, care of these champions and it really became a personal thing Because it really for them to feel to build this community, you need to have a one-to-one connection with each and everyone.
Speaker 4:So the group is a hundred people, which is manageable just having this kind of work. And then it all became a community within champions which they also help each other and also share their understanding and knowledge about their problems they have, and so it's so we have a tight community which is really really working.
Speaker 2:So I get some. I get some old school vibing. I say it old school now, when we had the spotify model and we had the tribes and we had the guilds and the chapters and all that. But it doesn't matter what you call it, but it's, it's. It's the fundamental topic of having a community to turn to around the topics, right and having a super user, an ambassador that you can turn to yes, so it's.
Speaker 1:It's about two things, right, it's about focus and relevance. Yes, so focus and relevance, that's the craft. And then these individuals, these champions, their multiplier effects. Right, we're a very small team, of course, like we can't reach 2 000 people. They effectively multiplied us and allowed us to reach much deeper.
Speaker 4:We were four people in my team.
Speaker 2:You were three at that time yeah at the start, but now I'm missing one topic that I think is in here that you forgot to mention. I think you should have a new line in here. Let me test it out.
Speaker 2:how did you do train the trainer? How did you get to a view of how you could frame the job of the champions? So they are a little bit consistent, because it doesn't say anything about how did you get the champions to get the quality assurance how they were teaching and coaching. Then you have a blind spot, a gap of train the trainer, of where do you bring the cake champions together and work with them as your megaphone? Did you do that or not?
Speaker 4:We did not train the trainer.
Speaker 2:This is the next one I added to the list, but yes, but that absolutely.
Speaker 4:I mean that's super important. But I think maybe we did in like another way. But you must have done it, yeah because we shared. In the cake community, in the cake champion community, we shared what we did with everybody and also all the content that we created as an AI enablement team, which consists of people that we know how to teach. We know how to teach different specific, complex topics, so we shared a lot, everything. And also I mean we had OpenAI joining us to help us with the workshops.
Speaker 2:And then you invited, of course, the champions. Yeah, we have.
Speaker 4:So you got there.
Speaker 2:Okay, let me flip it. Do you have a champions channel? Yes, of course. Okay, so train-to-train doesn't need to be you know.
Speaker 4:So you have a community of champions, exactly and within the community of champions.
Speaker 2:You share your approach and methodology and coaching approaches. This is train-to-train.
Speaker 4:Yeah, so we did change it. And also I mean, we have, I mean some some champions like Chris Cole in finance. He's brilliant and he is like I mean he did, we learned from him a lot. And then it's also so we, when, when, when a champion like Alexandru in catalog games made it really good with, just oh this is brilliant, just do this. And also, honestly, I mean, that's really a part of why we are here today, because we don't know everything.
Speaker 4:This is linear. That's why we do the linear here. So each of these toolbox, the tools in this toolbox, we have learned along the way and that's why we want to learn from you as well. And this is also so. And then so, by listening to these champions and acknowledging that they have brilliant ideas, and actually, I must say, fun part, that we brought OpenAI along to our workshops, and they kind of said, oh fantastic, so we're going to want to see. And then we said do you have any content that you can share with us? And I shared everything with what we've done. And then, like three months later, I asked them can I get some content on how you enable other companies? And they gave it back. And then it was like my content.
Speaker 2:And then they opened their brand.
Speaker 4:Yeah, it the best in class.
Speaker 2:So we really did something good. We're going to continue down the list.
Speaker 5:Can I just one small rabbit hole? You were talking about the data expiration you did in the pilot. Did you do something similar after this workshop to evaluate the success of it, or how? How did you measure success? I'm asking because I'm standing there myself and planning, planning something much smaller, but still want to have some numbers out of out of a success.
Speaker 1:So I can't give you the number, but I can tell you that we have a study that's running, and this is a collaboration with three universities, and there we brought a lot more scientific rigor to the process.
Speaker 1:So the pilot itself was just us doing things basically and trying to figure out the best way out. And here we introduced sort of scientific rigor and put this up as an actual experiment. So we're still waiting to actually look at the data for this and really understand it. So we're still waiting to actually look at the data for this and really understand it, but it's it's a really uh, like there are things you can do there to really understand.
Speaker 5:We're effectively experimenting on the training right I can flip the question if it's uh. If it's uh, maybe easier to do it that way. The the thing that I'm planning and it's not my idea, it's somebody who came up with this idea to just do, uh, in our case a, a GitHub co-pilot session, to get a team and look at how much they're using co-pilot before, then do an intervention and then just look at how much are they using it afterwards. Do we see any patterns? Do people use it? The not-so-enabled before Do they become enabled afterwards?
Speaker 5:That's a simple way Does that make sense it makes total sense.
Speaker 2:But I want to lean in to what Patrick said. Scon is doing the same. Be smart, work with a master's thesis or a PhD thesis. There are people who really want to get some hands-on work to do, so I think it's to get the scientific rigor on someone else's effort. It's the way to do it. So, basically, you have your idea and you have that, but it also builds your credibility and it builds rigor and you get the published paper out of it. And equally happy as you are is the master thesis student that does something really meaningful. So I think you should think about whatever you want to do. Master thesis student that has something really meaningful. So I think you should think about whatever you want to do. I think that's the way to go, the way you did it?
Speaker 2:which universities?
Speaker 1:I'm curious, uh so it's duke university, georgia tech and a university in barcelona.
Speaker 2:Okay, so here we have king being a little bit more cool, and then and then good old scania stays with Koti.
Speaker 4:Ho right, that's cool as well, it's cool, but I also want to talk about how to enable one of the other tools, talking about this psychic log into safety, of being able to ask questions and not being afraid of okay, I don't understand this. So we have an open help channel that anyone can post questions, and that's basically one of Patrick's members in his team, ash. Shout out to Ash, he's amazing. Shout out to Ash, he's amazing. He has basically created the tone of voice in that channel and it's been really safe space and ask anyone. He's super fast and super knowledgeable and making sure that people really understand and when he has not answered fast enough, then always the champion has come in and answered the same question. So really, really people learn from that. So everybody from top level management the same questions. So really, really people learn from that. So everybody from top level management to interns post questions in our channel. So I've been really successful.
Speaker 2:Yeah, so, but now we're a little bit more halfway and we have a segue. Oh, a mid-session topic. It's time for AI News, brought to you by AI AW Podcast. Yeah, so we started that maybe a year ago and when we realized that we've just been crazier and crazier. So let's talk about something that caught our eye in the last week, so something like that and under the umbrella of AI News, like research, whatever, so, yeah, let's go there and then we spend a couple of minutes on each topic and then we package that and go back to the story. So anyone wants to start, anyone has a leading topic. Have you thought about, did you see something that caught your eye in the news or the media feed, or whatever?
Speaker 1:I mean for me the latest thing and I don't want to advertise for anyone, but it's the recent release of the autoregressive models by OpenAI, the image model. I mean. I had a second sort of GPT-4 moment when that happened.
Speaker 2:Let's go here, don't mind, but that's here, okay, so frame it. It's not mine, but that's fine.
Speaker 5:No, it's not mine, but let's frame. This is a good one. It's the most obvious one, I think, so frame it a little bit.
Speaker 2:Like we talked a little bit about this, but in your opinion, what's the background? What are we talking about, like if you frame it for someone who hasn completely in the know?
Speaker 1:All right. So anybody who's seen AI-generated images before probably knows how to detect them. Like people with seven fingers and those AI smudge around the edges, you know pretty nasty images that did not even look good to look at, right? So there's been a problem there and Dali wasn't very good, to be perfectly honest. That's my personal opinion, of course, but I mean, what we are getting out of ChatGPT wasn't that great. And then suddenly this new model comes out and it's completely different, right, it's completely different how it behaves and what you can do with it, and that's quite magical. The moment where you first interact with it and you talk about an image and get a change in that image, that makes sense. That's actually quite good.
Speaker 2:So the usability and the way it just works. That's the big difference. And we're talking about now the release of OpenAI chat GPT approach. It's part of normal 4.0, right.
Speaker 1:Yeah, it is pretty much. It's not a separate no, but it is a bit limited in number of requests, I think, depending on your tier and things like that.
Speaker 5:I think it's been retrained in some way. It's a new 4.0 model that came out and I think I've heard that the new model is also better at coding for some reason with this update. So it seems to be one of those cases where modalities interact in a nice way. But I'm curious to hear what kind of things did you try out? I haven't tried out so much yet with it.
Speaker 1:I'm going to skip the obvious Ghibli. I have the Ghibli angle, just like everybody else I know what I look like in a Ghibli animation movie.
Speaker 2:Thank, you that was nice? Yeah, that was nice.
Speaker 1:Yeah, whatever we tried I mean I want to hit that point a bit more what you were saying it's like it's the first time it really feels multimodal, because, honestly, it feels until now like a bit of a hack. Yes, like you you'd speak to through language and get an image back, sure, but you could feel that it was not. It was just a prompt being sent over to another model that sent back. That was that's not multimodality, right, but here we're getting much closer to it. We can talk about the image, about the content of the image, and get changes in there. It's not perfect, but it is a lot better and, and from a product manager perspective, I feel like we're much closer to the expectations people have about how these things should behave. Right, this is how you would think of it. If you talk to an artist, if you talk to someone that can change the image, you can point out what you want that person to change and you can get that. So that's the expectation, but you couldn't get that before. Now you can get it.
Speaker 2:So that's the shift. I don't know if we have the right technical expertise in the room, but I'm not the right person. But can we differentiate here? Is this the way it's done now multi modality for all this version? Is this different to diffusion models?
Speaker 5:Yes, as far as I know, this is not a diffusion model. It's the regular transformer model. Yeah but they made it work with images.
Speaker 2:So so. So maybe background here is like who's been leading the pack on image generators? I guess mid-journey, who do we think has been by?
Speaker 4:the story.
Speaker 1:There's so many.
Speaker 2:Black First Lab, yeah, but I mean, like so which one has sort of brought, got themselves into the advertising agency? You know, we had the guy here who sort of done you know AI fashion, you know. So I think that was more Midjourney was beating Dali, I don't know in this diffusion space, right?
Speaker 5:Yeah, it's hard to say, because there's also video. Yeah yeah, it's a sort of almost different game. I mean initially definitely, Midjourney was the first.
Speaker 2:Midjourney was the first to kind of get production grade.
Speaker 5:But then the for, for, and then, then. Then there came out sora of course yeah, sora is video now yeah video, and then uh, and then uh, google beat them with vo2 vo2 but this is video versus stills.
Speaker 2:Yes, so, and this is diffusion in stills in mid journey video is something else again. Uh, is this diffusion models or not in video?
Speaker 5:I'm not sure actually.
Speaker 2:I don't even know.
Speaker 5:I think it's diffusion still in the videos.
Speaker 2:I don't know.
Speaker 5:I think they just maintain some kind of coherence across frames. That's what I'm guessing.
Speaker 2:And now then, what is this then? That we're doing the traditional transformer in true multimodality, you said it right. It then we're doing the traditional transformer in true multimodality. You said it right. It felt like there was a transformer calling on a different model, maybe even a diffusion model, bringing it back and presenting it, and now we're doing true multimodal in transformer space. I guess, is that what we?
Speaker 5:I interpret it like that. I don't judge we don't know how it works because they don't say how it works but what I'm guessing is that it just takes parts of the image, encodes this, encodes it using, uh, some kind of token, so the the part of the image becomes a token, and then you just interpret the image as a bunch of tokens but why do we think this is now proper multimodal and not calling on something else?
Speaker 2:what proof do we have that that's what's happening? Because we feel it like that. But do we know? No, we don't, we don't.
Speaker 1:I mean I don't mind either way first of all, I don't know. I don't know how it works exactly and I don't think we've known before, but it feels a lot more like it's seamless right, and that's what's important. It's an expectation thing, it should work that way, and it's much closer to that.
Speaker 2:So under the hood we don't really know, but the way we expect it to feel, kind of the experience is there.
Speaker 1:It's the old quote if it's sufficiently advanced technology, it feels like magic kind of thing. But that's kind of where we're getting at and it's getting that feel, I think it's uh to think about the applications of this yes, um, there's been a lot of applications and most of them were ghibli style things, uh, but there's also.
Speaker 5:It's freakishly good at putting things at a google maps uh place, so I tried to do something really uh, there was this typical image from from open ai where you can see somebody's drawing on a whiteboard and you can see I think it's the golden gate bridge that's reflecting onto the whiteboard. So I tried to do something similar because I was sitting at uh, you, were playing geoguessr.
Speaker 1:Yeah, now hey is going to beat us there. Too bad.
Speaker 5:It's almost the reverse of you, gasser. I was sitting at the office in the Moot Galleria, where my office is, and I tried to do the same thing. It didn't turn out as good, but it definitely got the building style of Mr Shaman's Gatan and it looks believable. So that was quite cool. And that's one case, but the other more scary case I've seen is receipts.
Speaker 2:Yeah, yeah, yeah, the receipt. Have you seen that one? No, this is okay, we can segue now. What is the implication of this? New stuff, and one of the cases that really popped up everywhere is that if you have any type of software tool where people are supposed to report in images of something like receipts, bye-bye is I mean like the, the fake receipts that you can fake with a prompt that looks like ah, my expense report I I saw 10 different examples of that popping through.
Speaker 5:It's also interesting because I had a last day fika for somebody who's leaving the team. We bought cake, good, and I got the bill and I dropped it you dropped the bill it never happened to me, I did not. I did not. If I did I wouldn't say it. But yes, it's obvious that it would be super easy to just generate a receipt out of just saying what was in the receipt?
Speaker 2:where did I buy it? Yeah, so the whole deep fake has been a little bit like wishy-washy where does it go? But now, like, okay, there's a lot of our core financial processes that relies on documentation and we flip from the physical receipt to taking a snapshot. And all of a sudden that is in jeopardy. So we're back to papers.
Speaker 5:Back to papers potentially, I don't know. Thank you for that, albin. All right, let's stop there.
Speaker 2:Back to papers potentially, I don't know. Thank you for that opening. All right, let's stop there. Do we have any other news? Or I have a meta news on that same news? What was your? You had one. What were you thinking about?
Speaker 5:This was the one that I was thinking about.
Speaker 2:Yeah, the receipts.
Speaker 5:Yeah, no, the image model in general. Yeah yeah. I had some other applications, but we can skip that.
Speaker 2:Then I found someone doing a really good. A guy called Stephen Klein did a really fun. What the hell happened on the 25th of October, the meta news of all the marketing wars. So on the same day, we literally have an open AI releasing this new setup. At the same time, of course, the same day announcements was Google Gemini 2.5. And, of course, a new version of DeepSeek's open source, you know, deepseek R1.
Speaker 2:So the tricky one here is like, if you read in the roadmaps and the roadmap release notes of OpenAI, there's nothing fucking discussed around that this model is going to be dropped. So what happened? So, literally, we have a combination and this then is the argument how brilliant are they? Or how brilliant on their feet thinking are they? How you know? It's like you take this stuff like oh my God, I have two different key players who are actually doing something that is threatening my core business model. Deep Seeker AI is doing something that is like showing to be the same, on par with my capabilities, 98% cheaper in inference to run, and Google Gemini supposedly then outperformed, you know, turbo GPT-4 and several benchmarks, blah, blah, blah.
Speaker 2:And all of a sudden, now let's use out of the nowhere do funny, you know, do images that we like. And then you know the Machiavellian story let's do it on Ghibli style. So we get an outrage and we get a completely different conversation going on here on the ethics and stuff like that. And yeah, you know, you know, and then two hours into the whole game, sam Altman is typing the servers are melting. So we have the whole story about. This is crazy, we are winning and this is like this is proper Joe Labero stuff. Look over here, look over here, forget about what's happening over here. So, from a marketing point of view, from how they play this game, in terms of getting the, getting a lot of attention, getting even a lot of, you know, rage and you know the ghibli, you know, should we really do this and the whole spin-off effects of like is this really ethical? And it's that's the main story on social media and in comparison to other major launches, that was sort of properly threatening to. I find that interesting as a meta story here.
Speaker 5:Yeah, yeah, yeah. And it's the way OpenAI constantly plays upon always having a rabbit out of their sleeve just when somebody drops something, then they pick up look here, look here. But the way they work must be like they're building on all these things that are ready to launch and they're just waiting for the proper moment.
Speaker 2:But I I have the deepest respect for the marketing department and the strategy department in terms of marketing, competition I I, I don't want to, I don't want to bag them in any way. I think it's like wow, this is like a fifth four-dimensional chess. It's super. You know like we can we? You know we are taking for a ride, of course, but who is not? In every propaganda and every marketing? So, from a marketing department, the way they are playing their game, wow wow, but let's try to test it.
Speaker 1:You, you said constantly, you're absolutely right, it's been done before. Remember the release of sora look at the timing. It was the same day as gemini was coming out with the big context um, so this is, this is no luck this is no luck.
Speaker 2:This is not fucking luck. They just know what they're doing and they're doing really good at it. Strategizing on how to win the viral space Super important in this game and they're doing it really competently.
Speaker 1:Kudos to the team that's doing that yeah, yeah, they're doing a really good job.
Speaker 5:We're all asses and fools, but you know, and they own the internet Bloody but you know and they they own the internet how the hell they had a long time to build this up and they started with so what?
Speaker 2:what is your argument here?
Speaker 3:man they are losing. They were charging 200 bucks for this, okay, so now they are giving it for free.
Speaker 2:I mean like so. So if you look at the business game and all that, there is a bigger game and you you know, are they on the right track? Blah, blah, blah. I don't want to go there, I just want to go. I'm used to reflecting on their marketing prowess, how they're doing their PR and how they're doing their releases. Is it the? Marketing genius or panic mode of action.
Speaker 2:It's a combination. I mean like so marketing genius can save the. You know it's old school VHS versus beta. You know who won. Vhs won Better marketing. Everybody knows Betamax was better technology, so I don't know.
Speaker 5:So let's see, man, I think it's a valid point that it's getting closer to panic mode than marketing, because the competition is catching up very fast. But what I was just going to say is OpenAI had a long time when it didn't release anything After Gypsy 4, it was silent for a long time because they knew they were so far ahead. So my thinking is, they built up all these things during that time and now everybody's catching up to them.
Speaker 2:Yeah, so I was not trying to get into the topic of who is winning or who is the best edge. I was used merely reflecting on on the competent, competent, uh approaches, in my opinion, to how do you stay on top of the media buzz like that, and that was my only reflection, but if you want to look at winners and losers, I mean think about who started all this, who built the technology first, who knows this best?
Speaker 4:That's Google right, of course it's Google. They were first.
Speaker 1:And they're pushing the boundaries of this space as well.
Speaker 2:They've been fucking up the marketing.
Speaker 1:Yes, and they're failing there.
Speaker 2:They've been failing in their fucking demos even, but right now Google is on top of it. Now they're bouncing back.
Speaker 5:The newest model is best at coding. That's no small feat.
Speaker 2:For sure, we know so much fun Traditions in tungsten man. It's still impressive. Look at this, this is. I was playing around with it, but it was Sora new release as well, so we didn't talk about Sora, so Sora is open. Ai. Sora has been for a long time, but they release it. So another story now is that on the 25th they also released access to Sora. More, yes, everything is there.
Speaker 2:And the guardrails are complete. On the 25th they also released access to Sora more and then okay. So the final game here, if we're going to talk about this news, about starting with the release, is something about that. If there were any guardrails before in OpenAI the climate I guess with Trump and with Grok and with everything else there seems to be less guardrails in what you can do, right. Did you pick up on that?
Speaker 5:I've seen news about that. I've heard an interesting take on that. The guardrails are being lowered, but it's also probably the fact that they can do all the equivalents of RLHF Afterwards. They do it now synthetically so it can be done much quicker and they probably found a level which is acceptable and doesn't annoy people as much. I see the point of that. Garbage are lowered, but there's still limits. There's like you can't do actual porn with, with sora, for example. No, you can. You can do uh, uh scantily clad, but you can't do naked, or you can.
Speaker 5:If you can do naked, it's not porn at least. So there are guard rays, but there are being lowered. That's that's my sense of it.
Speaker 2:Well, do you agree? Did you pick up on the guardrail? Uh, lowered like this, because that was the story that was also floating around as a meta story I mean in my usage day to day.
Speaker 1:No, not as much uh for sure. But I mean to me it's like the, the recent meta uh shift right in their, in their internal policies. They were the first one, I think, to really make a splash when they shifted, and I think it's quite a natural sort of evolution of it. It's like, okay, we don't have to put all that much effort in this, specifically not moderate everything that comes out of it. We can let it a bit more loose, and to some extent it's a good thing, I think, for the models, because the models perform better. When you don't Like, if anybody has tried playing with the system prompts, you can mess up the output of the model and you get really poor results out of it, right?
Speaker 5:so it's really a matter of how it's done and I guess it's also adapting to reality, because what have there haven't been this uh, this uh disaster events that everybody was expecting at some point, where ai is super dangerous and I? There have been a few issues, but not that, because it's this overreaction I mean like.
Speaker 2:And then you have the whole story of free speech and truth from Elon who basically says is what is really dangerous with having frontier models, that is not valuing truth most? I mean that's his argument, right? So I don't know. And then so you can completely see how it follows the EU political climate and the political climate left versus right, pc's Vogue stuff versus not right, and there is something going on in America in terms of shifting LLMs are political.
Speaker 1:I think that's the conclusion Of course they are.
Speaker 2:Anyway, I think that was a cool new section. It started with the image and we could flip it and it was some cool stuff. Let's go back now. I actually want to sort of go back to the core sequence and core topics of Cake, and we talked about human-centric approach, we talked about the pilot project, we talked about the mandatory intercourse, we talked about Cake champions, hands-on workshops. So we have a couple of more topics that I think are really important that you highlighted here in your cake approach. And, I repeat, cake was the acronym for Cognitive AI. King Enablement. That's cake. Yeah right, cake is cooler than that. Do you need acronyms like this?
Speaker 4:Yeah, I love it Especially if you can buy a lot of cakes you need. Yes, yeah, I love it. You need to brand shit.
Speaker 2:You need to brand shit. I like it, I have some nice cakes, but let's let you have some core topics here that I find. Let's go to the use cases. If we try to be linear, yeah, when did you think about, or how does the use case angle come in here? What is?
Speaker 4:that all about. I mean, it's the same thing as with the cake, with Croft cake crafts, that we realized that, since it's the white paper, you can do whatever you want, but if, and and you don't know where to start. So but if you, if you collect all the use cases for each craft, then you have a starting point. People can start to think about how they really can use this, and that's why you also need to collect them, because Build a library?
Speaker 2:essentially yeah, basically yeah. And how did you? Practically? What is this library? Is it a Wiki or how? Did you do it.
Speaker 4:It's been many different attempts. We have had PowerPoint presentations and we collected them in people's heads. We have had them in documents and we have created the GPTs To ask the GPT for the use cases in Minecraft.
Speaker 4:Yes, exactly, but I also think that, coming back to champions, that them being some kind of hub that you can ask, but it comes down to the use cases are super important, but you don't have to collect millions of them, you just need a couple of. So you get each person to understand ah, I can just free my mind and do these things, but within these boundaries, because this is what you can do and this is what you can do and this is what it can't do. Because sometimes, when you get this white paper, oh, I can do whatever I want, well, you can't right. It's still a generative AI, it's still an element. It doesn't do everything. It does some things really, really good. And then, for that case, use cases.
Speaker 2:Yeah, so use case approach and try to collect it in all the way and forms.
Speaker 4:Yeah, so that's so. Then we found really really cool use cases. So, especially that someone that's we as a central team, we would never never thought about these things Like from the globalization team. They have a lot of really good use cases.
Speaker 1:And we cannot not mention custom GPTs but also assistants and agents, or whatever that's becoming. Those are really relevant. This is how people express their use cases in a lot of cases. So, at King, after one year we had about 2,500 of these custom GPTs built by the community users themselves, and that's a token of like people want to make sense of this. They want to personalize these tools so that they're usable in their space.
Speaker 2:So how far did you come down the point of having this other conversation when this is kind of to the place now? Oh, now we need to have some guardrails and frameworks and we need to create a marketplace and we need to deploy the custom GPTs in a good way. Or, you know, have you gone down the marketplace idea here as?
Speaker 4:well the interface for GPTs in OpenAI Azure GPTs.
Speaker 1:OpenAI. If you're listening, you have a problem. No, no, no, I have the solution. You called me afterwards.
Speaker 2:Yeah, that's really really bad.
Speaker 4:You know, what I did. I created the GPT to find GPTs.
Speaker 2:But this becomes the next topic, right? Because very soon we get into Excel. Hell for.
Speaker 4:AI and Gen AI. Yes, exactly, we're getting there.
Speaker 2:It's happening now, right. So, and that's maybe something you know, how do we create the governance and the frameworks?
Speaker 4:That's also why I think that it's quite important to not make it too complex. And also, if you have, you don't have to collect thousands and thousands of use cases. You can collect a couple of them and then make sure that those are good and it also comes down. You don't have to create a hundred different kinds of workshops. You don't make one or two that are good and then you can let people do whatever they want and whatever they need for themselves. Then it doesn't become too complex because you have all these islands that can have loads and loads of different use cases, but you don't need all of them in the central place.
Speaker 2:No, but it's a way to find your way around so people don't build the same shit twice or three or four times. Yeah. And then comes the next topic, when I think where is this going? Like multi-agents, bringing a couple of agents together, yada, yada, yada. Then the whole interoperability topic shines with this ugly rear end stuff like this, right? So it's a little bit like generation one. Let's get people involved, let's democratize. Now comes generation two. How do we scale this? Maybe not the first topic you start with, but you're getting there, you're already getting there.
Speaker 1:We are there, and I want another piece of news. Maybe very relevant for all of us, is the recognition by OpenAI of MCP as a standard, and that's really important. What is MCP as a standard? So well as a standard, it's becoming a standard de facto because a lot of companies are recognizing it as a way to build interfaces between LLMs and, essentially, applications, and could you elaborate on MCP specifically?
Speaker 2:What does it stand for? What does it mean?
Speaker 1:So it's the model context protocol. Is that correct?
Speaker 2:Yes, MCP and who came up with that? The MCP is important stuff, guys. Yeah, it is.
Speaker 1:It comes from Anthropic.
Speaker 2:Anthropic's MCP paper. Important stuff, guys. Yeah, it is. It comes from Anthropic Anthropic's MCP paper. When was that coming out?
Speaker 5:It was last time I was here. You talked about MCP last time you were here.
Speaker 2:And you sort of. So I'm like we're talking about shit that will make a splash.
Speaker 5:And now we're coming in there. It took some time. It started out, was it in autumn? I think yeah in autumn?
Speaker 2:I think yeah, but please back up the tape because this is really important stuff, that if you don't see this when you're a newbie, but if when you get to 2000 GPTs, this stuff matters. So what is MCP? What are we talking about? You just give the help each other out. This is good stuff.
Speaker 5:Yeah, do you want to start?
Speaker 1:I mean, I just gave the name. For me, it means that suddenly we can interoperate, right, we suddenly can connect things together, and that's the essential part that we've been missing with LLMs, right, you were dependent on whether things had an open API spec that you could work with, kind of thing, but this is really like standardizing that Now you can build your MCP service, your MCP clients, and then you can go ahead and integrate things together.
Speaker 2:So the MCP protocol. So take us back to the paper. What was that all about?
Speaker 5:So it's a way of standardizing how LLMs can choose between tools, basically. So it's the tool calling and it says that this is the place to go to when you want to ask questions to a database, for example, and this is the place you go to when you want to search the web. That was one of the initial cases that Anthropic built for Cloud or an integration to, in their case, the Brave search API, in their case, the Brave search API. So that then becomes a standard and you can download this MCP server, run it on your own computer and it would use your own API key to talk to the Brave API. So if you're building a lot of stuff, then you can just reuse this connection once you've built it and you build another application that also needs to talk to the Brave API, Then you just talk to the model context protocol for that specific service. So it's a way of designing reusable API integrations.
Speaker 2:You can say yeah, and I think the protocol understanding is key. I don't think it's enough. I think you know, when we look at data product management and we go into poor data engineering and we talk about data product, how do you put the wrapper on to make the data as a product interoperable? You know the principles of data mesh and all that. I see the same stuff happening with agents that you need to build the wrapper around the fundamental model as a data product, as an agent product. The protocol is the input, output, port, storytelling, how we can connect to it. But you need the metadata, you need more right To be able to consume it safely. So I think the way we've been going at it you know, from a data product point of view and all those ideas, how to put a wrapper on things is gonna be critical also for agents do you agree?
Speaker 1:yeah, I do. I mean, and we can say this way, we've we built the agent interface right that this is what happened. Yes, and this is really essential starting somewhere, yeah, starting somewhere. So we have an interface, and there's a lot of things to be figured out still, but it's a it's a great starting point and the fact that it's recognized as a de facto standard is a great step forward. It's a great step forward. Now we can work and start building things on top of it, interoperable and consumable shopifiable.
Speaker 2:Someone tried to explain this like okay, so you sell Snooze, snooze, okay, if you want to sell snooze on Amazon, come on. You need to put it in the fucking container and you need to have a price on it and you need to put a declaration on it that is true snooze. Then you can sell it on Amazon. You can't go and use, you know, buy nicotine off the shelf, you know. Or someone said flour. You don't go to Ica and buy flour by the kilo, right, it's in the, with a declaration that means you can check it out through the checkout. This is what we're talking about in relation to data and in relation to models, agents, gpts, and I think that's what's needed.
Speaker 4:That was a good explanation. You like that right.
Speaker 2:I like that one you can't go to EEC and just buy off the. You know, I'm sure you could buy flour by the gram 100 years ago, yes, but you don't do that now. You buy it in a one kilo package or 500 with the declaration all this. So what is that wrapper you need? On GPTs, on agents, on models, on data this is my vision of this. Do you agree with it? I mean, I think this is where logical projection of what this is going. I don't know, I'm on a rant mode here.
Speaker 1:I think you're talking about standardization, and I think that makes a lot of sense, right? It's like standardize how to interoperate, and this is how we're going to start seeing more ambitious type solutions take shape.
Speaker 5:Is this still within the CAKE framework or is this like the next iteration we're talking about now?
Speaker 1:Well, CAKE was such a good acronym, so we're going to keep it around for a while. But it's definitely on our radar. It's going to be the next big question. We have a ton of internal system. We have a ton of internal knowledge and data sources and everything all over the place. How do you make all of this talk to each other?
Speaker 2:But I like it because it fundamentally is all about enablement and increasing the productivity of the people using this, and as more mature you get to, it's like okay we had an innovator problem. That was these five frameworks. Now we may be going into scaling problem, so it's a little bit like the cake framework needs to evolve. Well, are you in the innovator space? Are you in the growth hyperscaler, internal hyperscaler? You know widespread adoption, you know so. I find this is cake, but it's like moving with the adoption lifecycle, I guess. But you had.
Speaker 1:I mean, I used to Call it a maturity model. Right, it's about maturing and there's a lot of things to it. I mean, we're talking like the initial reaction. We had LLMs at first and they could do things. It was impressive, but very quickly people started asking questions like why doesn't it know about the stuff I know about? Like, why doesn't it know this? And then we started talking about RAG Like that was the answer. We're going to make data available to the LLM when it needs it, kind of thing. And make data available to the LLM when it needs it, kind of thing.
Speaker 1:And that was, yeah, a step forward. It allowed a lot more things that we couldn't do before, and that is great. It actually moves the needle a little bit In some use cases. It's pretty good, but it's still not the full answer, Because now okay, now we can retrieve things. Okay, retrieve information is important. Now how do we make information usable? And then how do we start automating on top of it? And these are like steps of maturity and adoption of AI, so you could draw a line of maturity. But let's do this. I mean like.
Speaker 2:So let's just wrap up the quotes. There are two more key topics on the cake model and then I think we should move into where is this going? Scaling, adoption, la, la, la. That becomes sort of where is, where are we going? And then we can go even more philosophical. But you had you had two more key topics. I mean like management support we mentioned, but the two topics we haven't talked about is the help channel and the metrics. That was sort of in the pdf that we you sent you know that you, yeah.
Speaker 4:So the help channel that's meant that we mentioned it quickly it was uh uh, the slack channel where uh ash uh was the king and helped everybody to uh to succeed, basically. So that is super important, but also with the psychological safety that you really can ask any question.
Speaker 2:But the community and how to build a community, and using these core slack or whatever yeah is a quite core strategy.
Speaker 4:Oh yeah, absolutely absolutely, because, uh, we don't know everything and and it's all about the word multiplying, uh, so multiplying ourselves, making sure that people are confident enough to answer questions and also, uh, confident enough to uh that they are allowed to answer questions, and also confident enough to that they are allowed to answer questions as well. So that that's super important. So community building is super big.
Speaker 4:Yeah so so we, you know, so I I think also with within these crafts. I mean, we have a. They build communities within this craft as well, that they're gonna help each other and share their knowledge and different that we have not been a part of. So we, these things have popped up across the company that we didn't. So we realized, oh. Two months later we said, oh, someone is doing something amazing over here. Let's uh, let's try to see what that is. And oh, that's super cool. Let's, uh, learn from that. So these things have popped up all over the place, which is super cool.
Speaker 2:I think this becomes obvious when you go grassroots, as you explained. But if you look at this, how big corporations do this and they kind of oh, we do the training, we do that, and they don't realize that we need to have a federated sharing and governance. We need to have a knowledge sharing mechanism by design.
Speaker 2:So we are talking about oh, it's so important that you share, but we create no arenas for it and we don't design it so if you want to do this smartly, you need to design community approaches, and I I got the bloody book on this how, how to build communities, the super users, the champions, and how to create la, la, la, la la. And this is super important, and I see large corporations. The grassroots are doing it because they want to, but is it a strategic investment from the top? No, they don't get it at all. And you're just highlighting now this community knowledge sharing mechanism is a strategic design choice. Yes, you agree as well.
Speaker 5:Yeah, that's super important. That's my experience as well. The grassroots thing is super important.
Speaker 2:And if you know, you know anybody who has coded anything. We're living in open source. You know everybody that knows anything about engineering is part of a Slack channel and is part of an open source community of their preference. And wake up enterprise. You know, you guys get it. I mean like you have it by birth. You know in terms of being a digital native, but this is not. This is underground stuff since gone.
Speaker 4:Yeah, maybe it's that, maybe it's. I mean, I'm very much open source and, coming from the maker community, I was part of the ones who started Stockholm Makerspace. So for me it's like yeah, focus on things just being able to create stuff, and this is the digital way of just being able to create stuff.
Speaker 5:So enablement and community, some kind of closely linked stuff here, right, yeah, absolutely yeah, I think there's also some element of, uh, having fun and not not putting it so much into productivity at once. It's like you have to unlock the creativity by sort of finding people at their own terms, where they are and what they want to do, and that's hard I find.
Speaker 4:I think I mean also fun is super important, because when do we as children learn? It's when we play. So if you put in the concept of playing, like we did in workshops, as a play, do fun stuff, then your brain shifts into learning mode and then you learn stuff and then it doesn't feel organized hackathon.
Speaker 4:Yeah, we did a, really fantastic hackathon, uh that uh last uh, november, uh. So, with the whole company company came together and we created the hackathon, where we grouped all different crafts in groups of eight. So we had everyone from Shodol, our CEO, to everybody was in there and they created features for our game Not to be published, but just to give you the idea and they got two hours, three hours, to create something together with Chachapati and it was such a great. Everyone was like and they had so much fun and they presented and it was so creative. Some things was amazing and some things was not amazing, but they had so much fun and they learned. And also with building these groups together, we didn't say that, hey, you developers do this and you HR do that, but we mixed them together, which meant they built bridges across the companies, so they know more people and you have this experience together.
Speaker 4:So these hackathons, if you listen to it, do hackathons, because it's so good, but let's go a little bit deeper.
Speaker 2:The hackathon you know the research, everything says like this, like how, what's the difference? And we teach kids, some people learn and some people don't. What. What is the fundamental stuff to be in the context of learning is the inner motivation, is the inner engine right? And I can see it in my children was where, where it doesn't matter how smart the kid is, it's about his inner drive right. And then you can go on with books like atomic habits and talking about how do we do drive motivational.
Speaker 2:You know behavioral change, and how do we? If you want to start stop smoking mentally, I know I shouldn't snooze or smoke, but I need to want to stop it, you know. And all of a sudden now, this is not rocket science, it's fundamental behavioral science, change management stuff. And then all of a sudden, now what we are doing is, in my opinion, we are thinking about how to flip from pull sorry, from push I'm pushing something down your throat to what I'm doing to creating pull. Yes, I want to eat this. I want to eat this In its fundamental mechanism right From push to pull.
Speaker 5:Isn't that? I agree, I agree.
Speaker 2:Fairly simple, right yeah?
Speaker 5:I can only add that, from my point of view, that's the one component. I would say If you're starting out with something like this, try out hackathons. That's the first thing to try out. Do you agree with that?
Speaker 4:Yeah, some kind of if you don't know anything, then you should just have a small intro as well. And also you should make sure that the groups that are included, so you have to think on who's part of which group so they learn from each other. Depending on what I mean, we have had hackathons where we had more competition style. Then maybe you should want a specific group, but this learning way you have to think about this.
Speaker 2:I'm happy. I don't care if it's a hackathon or not. What I care about is that the strategic approach that we are taking now and where we can now be creative and creating that in many different ways, where a hackathon is one vehicle is the fundamental design choice. Do we go for push versus pull? I think the strategic design choice of enablement lies in the push versus pull. Do you agree with that?
Speaker 4:Yes, of course there's either or, but because we said in management support. We said in management support that we also had OKRs that said that each part of the organization should use this amount of Chachbi each week, which meant that it don't use it as much. But instead of like, go whip, don't use it, we said, okay, they need help, so how can we help them? And then it's also from the management level who measured on the OKR. They also felt that we, instead of working on this delivery, we need to focus on understanding.
Speaker 4:Exactly, and that wouldn't have happened if those OKRs would have been in place. So it was tough work, but there are several things in there right now.
Speaker 1:Definitely, there's several angles, there's a push, there's a pull. The hackathon itself, if we're on that topic. It also demonstrated that leadership was supporting this, and we got this quote from one of the champions who was simply saying I want to thank leadership for giving us space to play around with this in our work time. Making the space for it is a token of yes, we want you to explore it, we want you to figure it out, Exactly so.
Speaker 4:The whole company working on this, playing with this for four hours. That's a and about communities.
Speaker 1:we shouldn't forget to mention we have another channel called Play Fun With AI. Yeah, and that's a pretty good channel also where people just you know show the latest thing they've toyed with or found with AI.
Speaker 2:Yeah, and it's 800 members strong or something like that, which is a lot. And then you come to one topic here it's highlights in the kick model as metrics, and you already now mentioned OKRs. So how do you communicate or how did you go about discussing metrics? And you know, on soft side, on hard side, OKRs. So what was your strategy to metrics? Because it's in there. So let's unpack metrics. What do you mean?
Speaker 1:there. I mean, there's the obvious sort of OKR driven type metrics, and Kalle suggested it was very much driven on adoption at first.
Speaker 2:So give us an example of a concrete OKR A concrete OKR could be.
Speaker 1:This many percent of the organization should be using AR on a regular basis and that again, the enablement effort made a lot of sense. We were pushing enablement to enable as many as possible and if that had an impact, we would see people adopting it more and you were quite adamant that your OKRs were also defined in a way that you could also agree upon how to measure it.
Speaker 4:Yes, yes, you were that rigorous, absolutely.
Speaker 2:Because I've heard OKRs and I've heard plans and goals, and then there's no rigor on how to measure it. But you were trying to be even if it's a survey, I want to know how you're planning to gauge the success.
Speaker 1:For sure, and this needs to come at the time of creation.
Speaker 2:Yeah, exactly, gauge the success for sure.
Speaker 1:And this needs to come at the time of creation, exactly, okay, all right, you want to talk about it early so that you have a good idea exactly you're going to do it going forward. But you.
Speaker 2:That is part of the process to define something but also think about and agree upon. How do we measure that exactly? Yeah, and that's not easy it is absolutely not easy.
Speaker 1:and even if something sounds easy like this, like, like talk about adoption on what level? What are we talking about? Who are we talking about? Just saying who are the employees we're measuring on Exactly? That's not an easy question to answer. It's like what is the base of measurement? And all those questions need to be explored before. But this is key this is.
Speaker 2:I think it's more profound because we cheat a lot around these topics, right?
Speaker 3:So why is it important? Why is it important that we actually take you know seriously why shouldn't we cheat here?
Speaker 2:Because I heard start hurting and in the end you know I gave up on a couple of occasions where I fucked this up. Why do?
Speaker 1:we cheat is an excellent question. So why would we cheat in a situation like that? Why would we want to do that? I mean, let's look at what the purpose is. Why do we have okrs and things like that? Well, we have it to set a direction and push us in a certain direction, and if we agree on the spirit of the okr, then it's much easier to not cheat, because then you can strive for that spirit and not just hit a target right, yeah, and and as long as the efforts that you put in are in line with the spirit of the okay heart, then then it's much more about it but I can't believe I'm saying these words but this whole thing that we are not taking
Speaker 2:measurements. What do you think, jesper?
Speaker 5:I mean, like, you know what I'm saying, right, it's really hard totally, I'm I'm trying to find my question that I want to ask here. Um, it's like this okr, was that put on your department?
Speaker 1:or on the whole company? Good question, I was the whole company, but we were the owners of the okr.
Speaker 2:Did you frame it, did you? You happened to wait. So I.
Speaker 1:I myself was not involved as much as I would have liked. Frame it for you Not going to rant all that much but but yeah that that process could be more transparent.
Speaker 1:And again here it's about. It's about the spirit Like what is it we're trying to achieve, that we're setting this OKR for, and if we can have that conversation, then it's a really helpful conversation. It's enriching, it gives purpose and direction to people, so it can have a really good purpose. And, to be honest, I never liked the adoption-based OKRs and we had two in a row for a whole year and I'm like that doesn't make a whole lot of sense. It's like what are we looking to achieve with this? What are we trying to do? What's the value?
Speaker 2:Yeah, what's the value? Let's talk about impact instead.
Speaker 1:Why don't we talk about impact? What is it that we're trying to achieve?
Speaker 5:Because you need to be able to talk about this what I wanted to get to, and I think it's a super interesting discussion that we already had. But what if it was just your OKR? Would it have changed how you had worked? If it wasn't the whole company's OKR, what would that have done?
Speaker 4:But it was the whole company's. Okr it was just that it happened that we were the one the central team, and I don't even think that we were the one team that. So this is the team that's going to own it, but then at the end of it, we are the enablement and productivity team. You're pushing it. So who's going to make this super important high-level OKR happen? It's us seven people that's going to do that, so we had to push it.
Speaker 2:The key question here is how this is getting screwed over when you're getting down to siloed even competing OKRs and you have a ridiculous OKR on yourself but no one else owns it or feels for it.
Speaker 5:This is my question, thank you.
Speaker 1:But that needs to be baked in, right. And so that particular example is a good one of. We owned it and few other teams cared about it. Right, and that's not great, because you're trying to make a change and if that's not clear in the intent, then who else is going to care about making that change?
Speaker 2:Right, and that's kind of what needs to be there, because then the tricky point is then that you own this OKR, which is a KPI across the board. Yeah. But could anyone of all the other ones within any way measured on it or had a performance metric that looked bad when they didn't go in your direction. So this is the tricky point, right when the central AI team is getting a metric and it goes actually against their metric.
Speaker 1:No, if it goes against, then we cannot. So this is a tricky one right.
Speaker 2:But did you have any connection to their KPIs? I mean, that's the core question here, or not? I mean, you had yours and you were pushing it hard and you were succeeding with it, but were the P&L owners, so to speak, measured on the same topics in any way?
Speaker 1:I mean they were if they had adopted their OKRs to align with those as well. Like if they had officially stated how they would contribute to that OKR, then there was a natural point.
Speaker 2:So this is a connection of KPIs that should have, could have, would have.
Speaker 1:Yeah, yeah, but that's that I mean OKRs should do that, they should trickle down, and then that's the idea. The question is, do they do it or not? If they don't, then you don't have a natural connection. You can still point to the top of the AR and say, look, it's up there.
Speaker 2:Yeah but the connected dots lineage is not always there. Exactly, all right, a lot of good stuff here and we are running out of time. I mean like we are two hours now and we haven't even got to the philosophical questions.
Speaker 5:So let's end with that.
Speaker 2:No, I want to.
Speaker 4:The segue is Before we leave this cake topic, I can share all those things and PDFs and the presentation and everything with you so you can share with me, share it with us.
Speaker 2:Or, even better, I'm going to do a link. I'm going to do a link, I'm going to do a on LinkedIn or whatever. I'm going to talk about this and I'm going to ask you to share it in the comment section so it becomes even truth.
Speaker 4:That is your stuff and you can.
Speaker 2:You know this is good stuff, but it's not rocket science but it's also we've tested it and it works 24, 36 months, because the speed of invention now is insane, right? So where is this going? You know what is on your radar. If you want to predict where, you need to take this.
Speaker 1:So we have two different focus and I think it will make sense for both of us to answer on this. I'll focus on the application side of things. So part of our responsibility as a central team focus on ai is to stay on top of it.
Speaker 2:Essentially so that's for us to make sure that we are aware of what's going on and we can bring the organization with us um and what does that mean like keeping a track of what is on a current toolbox in and out, new, how our toolbox involves, how we're gonna, you know what, what?
Speaker 4:do you mean staying?
Speaker 2:on top.
Speaker 1:What are the?
Speaker 2:trends in the market, of what is on the current toolbox in and out, how our toolbox evolves, how we're going to. You know what do you mean? Staying?
Speaker 1:on top. What are the trends in the market? What are we seeing, sort of just recently, with the reasoning models? Like that was a big change right and being aware of that, understanding the implications, sharing that knowledge, like making sure that everybody understands sort of what is happening. And see how that impacts. Are we missing?
Speaker 2:toolbox. Explain why.
Speaker 1:NVIDIA stocks behave the way they do. You know like things like that, but no, but really it's like it's about you know, allowing people to digest all this, because it is hard to keep up with it. So it's part of our responsibility simply to do that, to just let's try to help the organization keep up with it, so that we can all be part of this journey and and what do you so as a follow-up to that?
Speaker 5:so where do you see things going, for example, with the reasoning models? Um, I have this feeling that we're on a brink of something really profound in this second axis of scaling, opening up that you can.
Speaker 5:You can continue to train on the reinforcement side and do things during inference. Yeah, and if you try to like, squint a little bit and look like six to twelve months into the uh, what do you see? And do you see that there's a big gap between, like, the people who know and the people who just use chativity as it was originally put into, uh, the chat gpt website? I?
Speaker 1:I see that there's a big divide there but potentially yes, and I think I think there's a challenge there to be addressed through learning. You know, we didn't say the words, but earlier when we were talking about training and so on. It's about improving AI literacy, right, getting people to understand the topic is really helpful, to understand its limitations, but also its use cases and so on. So it's really essential that all the know, all the trainings that calvin and his teams does is, you know, helping people keep up with all this. But there, but there are some potentially profound things happening. I mean, I don't want to make predictions about the industry because it's very hard to say how quickly this could change over time. Like, are we? I mean, things are moving pretty fast, but is it accelerating dramatically or not? That's a little bit unclear, right?
Speaker 1:Reasoning models are incredibly capable suddenly, and they can do things that LLMs really couldn't do not too long ago. And how much does that change the game? Well, really, it's a different use case, right, it's a different area of application. You have to think about it slightly differently, and even you want to interact with it differently, like this whole chat interface. That's a big question, right, does that make sense? Not always it doesn't, and it's different for a reasoning model than for a traditional LLM. As a matter of fact, providers keep both models around, right, it's not one replacing the other. They have different purposes. So maybe a bit more specialization going on in these areas, with improvements in capabilities most likely, but also like multimodality, of course, like better or better impression of multimodality, is what I think we're going towards.
Speaker 2:And how far have you discussed that? Okay, Gen AI as terms of having a toolbox that you prompt, that you know is quiet until you prompt it, and stuff like that? How much are you thinking about agency, that we're basically handing over agency, the whole agentic thing and the whole thing about I'm actually I want to outsource part of my decisions, and are we going down that path as well, or is that you know? How have you thought about that part?
Speaker 1:potentially right. I mean, it's becoming possible for one, so the capabilities are getting there, so let's explore it. But again, here it's like what do we use it for is? The essential question is like how do we apply it and where do we apply it? I mean, do you want to take your most important business decisions based on the system? Probably not, we're not anywhere close to that. But do you want to gather information that allows you to make a better informed decision? For sure, so long as there's some level of transparency and that you understand what's going on under the hood, right, so that you understand what's going on under the hood right, so that you're not trusting just generated data because it's a as an approach.
Speaker 4:I mean that's so we also have a bad approach. So that's basically what we do. We had the cake project and understand how we should use ChatsheputDN or if, and now, with these things, come along, we do another pilot because we really don't know. So we so we use, we take, we do a pilot to see what can we learn, and then we might come in in a year's time and a half years time. So here's the toolbox for for this, because we don't really know, but we will learn, and then and then. So your, your team, you spearheading it, you have your pilots, and my team is the enablement team. So we're trailing along and see what we can get from that, and then when and if we're going to do this in the biggest, then we have something that we can enable the rest of the company with this.
Speaker 2:I want to test my prediction where this is going and a mindset, a mental model about this that might help. So I think the first generation now, when we have started to do AI literacy and we're starting to build in just GPT enterprise programs, like you've done or what I've seen at Scania, I think the first frontier has been personal productivity, in the sense that we are allowing people to increase their personal productivity. A little bit like you need to be able to use a computer, a PC, in any work job you do, you need to be able to use and build personal GPTs. Okay, I think there is a level here where, if you really want to unlock proper productivity, any type of enterprise settings so let's say, anything more than 500 employees you are part of a team. Even if you were in your craft, the fundamental flow, effectiveness of the company comes from your team contribution.
Speaker 2:So I get to the example where, okay, someone was able to write a report faster, but the decision making about the decision, you know the report was part of a decision process and the decision process as such has not sped up. So I think there is. There is the next level here how we, how we're going to discuss, even if it's just Gen AI, let's, let's forget about again Dick enterprise grade. So how we get to uh workflow, uh engineering and agency, which is then another story, right, because then we need to start engineer agency around the gpt's in relation to the teams rather than the individual do you see this for sure, and and the importance of this enablement program that we've been running is like starting grassroots building, you know, literacy on individual level.
Speaker 1:And and we always had in mind that the next step will be like, what about teams and organizations? Like, how do we target those and what do we focus on when we do that? What does that mean? And then we started talking about processes and workflows and different kind of things, and not the individual work that's being done, but rather sort of how do we, how do we enable?
Speaker 2:people. Yeah, so I, I made a ladder for this as an ai excellence program where first, first, the first line I draw was personal productivity versus enterprise. Yeah, great, so. So then it's like mastering gen ai for my personal productivity, and then what does it take to master it within a team, and then between teams in a business unit, and then enterprise context, and then all the way out to marketplace in industry ecosystems, stuff like this, and I think it each step you take here there's a new, new type of complexity in terms of interoperability, governance, blah, blah, blah. Yeah, of course, so does that make?
Speaker 1:sense. No, it seems like you're on the same thinking. We're definitely on the same thinking, and that's kind of also when you look at what you're trying to effectively impact, like what you're measuring this on. That's also an important component of this, and if you look at an organization or a team, then suddenly you have much more sensible measurements, like I think, of productivities in these steps, like if we talk about developer productivity, for instance, you can imagine a four-box waterfall going upwards if that makes any sense.
Speaker 1:You start at the effort then you have the output, then you have the outcome and then you have the impact, what matters most.
Speaker 2:I mean it's obviously the impact. It's the same as the objective output outcome.
Speaker 1:Yeah, but in that model it's like where are we focusing on right? We focus on the effort, the effort that people put in, and then, if we could take one step further, we can go into output Like what is Do you have a?
Speaker 2:model for this. Have you modeled this, how you're thinking about it?
Speaker 1:Yeah, totally, and that's kind of the idea. It's like let's see if we can move up this model so we improve the relevance of the impact that we're having.
Speaker 2:But okay, so we're not going to go into the ROI topic in detail, but this is why this is such a bullshit conversation If you don't sort out output improvement versus outcome improvement versus objective improvement, because you have someone who has a bottom line objective of you know. We want to have a better roi over here yeah but that is really as a a result kpi of these levers yeah of these levers?
Speaker 2:yes, so so to to connect improvement in output to outcome? Yes, this is at least three steps. Yeah, for sure, that's. That's why be very careful how you put ROI metrics on this, my friends.
Speaker 1:No, but you see what I mean it makes total sense, and but another way to put it, and very so visual way, is like imagine people running. It's like what are we doing with this thing? Are we making people running faster? Cool, but are they running in the right direction, exactly? Well, I don't know. I don't know, like, unless you make sure that you know and you can define that, and then you can start talking about potentially the impact that you're having with these changes.
Speaker 5:I have a question First back to you and then you can pitch in. So you mentioned that we want to increase the team productivity. What do you think it takes to impact the team productivity, and can we do that with the same processes that we currently have, or do we need to restructure how we work? That's my take. Restructure how we work that's my take. I sketched it out on a piece of paper, a napkin, when I started at Volvo Cars in my new role that the teams will need to change. We will need to have smaller teams and we need to be end-to-end within the teams. We need to be in the enterprise setting. We need to be more like the startup companies which have faster decision routes. Everything needs to be able to take this individual speed. We need to find a way to model that to something that can work in an enterprise setting.
Speaker 2:Now you're getting into DADAC's research for the last five years and you're getting down to the world model, all the way down to the heuristics we are using in order to steer an enterprise, and we jokingly talk about the going rate of how we steer organizations in a functional division of labor setting. You can summarize that as the MAD heuristic management of agent deviation. So you have a principal who is the boss, who decides that we should do a process in a certain way and then we manage deviation next to that process. This is manufacturing right. How much yield do you have in the process? This is Tayloristic, scientific management. It works well in a slow moving context when the process is stable. It doesn't work so well in a highly uncertain and adapted environment, because management of agent deviation means that you are killing both positive and negative deviation.
Speaker 2:So you need some. You need another steering model at the fundamental core of economics and we call we call the new steering model of the future the booster-reacher heuristic. So then we can go down this research. You need to go. Then we can read a couple of papers on that and we getting ultimately. Then we can read a couple of papers on that and we're getting ultimately to something that we would organize agent-based, and this we call it agent-based organization. Before agentic was a thing, so it has nothing to do with agentic workflows, but it has to have proper agency, which means autonomy and alignment between different teams. You need to have a way to frame what we call two core dimensions of understanding agency. We call it Puma Puma. We call it Puma Puma. You need to have a team that can have purpose, purpose mastery and autonomy cross-functionally within a team. Whatever task they are set to do, they need to have Puma in order to solve it.
Speaker 5:End to end.
Speaker 2:End to end. So that is very much a product-centric mindset in each team, and then, ultimately, the way you organize needs to be then product-team-centric, and then you can start thinking about the boundaries of that purpose and autonomy. And then the second core dimension you need to understand is that in order for you to be an autonomous agent, you need to make sure that you are not a sovereign agent. You need to understand that autonomy is in alignment with other agents. If you're a pupil in a school in your class, it is not anarchy. You need to act within that setting. That goes for every team. So then you need to understand what is my agency in relation to others, and therefore you need to encode the feedback loops.
Speaker 2:This is cybernetics. So basically you need two core ingredients Puma and the feedback loops, and within that space then you need to organize that accordingly, because then now you can add agency of the team and now you can start thinking about agency of agentic workflows. If you don't organize this shit that I talked about, you end up with the very scary prospect of having someone putting up an agentic workflow with AIs that has no master, that has no agency. The agency of the teams versus the agency of the AI is not in a line, so it's very, really hard to understand how the decision is there's some kind of coordination.
Speaker 5:Is that another agent that coordinates?
Speaker 2:I.
Speaker 5:I would argue that the agency alignment between ais and teams are essential and, as you're saying, we can, we can skip the ai is because this is just like uh, an agent can be a human, it can be. It can be an.
Speaker 2:AI. But if you flip it, if you haven't got agency right in the first place, what happens when you try to take that artificial scary shit?
Speaker 5:If I understand you correctly, what you're saying is we need to coordinate somehow.
Speaker 2:Yeah, we need to find the ways to define the agency of different teams in relation to each other.
Speaker 5:Do you agree?
Speaker 2:This was a deep rabbit hole.
Speaker 5:Do you agree with this? Does it make sense?
Speaker 1:It actually makes a whole lot of sense and I think it's an interesting way to sort of consider it. What is the unit, what are we producing? Then we can think about the impact of this right In that context, and what does it fundamentally change, if anything? Like what are the important ingredients? And it's like everything else, like people, when you know, this latest AI wave came about, because it's not the first one and we freaked out about it before. It's like, how do people react to it? Well, there's all these doomsday scenarios. We were too far ahead imagining things and then we worry about it. It's like what is important to think about when that happens? It's like let's go back to basic principles. What has worked lately, what do we understand of our world today? And then let's look at what it changes and what's the impact.
Speaker 2:But did you follow the first principle? Fundamental change I did, compared to what you're doing today, is that, in order for this to work, when we have defined work in relation to roles, we have made the role, the craft, the minimum atomic unit of a company correct, and you are needing to now to measure the atomic unit of craft. Yes, when I talk about agentic separation of concerns, I never go beyond the team as the minimum atomic unit and I treat the team as the cross disciplinary organism, like we have in our bodies, and I can stay here and talk about productivity on this level. This is very different. This is very different. So then, because today you were trying to measure team level and you couldn't? Yeah, because you haven't defined an agentic separation of concerns with the product outcome of that team is the core contribution isn't it like going up in the level?
Speaker 2:of abstraction? Of course it is. This is what's happening. That's what's happening.
Speaker 5:And I agree that it should be something that optimizes for the output.
Speaker 2:But the core, fundamental topic, then, is that we are ingrained, or indoctrinated, in a functional division of labor, driving crafts, and what we are talking about now is an gigantic view of a cross-functional team that needs to have purpose, mastery and autonomy, cross-disciplinary.
Speaker 1:That is different. But think about it at the individual level. Like I, as an individual certainly have access to knowledge and capabilities outside my core expertise.
Speaker 4:This is magical right.
Speaker 1:This is me producing code to analyze my own sort of little pilot thing. It's like I just got access to all this. We're breaking down specialization. I think this is what's happening. We're breaking down silos and we're augmenting people. Now what does that do on a team level?
Speaker 2:Yeah, exactly.
Speaker 4:Can we break those silos Allow?
Speaker 1:teams to reach beyond sort of their space to deliver value.
Speaker 2:Anyway, I think we're getting ourselves over time and we are, in a way, talking philosophical or patterns that sort of is what are the relevant patterns for the future for anyone who followed? But I want to end up with the final question that we always ask, which is going all the way into AGI. So, first of all, I need to ask, you know, are we all in the belief that we are seeing? Will AGI come? Is it a matter of time or is it? You know, how do we feel about that? That's the first question. Is AGI something? Is it a matter of time or is it? You know, how do we feel about that? That's the first question. Is AGI something that is inevitable?
Speaker 1:Well, it's a matter of definition.
Speaker 2:Sorry for the boring answer.
Speaker 3:No, no, no. That's the leading question. It had to come from someone I guess.
Speaker 1:But let me say this and this is also about enablement and about getting buy-in, getting people interested, sort of. It's something I I repeat constantly as a mantra when people express concerns and worries about it, because if you go too far, people are worried. It's like, look, we don't clearly know, nobody can predict that future very well right now. But what is essential is that we understand what's going on now and therefore we have to engage. And this is super important, right, we need to engage to understand. We need to engage to be part of a conversation. Yes, same kind of conversation we're having now, but the conversation on a broader scale Like is this going to have a societal impact? Maybe, but we don't know until we talk about it and we try it out and we figure it out together. How do we get a?
Speaker 2:point of view, if we don't experiment with exactly so we absolutely need to experiment with it to find out and we talked about the definitions around okay, with the knowledge dimension, and then from knowledge, maybe, to be able to synthesize and reason, and then from synthesizing reason, we need to be able to contextualize and take part in the world. And Anders is the guy always doing this definition, where it's like equivalent to most coworkers actually doing their job on a daily basis, which is not only knowledge. Then it's actually taking part, actions, doing their job right. So then you have what you have, you have your world model, you have reasoning, you have knowledge and you have action and control as fundamentals, and then you can talk about that in the digital space and then you can move into the physical space and then you need the robotic stuff and then the last level of that dimension is like well, now we can do it in the lab.
Speaker 2:When is this universal? So AGI, universal is literally that this is all around us, and the typical conversation is like we kind of think it's doable. Now it's a matter of will it be 10 years, 50 years, a hundred years, but we see the trajectory. I don't know. I I believe in the trajectory. I can't put in on it but I believe in the trajectory.
Speaker 5:What do you?
Speaker 2:think.
Speaker 5:Yeah, and I think what's interesting is if we think that it's reachable, if we take your, what did we call them? Uh, the different crafts we use today. When, when, the agi can, can if we get to the level that an ai can perform all those crafts, what happens then? Does our models break down then? Because then we can just replace us all with an ai and then everything can just go spinning without us. Is that? Inevitable when we get to AGI.
Speaker 1:I guess it's the P of the Puma model. Right, it's purpose, and is it still? What's the purpose of?
Speaker 4:this. Why do we do it? Why would we have that and what does it do?
Speaker 1:I mean, what's the end goal of this? So that is technically feasible and that we're going there, maybe, probably, as we're're moving in direction where models are very capable, they can generate code, they can do things we thought were non-human. But this, this is the history of humanity, right?
Speaker 2:it started long before ai and it will continue, right um so, but but one way to answer your question or speculation, is that we coined at Deradux the AI law. Automation inherently, logically, augments workflows of humans, workflows of humans. So if you think about it, so when we are talking about automation, it meant we went up one abstraction level and even if you take away a whole role, there is some human prompting some problem and then some human receiving the output. And if you don't agree with that theory I have, then I believe we are an ASI. You know, if AI and automation inherently augments workflows of humans on a higher abstraction level, then my head speaks S is a simulator, artificial super intelligence.
Speaker 2:So for me, agi means you're still augmenting human workflows. So it means a little bit like what is our purpose, what is our job? Like the one man unicorn.
Speaker 5:Thanks for the definition. Go back in history, I think there's something called Casparo's second law. Yeah. Which is stating that the human plus ai will always beat the ai and the human. The coordination of the two, yeah, but I'm not sure that that holds anymore I don't know, I'm not sure that's coined during the uh the chess, uh, what was called the deep blue, deep blue, exactly. And that was the truth at that point.
Speaker 2:But I think now but you agree with my law that automation inherently augments workflows for humans, or are we thinking beyond that path?
Speaker 5:I think automation in itself is just doing things automatically. It's just doing a workflow. The workflow has to. I mean, if the workflow isn't right, then I mean it doesn't work anyway.
Speaker 2:No, that's true.
Speaker 5:So I think it's not necessarily so that we get to ASI with just automation.
Speaker 2:No, but my point with my thinking is that there is never really. We talk about human in the loop or human out of the loop. Well, it's a matter of perspective on abstraction level. How can you ever have human out of the loop, is my argument. You ever have human out of the loop, is my argument. Yeah, if I, if I go on the on, you know, at some point we do a process human out of the loop but then it there's another process on a higher abstraction. This is this is my argument. What I'm trying to say, that what I'm saying is that, as we are getting more and more from low, low level rPA or very simple workflows, to more knowledge intensive, but isn't it always a human in the loop on a higher abstraction level or not?
Speaker 5:I don't think so. Why should it be? Not by necessity, Of course. If we want it to be, there will be.
Speaker 1:But at least for the purpose.
Speaker 5:Yeah, for the purpose. That's what I mean.
Speaker 4:That's where it comes down. I mean, what are we building?
Speaker 1:here. Are we building something for humans or are we building something for, you know, for its own sake? So even a one-man unicorn.
Speaker 2:If you, if you play with the game, a one-man unicorn is the human in the loop, right, because the human is starting the prompt and and setting it all up and then it goes back to that human. If you take the human away from the one-man unicorn, what do you have?
Speaker 5:it's um. It's a question of if we want it to be that way yeah, yeah, that's, that's that's how I feel if we if we don't want to be in the loop, that we don't need to be in the back to that conversation.
Speaker 1:Right, this has to be a conversation and we need to keep discussing it right and how far we come where we're headed. We get a better picture as we go and therefore we need to engage and better understand what's going on.
Speaker 5:I think we all agree on that. We want to be in the loop.
Speaker 1:Yeah, we humans.
Speaker 4:And I also think that we are super users. All of us live and breathe this, so I think it's super important that we also make it possible for everybody else to understand this as well.
Speaker 2:You do that by enabling them and having the conversation on their level and you can't have the conversation if they don't have any reference points, If they haven't tried it. You need to start somewhere.
Speaker 5:That's why you're tying it all back together now. That's why the enabling is so important so that we can have all these discussions before it hits us.
Speaker 2:So final question then if this was a spectrum of we think AGI will come, we can decide how much we are in the loop or whatever. But if I paint a picture of a spectrum from dystopia you know the terminated story and you know this went to shit versus utopia, where you know we live in a world of abundance and everything is happy we saw the climate crisis and all that how do you reflect on the future and you know where on this spectrum would you? Are you an optimist or a pessimist? Or how do you define? How do you look at yourself on this spectrum?
Speaker 4:I know what you're going to say my wife called me Mr Positive, so I was born positive, so I'm positive.
Speaker 2:Yeah, I know about you, patrick.
Speaker 1:I am absolutely positive about this. I mean, I don't really trust us humans with everything that we do lately, but I still think this is a positive and we're going in a positive direction. I mean, we'll course correct, we'll figure things out, we'll get it wrong as well but we'll also get it right and in the end I think it's for the better. Technology has proven that over and again, so this too, I believe that.
Speaker 2:Have you changed your mind or have you?
Speaker 5:updated your view on this since last time we asked you. I think I update my mind every other time, so not this time Not this time every other time Bitcoin. No, I think I'm definitely positive. I can't change my thinking around positivity. I think all change is interesting and therefore it's good. Of course, if we come to existential questions, then it becomes a different game and I think at that level I stop working. When we get into super intelligence, who knows what will happen? It's so abstract. When we get into super intelligence who knows what will happen.
Speaker 5:I definitely see positives in having maybe some sort of considering how the world looks today with tariffs and all these things that are happening. I definitely see the point of having some sort of AI bridging across the divides and are probably better to negotiate than our leaders are.
Speaker 2:On that level. But this goes circles back to are we really worried about the agent or the AI spinning out of control, or is it the humans in control? That it up exactly so we are more worried about the power and the misuse from humans into this technology than the actual moment of agi yeah, but that's what.
Speaker 4:That's why you need to uh make sure as men as possible know how to use it exactly Exactly.
Speaker 2:So the way don't put your head in the sand and let a few guys or girls actually guys, bros run with it.
Speaker 1:That's the worst scenario that we cannot accept. That we cannot accept.
Speaker 2:Let's end on that note. Let's take charge. Thanks for this guys.
Speaker 3:Thank you so much. Thank you for having us Super fun yeah.
Speaker 2:Take care. Take care.
