Beyond the Licence: Lessons from a Global Microsoft E7 Transformation and AI adoption

Ally Ward and Kevin Spence discuss the practical realities of moving 2,500 of 8,000 users from Microsoft E5 to E7, covering migration strategy, agent governance, consumption costs, Entra Suite, and lessons learned from a global legal practice.

• Why E7 is far more than a simple licence swap and the operational complexity behind it

• The hotel room analogy: how unassigned E7 licences impact your E5 capacity

• The double-hop migration process and why the order of operations matters to avoid service desk chaos

• Agent 365's promise versus reality: regional reporting gaps, licensing for agent consumers, and governance challenges

• Entra Suite as a sleeper benefit bundled into E7 and the partial-coverage problem

• The Cowork consumption credit conundrum: 175,000 credits burned in under a month by 20 users, how do you measure value?

• Why agent governance is not an IT problem and needs business-wide engagement

This episode is sponsored by Luware Recording - a cloud compliance recording and analytics solution that helps organisations securely capture, archive, and analyse communications across multiple platforms to meet regulatory requirements.

Transcript

Copy

Ally Ward: I'm seeing some really interesting usage. My most senior people are using more credits than anyone else, and what I really do need to do is to spend some time understanding are they getting value out of those? 'Cause I'm tracking every single prompt I put in the Cowork. I'm tracking what was the prompt, what was I trying to do, how much did it cost in terms of credits, 'cause you can do slash costs after all of it, and was it worth it?

Tom Arbuthnot: Hi, welcome back to the show. This week we are talking about Microsoft's new E7 license. This is the first customer conversation we've had around the enterprise customer going to E7. We talked to Ally and Kevin at Norton Rose Fulbright around their journey. They're going down the E7 road, what it means for them in terms of Copilot, in terms of the Agent 365 conversation, and also some of the Entra Suite capabilities that they're excited about. Really great conversation with Ally and Kevin. Many thanks for them for jumping on the podcast and sharing so much. And also many thanks to Luware, who are the sponsor of this podcast. We really appreciate all their support of the community. On with the show. Hey, everybody. Welcome back to the show. Really exciting topic. This is the first podcast we've had, of someone going down the E7 journey. E7 is Microsoft's new top-tier license with Agent 365 and Entra Suite and a whole bunch of other bits and pieces in it, and it's gonna be a big push from Microsoft as they see people adopting lots of agents and, kind of really expanding what they're doing with AI. That's the major push for E7. And, excited to have Ally and Kevin on who are going through that journey right now. Ally, maybe you want to do an intro first. I think quite a few from the community probably know you now. You've done some stuff with, events and things with us, but, for those who don't know you.

Ally Ward: Yeah, so Ally Ward. I'm the... Essentially, I'm the Microsoft lead here at Norton Rose Fulbright, big law firm, and we've been running Copilot for three years now, since the original trials, and we've, we're starting on our Agent journey. We know that we need to put some governance around it. We're building our models and how we're gonna manage all of that. So it was a logical move for my leadership to say, "Okay, well, let's adopt the E7 licenses so that we can have Agent 365, Entra ID, and Copilot with a not very different price on it." So it, it was, it was kind of a no-brainer from the leadership. So now Kevin and I are working on how does that actually work.

Tom Arbuthnot: Yeah, how you get from, just, just, just buying the licenses to actually, activating and using all the features.

Ally Ward: Yeah. Yeah, and it's s- it's not... You know, it sounds like a- an easy license change. In practice, identity reporting, regional controls, Agent governance, the way we manage adoption, that all fits into, "We've got E7, what are we gonna do with it?" So it- it's actually quite a lot to do to adopt it

Tom Arbuthnot: Absolutely. We're gonna get into that. And we've got Kevin as well. Kevin, do you wanna introduce yourself?

Kevin Spence: Yeah. My name's Kevin Spence. I work with Ally. Effectively, I'm her technical lead across the whole of the M365 platform.

Tom Arbuthnot: Awesome. So you're gonna be responsible for the execution of some of these license changes and coordinating, and we'll talk about what a non-trivial thing it is to get from E5 to E7 for some users, but not all users.

Kevin Spence: Yeah. Yeah. It's... I suppose it's the, the actual pulling levers bit that I'm responsible for

Tom Arbuthnot: Nice. So Ally, maybe we could start with, you've gone E7, but for, for a subset of users, you've been early on the Copilot journey as well, so you've had a lot of Copilot users. Maybe you can just take us back to what you've done with Copilot so far, h- how many E7 users you're gonna have, and what the thinking is there?

Ally Ward: Okay, so yeah, previously all of our users have got E5, and then there's... And that's, l- let's call it 8,000 users for, for a round number. And then we've got, about two and a half thousand Copilot users. So every one of those Copilot users is going to get an E7 license. So we had the flexibility with that, and we had, multiple groups for E5 actually ha- hang over back to, to days of turning on Teams and turning on Power BI and things. We actually had multiple groups for managing our E5. So these, these people can use Teams, these people can use the Power Apps, these people can use various other bits of them. So hang over to that multiple groups managing E5, and then separately multiple groups for managing Copilot. And bear in mind, this is all multiplied by multiple regions that we've got, Copilot that can record a transcription, but Copilot that can only use transcription. So there are nuances of that as well. So what we're now gonna have to do is to crash that together to say, okay, what groups are going to be able to use E7? Are we going to maintain that same complexity of what you can and can't do with it? Or actually are we gonna say, "Do you know what? We've, we've gone a long way now, let's just give everyone all of the E5 stuff, and if they've got E7, then they can use Copilot as well." And I suspect that's the way most of the regions will go. But, we'll see. It- it's, you know, it, it sounds like it, you know, just simplify it. Just take it from two licenses, make it one. And that's not, it's really not that simple because there's a operational complexity behind it for automated provisioning of users. Just not that simple.

Tom Arbuthnot: Yeah. And you've got users that for either for adoption and change reasons or for compliance reasons don't have a certain feature or a certain policy. So there's two separate things there. There's, like, the feature in the E5 or the E7 is on or off, and then the feature is on, but you say, like, they've got transcription but not recording, or they've got X feature but Y, or more recently we've seen the, with the models, like, they're allowed... This region is allowed these models potentially but not these models. There's a lot of variables there to consider for the users.

Ally Ward: Oh, I think the, the pragmatic way of approaching this is going to be separate licensing from functionality. So you might end up with, you know, d- different groups, so only one of the groups has got a license attached to it, and then you can still manage the functionality if you want to separately. But it's, it's changing those automated models. We've got automated scripts, automated processes that manage it, so there's, there's a back end having to unpick what we've done one way and decide, well, do we continue that way or do actually we sort of split it and do something else or actually to simplify it? And that's, you know, those decisions have to be made within the window of the grace period 'cause we, 1st of July, we've got these new licenses, we've gotten 90 days, we think, that we can go from starting E7 to the point at which we're going to lose the old licenses. So got to resolve it in those three months, which in three months is a long time. In a complex firm, actually, it's not a long time. When there's a lot else going on, it's, it's quite a substantial additional work that operational teams may not have been aware of.

Tom Arbuthnot: Yeah. And it's fair- That's fairly typical of Microsoft, that 90 days that you have a grace period. It's gotta be like you're stepping up. You've got a bit of overlap time while you sort out your house, but after that, we're gonna charge you because you're using those licenses beyond the, the agreed term, unless you've sorted all your licenses out.

Ally Ward: Yeah. Basically.

Tom Arbuthnot: And talk us through, like, like you were on the prep for this, you talked about a kind of hotel room analogy, which is really interesting. I hadn't considered with Microsoft you tend to commit to a certain number of licenses, so now you've got a certain number of E5 and a certain number of E7, but that might not align to what the way the business was working yesterday necessarily.

Ally Ward: Yeah. Okay. So, a way of thinking this, so imagine you've got 1,000 standard hotel rooms, that's your E5 licenses, and you've got 200 upgrade vouchers. That's your Copilot licenses. Under the old model, everyone had a standard room, and some people got an upgrade pass. If they didn't use the upgrade pass, it didn't affect the number of standard rooms. So with, at E7, imagine that actually now you're gonna convert those standard rooms into executive suites. So after the grace period, those rooms are suites whether anyone's staying there or not. So after the conversion, you've effectively got 800 standard rooms and 200 executive suites. Okay. If you only assign 100 people to those executive suites, the remaining 100 suites sit empty, which is the old licenses. "Oh, that's fine. We'll keep this fair." But with licensing, that's not the way it works. Those are... They've already consumed those 100 rooms, so now you've only got 800 standard rooms. You've got 100 used executive suites and 100 unused, so- It's a problem, though, that E7 license isn't going to be sitting on a shelf waiting to be used later. It's already consumed that E5 license underneath it. But I think it's possibly quite a help- helpful way of thinking about the, the move on it is.

Tom Arbuthnot: Yeah, it's really interesting 'cause it's like, like it changes the dynamics of how you apply these licenses. And Kevin, you were talking through the practicalities of that 90 days and shifting the licenses, and you've also got You had a slightly different E5 than the new E5, so you've got multiple hops. Maybe you can take us through that, Kevin, and kind of the practical challenges of aligning those licenses.

Kevin Spence: Yeah. It was the, I suppose, the whole... We also went to a new E5 SKU as well as part and parcel of this, which further complicated the whole process. So, what we've advised is, is that you, effectively, you move all of your Copilot users to E7, make sure that they've taken, and then you remove the Copilot license, and then remove them from the old E5 SKU. And it's, it's really imperative that you do that because a lot of people, when we were first having the discussion, it was, "Well, fine, we'll unassign the E5s, we'll unassign Copilot, and then we'll assign E7." And technically speaking, yes, but you're gonna end up probably with an overlap at unlicensing the user, so suddenly you're gonna get loads of calls at the service desk, "I can't access Teams, I can't access Word." So it's really imperative that you assign the license, wait for it to take, and again, you know, the, the normal rule of thumb is give it a good 24 hours to make sure that everything's all in line, and then you remove the license thereafter. So and then once we've kind of... The advice we- we've given out is once you've sorted out your Copilot move to E7- And you've removed from the old E5, effectively you know what you've got on your E5s, and you can then just move the existing groups, if you so desire, to the new SKU, or you can revamp and do completely new group straight over. But even that, again, it's had challenges within it, because we've had spare Copilot licenses, and suddenly it's like, okay, fine, you know, do we rush the training through? Do we allocate the licenses? And as Ally alluded to, you know, one of the regions has gone, "Right, fine, you can have an E7 without Copilot." And it's, it's continuing that complication of the whole part, really.

Tom Arbuthnot: Yeah, it's a lot of coordination, again, in that 90-day window to make sure that it all happens and is complete before you get into the, the period where Microsoft want to take those old licenses away.

Kevin Spence: Yeah. And, and again, just to add to that, it's not even just the fact of, oh, we're manually moving users from one to the other. We're having to update all the back-end processes and, you know, the service desk documentation. If a new starter comes, which groups do they get? You know, have they completed the required training? Have they signed off on the, required forms? So again, it's that, it has almost been a, a complete kind of rethink of that JML process on the back end as well.

Tom Arbuthnot: Yeah, yeah. Considerable work there. And as Ally said at the top, it sounds like you could just, well, just jump them from one to the other and carry on. When you're a, got multiple regions and multiple policies and back-end scripting and third-party interrupt systems that manages licensing and just those different policies again, like revisiting does, do we need this level of variance of these people have this and this, but not this, these people have this and this, but not that. And that's not a decision IT, I assume, get to make on their own. That is obviously a regional decision.

Ally Ward: Yes. So what that feeds into is it's not just a technical what we need to do, that there's an operational approach to it. So what we used to do is we'd buy chunks of licenses, and then we'd decide who we wanted to give them to, and that there is, as Kevin alluded to, there's a process that says, A, you've got to accept con- terms and conditions under which you will do this, what you won't put into it, you know, h- how you're gonna use it. We've got some terms and conditions. Then you've gotta be trained in it. You've gotta be shown how to use it, and only when you've completed those things would you be given the license. And that's where we've actually bought large amounts of them. We've... It's, it's, it's taken time, months sometimes, to get through the large numbers of people. So we used to hold Copilot licenses in reserve. They were sitting on top of the E5. It wasn't a problem. Now with E7, we need to think differently about it. It's not just an extra pass that sits on the shelf. It's an upgraded E5. So if we buy the E7 before we're ready, we're going to lose the underlying E5 capacity after the grace period without it being assigned to someone. We can't afford to do that because we don't keep a, a massive bundle of E5s that are unused. They- Yeah... We, we've got a, we've got a measure of them, but it's not an enormous number 'cause it's, you know, it's unused money if you do that, so

Tom Arbuthnot: Yeah...

Ally Ward: Yeah, so now we're gonna have to move forwards to a just-in-time purchasing so that the regions then request the additional E7 when they've got a known group of users ready to move rather than buying a large bundle, which means multiple approval processes. Yeah, it's... You get everyone that needs to to sign off on this. You, you'd think it'd be easy, but it's not. There's lots of different groups of people. So the, the smaller groups you do it in, the more you've got that overhead of work to get to the point of, of doing it so that there needs to be a sweet spot, or you don't buy anymore until you've got a really large group, know exactly who they are, and perhaps you pre-train them and say, "I'm gonna show you. I'm gonna train you in it." But what's the point in training a Copilot if they can't use it themselves?

Tom Arbuthnot: Yeah, I feel like nobody, nobody's gonna engage until they have the tool accessible to them 'cause they're like, "Yeah, I'm getting it in three months. I'll worry about it in three months," so.

Ally Ward: Exactly. So the, the operational, you know, not, not just the technical, but from the, the people that are organizing the buying of them, they're, they're not the technical teams as a whole. They, they tend to be the business that says, "I want... I've got this many users. Go get them for me." And then, you know, so someone in the regions allocates them, so it's- It's a lot more work So

Tom Arbuthnot: I, I wanna get into Agent 365, but first, Kevin, you kinda highlighted this on the prep. The, the Entra suite is a bit of a, a sleeper benefit to IT operations, but again, it's kind of, asterisks, when only a subset of users have got it, can, can Norton Rose full, make full use of it? 'Cause some, some are haves and some are have-nots. Can you talk us through why, why you're excited about the Entra suite component of the, E7?

Kevin Spence: Again, it is, I suppose, it's one of those of, it, I suppose, it's not just the Entra suite. The change, the Intune suite was also bundled into E3, E5. So there's that added functionality that came. Then you've got the, the additional security layers, and we've, we've been t- discussing having a IG, identity governance tool. And effectively, you know, we're getting that on E7, but it's not every user. So you're then like, "Well, actually, we've got the tool, but we can't fully utilize it, so only all- around a select few users." So yeah, that adds to the, the complexity of well, we've got these features, but we can't fully utilize the whole suite across the board because not everybody's going to E7, 'cause we're not giving Copilot to everybody. But yeah, it's, it's one of those, maybe it's justification that we could then look at if we can utilize it, find the, the, you know, the maximum benefit, especially as an IG, that we'd look at maybe just purchasing the add-on of the Entra suite on top of the E5s. So it, yeah, it's, it is something to look at definitely in the future. But yeah, it, it was very strange on the way that suddenly it's like, well, we'll just bundle it in on the E7. I was very much like, "Okay. I want it."

Tom Arbuthnot: Yeah. O- opens up another, another conversation, doesn't it? About, oh, well, we've got, we've invested in a subset, so actually, how do we... Can we, can we leverage that incremental investment, or does it not actually make sense until we hit a tipping point of 100% coverage? I, I guess 100% by a region or 100% globally potentially, either could be true.

Kevin Spence: Yeah.

Ally Ward: Or, or some of those things, Kevin, you, you talked about local admin.

Kevin Spence: Yeah. That's actually- And the other one within the Intune suite. So Yeah, it's the, the majority of the, bundled Entra Suite is all running around governance, so it's the, you know, the, oh, I'm trying to think. It's the VPN, it's the web filtering, the i- i- identity governance, the extended identity, so you can do photo, ID. So you've got picture ID as well as just doing your normal authenticator. So it's extend- extending out the additional com- capabilities of the Zero Trust and that two-factor authentication, so that you can be certain that someone is who they say they are, not just because they, you know, they're scanning their fingerprint, but they're also having to hold their phone up and get that visualization of the face.

Tom Arbuthnot: Interesting. Yeah, it's a whole, whole another conversation on just the Entra Suite in its own right.

Kevin Spence: Yes.

Tom Arbuthnot: Yeah,

Kevin Spence: Completely.

Tom Arbuthnot: I wanna g- I wanna get into Agent 365, 'cause that's the headline of E7, is y- y- you, you know, organizations are going frontier. They're going to be using hundreds, thousands, millions of agents depending on who you believe, Microsoft. Like, Ally, you're very close to this in a very practical way with your Copilot adoption. Like, where are you, where's your thinking and your organization with agents today, and then what does Agent 365 bring to the party, and what's it lacking that you think you need? 'Cause, 'cause it's an early, it's early in the product lifecycle for Agent 365, so I'd love your opinion on that.

Ally Ward: Yeah, so I mean, Microsoft is pitching Agent 365 as a- you know, dealing with agents as digital employees, a single pane of glass for all agents But they're not necessarily all visible. I don't think you see the ones that come from, Agent Builder unless they've... Actually, then do see the Agent Builder ones. But third party agents, only the vendors. I th- we were talking about if they embed it in-

Tom Arbuthnot: Yeah. They, they haven't, they haven't... Like, it sounded like magic when the marketing first came out, which is like, "You build an agent on any platform anywhere, anyhow, and it'll automatically appear in Agent 365." And there were lots of fair technical questions about how. It, it now turns out that there's an SDK for Agent 365. So if someone like Adobe or, Salesforce has an agent, they can embed the SDK, and that, that, that then automatically connects to your organization except the agent. So that depends on the third parties. And then as you say, on the first party agent, I don't have it off the top of my head either, but there are some scenarios where it just breaks in and works, and some where it doesn't, like Scout, which is very new, doesn't connect to Agent 365 yet as an example. I'm sure it will over the fullness of time. But there are first party scenarios in Microsoft Suite that don't automatically connect at the moment.

Ally Ward: Yeah. And then I've been just trying to look at the reporting on it. My, my challenge is that I need to be able to, a- any time I report on any usage of users, of agents, of, of apps or whatever else, I need to be able to say what region does it belong to. So Microsoft's pane of glass, it, it only does it at a whole firm level. I can't divvy it by region. So I h- will have to pull out all that raw data, and granted I've already got reporting for Copilot, but I'm gonna have to pull it out and enhance my Copilot reporting with what Microsoft have given me. They're, you know, they're, they're pitching it as a, a single pane of glass. You can see all your agents. You can manage them here. The reality is I can't. For start, there's too many of them. You know, even now there's too many of them to manage them in their pane of glass. I'm gonna have to create my own reporting at a regional level that says these ones need addressing, these ones haven't got owners, these ones haven't got whatever. It's all very well to have it in, i- in the one, you know, the, the one admin console, but that's not available to most people. I can't give that to end users. I can't give that to the regions that ought to be managing those agents. I'm gonna have to give them something else. I'm gonna have to build that. That's one thing, and then the other one is that in terms of licensing, Microsoft are saying you need this Agent 365 for any- agent or owner of the agent that's deriving the benefit of it. So i- it's unclear. You know, w- we've licensed E7, therefore a- Agent 365 just for our Copilot users. So they're gonna build agents. Those agents, when they get into production, won't be owned by those users anymore, they'll be owned by a service account. They'll be used by whoever. At that point, should they have an Agent 365 license? Should we have them for the whole firm, or is it just because that was built by someone else? It's really unclear how that license is gonna work. Are we going to get to a point in a year's time where Microsoft says, "You can't use an agent," or you, that agent's not gonna show up because it's going to be used by people that aren't licensed with it? It, it's really unclear what's going to happen going forwards. Are we going to be forced that everyone has to have Agent 365 if they're gonna consume agents within the estate? And I, I, I don't, I don't understand.

Tom Arbuthnot: No, I, I think it is unclear. I've spoken to quite a few people at Microsoft about it, and my, my takeaway at the moment is it's a bit like some of the Purview or advanced security features. It's like if the end user is deriving a benefit, then they should be licensed, but it's not enforced. So, like, if, if corporate IT are using Agent 365 as a tool to monitor that agent, and that agent is being used by Sarah and Jack and John, then Sarah and Jack and John should have license cover for the Agent 365 even though they're not there, you're deriving the benefit as an admin team, is my understanding. But t- to your point, there's kind of a, a weird line there where early on they were talking about agents having their own identities, and that's the big thing Satya keeps talking about, is they're gonna have Entra accounts. They're gonna be, like, virtual employees or virtual users. So do we end up licensing... I mean, they're not g- Microsoft not gonna do that for free, so presumably there's gonna be some license aligned to an identity for an agent. Does that cover them for Agent 365, or is it the user that needs to be covered?

Ally Ward: Yeah, no, I, I don't know, don't know how we... Well, I, I guess it will come out over time. My, my worry is that like other things, over time the controls around them will then be imposed, and then we'll be in a position of we want to carry on doing something the way we've been doing it, and we can't without then buying additional licenses. That, that's my concern. Now, whether that's a, a this year or a three years' time position on it, who, who knows? But it's, it would be good to understand where we think-

Tom Arbuthnot: Yeah...

Ally Ward: Go on that.

Tom Arbuthnot: What's your, kind of parking the licensing, what's your general thinking about like agents having a Entra identity and kind of embedding into the Microsoft ecosystem? Do you... Microsoft seem to be pushing this kind of virtual employee, virtual user kind of narrative now. Does that, does that align to your thinking or do you think of them more like apps? How do you think about them?

Ally Ward: So honestly, I'm not sure that we've gone that far yet. Our agents are still relatively immature. They're not productionized. We're a long way from an agent as a digital employee. I'm sure there are some firms that are getting to that point, and if it were literally behaving like an employee, I think I'd want it to have an identity like an employee, but then you've gotta think, well, are you licensing like an employee? Do they then need an E5, an E7, or, you know, do, do they need their own licenses in, though you've already paid for it as an agent? I d- I'm not clear how that's going to work. I mean, I think talking to Microsoft about this, what they were saying was it's aimed as much as the, the, the agents that are out in the wild that, that aren't necessarily agents you built, but agents that you're bring in from elsewhere. The ability to, to find them and report on them, give them identities Do I count? I don't know. Kevin, have you got-- Have you as to... You're, you're probably closer to Julia identity side and the technical on that than me. What, what do you think?

Kevin Spence: Yeah, it, it... It's funny though, it's one of the big ones of that whole governance around what's in, what's within the estate, what are they accessing, what permissions do they have, you know? And we've always traditionally wrapped that around an identity. So it makes perfect sense to do that because then you can turn around and say, "You know, this agent has this identity, it has permissions to these resources, it's licensed at this level." Now, it's the, the governing part of that, which then becomes very challenging, and also, you know, the, the ongoing monitoring and confirmation around all of it. So, and then you add in the, you know, do they need to be licensed? Not licensed, licensed. Again, I was reading some bits and pieces, and what we clarified be- at the beginning of last week was, you know, they were saying about actually giving these agents a workstation. So they would sign into an AVD workstation, and they would, you know, they would move the mouse, they'd open up Edge, they'd actually complete a task as if it was a user. And it's like, well, actually, that is a user. That's an identity. You know, that makes perfect sense at that level. So it's probably just the fact we're not there yet, where, you know, we're not getting that, you know, that clear definition and clarity.

Tom Arbuthnot: And, and if you park the licensing cost, which is a totally valid conversation, Microsoft have a very mature user model in terms of user accounts, RBAC, audit logs, control, Defender. So there's some, there's some logic to the, the, the kind of positioning of when you, when you employ a actual employee and you give them access to this thing on SharePoint but not this thing, and you give them access to this, this thing, this line of business tool but not this thing via their Entra identity, you're having the same level of control over what an agent can and can't do, and the same audit log of they try to access a thing on SharePoint and they can't 'cause they're not allowed. So I can see the, the positives of we have a mature model for managing people access. Can we leverage that same model for people and agent and not have a brand-new workflow and a brand-new tool?

Kevin Spence: Yeah, I mean, I, I see them more as more in line with service accounts That we previously had. So you've got all the benefits of it being a user, although it's not attached to a, you know, a person, but it's acting- Yeah as a service account, and it's just gonna be one of those, of, that we may well have to ad- apply additional licensing around it.

Tom Arbuthnot: That languaging, that language immediately sits better with me. I feel like it's more realistic to talk about it as a service account the agent uses as opposed to a, a virtual user. But I'm just thinking, I think the virtual users or virtual employee thing is a little bit, little bit too marketing for

Kevin Spence: Me. Yeah. Yeah. Yeah, again, it's, it's, mind you, I mean, even now, the whole, I suppose, language around service accounts has moved to an identity when it comes to Entra. So- Yeah... It's all just that one identity, and it's, I suppose, it's us old school trying to remember that actually, no, it's m- it's more of a service account than it is actually a physical user.

Tom Arbuthnot: And Ally, let's talk governance 'cause th- this is something we've talked about a lot and I think is really interesting, is, like, you and M365, this is a brand-new tooling, a brand-new problem, like a opportunity, depending which way you wanna frame it. Like, lots to do. Like, it, the organization is spending money on, you know, lots of money on Copilot and now Ag- Agent 365 and E7. How, how does it look in terms of managing this, in terms of getting the, the regional business leads engaged and deciding policies, all that kind of stuff?

Ally Ward: Yeah, so it's quite interesting. So we kind of started from the technical, oh, my goodness, we've got people doing stuff. Do we want them to be doing the stuff that they're doing? So we had started with a Copilot Studio. What do we need to govern around it? Do we let people send emails out of it? What connectors can you use, et cetera? And that's fine, although the decisions around that, they're, they're not IT's to make. And that, the, the challenge I'm having is that I'm IT. The, the people that are controlling it are IT. But what we actually need is, an approach as a firm on it. So yes, there, there's the, the technical platform and build controls. That, that's one part of it. But actually, what we need is a, is a strategy and oversight who's gonna govern it? Is this being governed centrally? Is it being governed by regions, or is it a combination of the both? Who gets to make the decisions as to whether or not we have this agent or that agent? If you've got, you know, you've got that agent, oh, is that not the same as that one? Do we need two of them? Do we enhance the other one? Who gets to manage all of that? That's not an IT decision unless it's been built by IT, and even then, it's been built on behalf of the business. And then there's the, the operating model on it. How are we going to make those decisions? What, you know, are there various boards that get to say what you do and do and don't do on it? The split of regional and global responsibilities. Other things on that is agents aren't just about the technical bits of it. It's what's it gonna cost? Are you using credits for it? Do you need to set up an account to do that? Or are you using pay-as-you-go money on it? And if so, who's allowed to consume those credits on it? So someone's got to, A, decide that finance on it, B, fund it, and C, manage that spend on it, and, you know, we run out of money, what do we do? Do we get some more on it? So that's the finance piece. There's a compliance piece. What's that agent going to be doing? What data is it accessing? Who's it going to be sharing it with? How often do they get to access it? How do you decide whether or not that data's out of date and then needs to be pulled back? Is it in compliance with whatever the regulations might be on it? It's all compliance piece on it. There's a There's a security piece on it, not just the, the compliance and what are you doing with the data, but who you're sharing the access with. Who c- you can do something, who can, who can change it, who can do something with the data behind it. What's happening to the data? Is it being stored somewhere else? Is it going into someone else's model? Decisions around that need to be thought about, as well as the very obvious, if people are building the, the agents, what are we going to let them do? What connectors can they use? Which again, fits into tho- those other areas on it. With all of that, there's gonna have to come policies of what you can and can't do, standards on how they're going to be done, processes behind them, and more than anything else, reporting on it. Dashboards, making it so that it's visible. What have we got? What agents have we got sitting around that no one owns or no one's using? Do they get archived after a period of time? How do you manage that? And we're, I'm in processes of building sort of a governance model on it, but I, you know, i- it's a big deal. And I'm IT, I, I can write this stuff, but I can't make it happen. I can't actually do the governance unless,

Kevin Spence: You know, unless that role-

Tom Arbuthnot: Yeah what I am- I think that's such, that's such an important point. Like I think, lots of organizations I'm talking to at the moment, they're talking about Agent 365 and E7, and it's like it can be easy to convolute, "I'm buying a solution to all of those problems." But actually, no, I'm buying a, another piece of software, but we need the engagement from the relevant business departments to complete that journey. So if, if you're going into this E7 Agent 365 story, I'd like to see more from Microsoft on this to be like, "Realistically, Mr. or Mrs. Large Organization, love that you're buying our tool. That's great, but you need to back it up with these roles, this time, this, this, this people layer to get the most out of the tool."

Ally Ward: Yeah. So we, we've had some of that engagement, and I do have a report from Microsoft that says, "These are your gaps," or whatever. They're, they're kind of the gaps that I've already identified, which is reassuring 'cause they're, they're not saying I've got gaps that I don't know about, which is great. Yeah. But there's a level of detail underneath that. It's all very well to say you need to do, but it's the practicalities of it. Wh- where's the, where's the templates? Where's the, the structure of this is what good looks like? Do, do Microsoft do it? C- can they show us what they've got, their structures? They've got thousands and thousands of agents. What, what have

Tom Arbuthnot: They- Yeah, yeah. They've got over half a million. I've seen the internal, dashboard, and it's- Yeah... Crazy. Now obviously they've been heavily encouraged to build agents and experiment, right? So it's a high bar, but they're like, "Yes, they have."

Ally Ward: Yeah

Tom Arbuthnot: And l- last thing I wanted to touch on is credits, 'cause you mentioned it there in the agents. 'Cause a lot of these agentic scenarios are credit based, and I know you've been experimenting with Cowork as well, and that is probably the hottest conversation in the space right now is Microsoft have just flipped Cowork to 100% consumption. Like e- even if you ignore the agent conversation for a second, like lots of people were and are excited about Cowork. It's a very, you know, powerful product. But how do you begin to, understand, allocate budget consumption spend on that?

Ally Ward: It's, it's really interesting. It's probably a whole new podcast all on its own. We, we started with a 30-day pilot just to say about... It's about 20 people, 20, 25 people that can use it, and whilst we haven't sort of said unlimited credits, we've sort of adjusted the credits as it, as it creeps up, and I'm seeing some really interesting usage. My most senior people are using more credits than anyone else, and what I really do need to do is to spend some time understanding are they getting value out of those? 'Cause I'm tracking every single prompt I put in the Cowork. I'm tracking what was the font- What was I trying to do? How much did it cost in terms of credits? 'Cause you can do slash cost after all of it. And was it worth it? And I'm being bru- you know, brutally honest of, is this worth the money that it's on? 'Cause I'm translating, you know, nought point... So, so one cent, one US cent is, is one, one credit. And, yeah, we've got sort of 175,000 credits in play, and we've used them all with those 20 people in not quite a month. Now, whether that will go on like that, I don't know. But the, the true point of this, and, and Kevin makes a really good point on this, and I'll let him make it for himself. But if, if you didn't baseline it, how do you know whether it was value or not? You were trying to do this task before. How much time did it take you? How much did it cost you? And I mean, I've... I literally track on every single prompt I put in, but I can't make users do that. I can't make people feel- No, it's not

Tom Arbuthnot: Pra- it's not at all a practical ask on a user. And should it even be a... Like, should it be a user's responsibility? I guess in this new world, and it's not just Microsoft, all the big I- AI houses are going to consumption for all of their tooling on some timeline. Because the cost model is variable for them, but it's a very new world to give a knowledge worker or business user a tool and say, "Be careful about how you use it." The only, the only thing I can equate it to, it's not a great analogy, is you're a manager of a team, and you think about how you allocate those people's times and their experience. That's... But, like, much bigger numbers, 'cause that's people's salaries, but that's the level of skill of like, do I do this in this tool or that tool, or do I schedule this daily or hourly or weekly?

Ally Ward: It is, but then you know how much you're paying for those people. The difference with Cowork is you don't know-

Tom Arbuthnot: No, you've got... You're right. You've got a fixed high bar, haven't you? Yeah.

Ally Ward: Nor can you ask it in advance how much it's going to cost on a... It might give you an idea, but till it goes and does it, it doesn't know. And, you know, you're gonna spend more time thinking, "Is this an expensive thing to do?" than, than asking it to do it. And for very senior leaders, their time is money for them. If they can do something in five minutes that would take them an hour to do, then it, it's probably well worth it regardless what it's costing But are they using it for absolutely everything? They could just be using Copilot Chat. And that was the conversation I was having with one of my very senior leaders yesterday, was, "What are you using it for? And I'd really like to talk you, go through your prompts-

Tom Arbuthnot: Yeah...

Ally Ward: Look at the costs on them, and say, was it value or not? Then I can provide some guidance on what to use it for and whatnot. We- we're still learning right now.

Tom Arbuthnot: I, I, I think, I think that's a product friction in Copilot. I think it's, it's tough for a user to be like, hit this tab and it's included, hit this tab and it isn't, and you have to mentally decide each time. Like, like, that is pure friction in the user experience to me. Yeah. Yeah.

Ally Ward: I, you know, and then, and it, it's going to end up being so much money that unless, unless we can learn to put some guardrails around it, and I don't see how we do, it, it, it's, it's really difficult. Will it just be for those senior leaders that they can say, "Well, I'm paying for it, so I'm gonna have

Tom Arbuthnot: It"? Yeah, or you've hit a tip, you've hit a salary tipping point where, like, their time is so valuable that even if they're burning some time on inefficient prompts, it's still net, net gonna be a win maybe. I

Ally Ward: Don't know. I, I don't know. I mean, some of the things it's done for me have been phenomenal, and they're saving days of work at a time. There's no question of it, and not just, you know, something I could have done myself, but there's no way I could have done the level of quality that it's producing. There, there, there's no doubt about it. But then it's done some other things in PowerPoint, I think, and why did I bother?

Tom Arbuthnot: Yeah, yeah. I, I, I want to like, a money-back balance to be like, "Wasn't worth $3."

Ally Ward: Absolutely.

Tom Arbuthnot: All right. Kevin, what were your thoughts on that?

Kevin Spence: Yeah. A- again, it's that of, I mean, I always work on the basis of if you haven't measured it before, you don't have that baseline, so you don't know the value of allowing to automate it. So, and, and, you know, coming from IT, we, you know, we've always done it. You know, JML processes, we automate because we've measured how long it takes somebody to do it manually, so we know what the cost savings are. When you're passing it out to your general users and they're utilizing it to clean up their mailbox, it's, you know, it's like, okay, fine, well, how much time would that have taken previously? You know, and again, because we've never measured it, you don't know what cost savings are. And again, I think even if you went to use them, like, "Well, actually, we're gonna measure how long you spend in your mailbox tidying it up," you're gonna get pushback. Because again, you're going to that level of monitoring. So it's a really, really difficult area. Very, very challenging. I find that I run with Copilot, and I put everything into Copilot till it fails, and then I move to Cowork.

Ally Ward: Me too.

Tom Arbuthnot: Yeah, yeah. Which makes logical sense as an IT person 'cause you're like, "I'm gonna thrash this included tool as much as I can and then escalate." I just don't see a, a typical business user having the time/motivation/technical understanding to make that choice, and the risk is you turn them off the tool completely, or you, they never get to experience the, the amazing power that is Cowork at its best 'cause they only use Copilot and they're like, "Oh, I wish I could've done X." Well, you can, but you have to mentally think through this incremental cost.

Kevin Spence: Yeah, and it's that additional training that then goes with it. So it's not just how do you go about utilizing AI with Copilot. It's then actually, you know, you do then have the additional functionality where you can go to Cowork and it's more of, it's a whole process. So- And I suppose what the biggest worry I have is, you know, all of the AI chats of... The people are using them like they would use Google- And if people then start going over to Cowork for that, those... All right, they're only small amounts, but they'll mount up very quickly.

Tom Arbuthnot: Yeah. Yeah. Well, this is a, a, a conversation to be continued, 'cause this is an industry-wide, or, I think enterprise-wide conversation at the moment. Let's say it's not just Microsoft. It doesn't matter what AI tooling you're going down the road of. I think everybody's acknowledging AI is part of our future, and, and all the big orgs are going, the big suppliers are going consumption. So, just like when dev went to cloud, and a lot of stuff became consumption, you know, our IT infrastructure started to become consumption in a lot of cases. We went from we have a server, we own a server, we can max it out, to this is, you know, consumption-based. It, it feels like it's going a similar way for a, a certain subset of the, the business line of process stuff as well.

Kevin Spence: Yeah, I think the, the difference being is, is that that was... I suppose that was an IT function. We measured, and then we implemented to prove out the cost savings. This is at the end user where you can't get in really to do those measurements, and a lot of it is, "Well, how did you... H- how do you feel that improved your work?" So we- Yeah... Again, you... It's very, very difficult to quantify.

Tom Arbuthnot: Yeah. Yeah. It's a, it is, it is, it's a fascinating one. Like, the, the cleaning the mailbox example is a good one, right? Like, let's say that costs a couple of dollars a day, but that person is, because they're more organized, they're functioning at a completely different level, and therefore they close the business or do the deal or do whatever the thing is they do much better. How do you measure that on Excel, which is what the CFO wants to see? It's like, well, they were 20% better. At, at, at how? Like.

Kevin Spence: Yeah. Or they didn't miss an important email or, you know, an urgent call- Yeah... Or whatever, and it's, it's, it's very- Yeah... Challenging around all of that.

Tom Arbuthnot: Awesome. Well, thanks so much for sharing so much. Really interesting to talk to you early on the E7 journey. If it's all right with you both, we'll definitely check in again, again when you're a little bit down the road. And, we've got some interesting events coming up around Agent 365 and Cowork and, and things like that. So we'll be... I think for, for this new Microsoft year, that's gonna be a, a hot topic to, dig into further. Awesome. Thanks, Ally. Thanks, Kevin. Appreciate the time. All right. Thank you very much.