Microsoft Mesh, Real-world Use cases and AI Futures

Jeremy Dalton, Head of Immersive Technologies at PWC and Cathy Moya, Senior Customer Success Program Manager - Mesh Team at Microsoft, dive into Microsoft Mesh adoption and use cases and how virtual reality and virtual worlds can enhance collaboration and communication.

• Mesh features include avatars and immersive spaces within Microsoft Teams meetings, enabling more engaging and interactive meeting experiences
• Virtual reality and virtual worlds, private conversations and a greater sense of togetherness when collaborating remotely
• Mesh use cases: training, cultural events, learning and development
• Future AI integration to create and enhance virtual worlds

Thanks to AudioCodes, this episode's sponsor, for your support of the Empowering.Cloud community.

Transcript

Copy Transcript Copied

0:05

Welcome back to the Teams Insider podcast. This is a double guest podcast. A really interesting one we get into Microsoft Mesh, we talk about what the use cases are, what's in Teams core, what needs Teams premium, and also what's being used by customers in the real world. Really enjoyed this conversation. Many thanks to our guests and many thanks to AudioCodes, the sponsor of this podcast. Really appreciate their support. Empowering Cloud on with the show. Hi everybody. Welcome back to the pod. I've been looking forward to this one. This has been a hot topic and something that I honestly need to get more more depth on. I pinged Jeremy on on LinkedIn because I thought he'd be able to give me some insight, and Jeremy's pulled in. Cathy. So, before we start, if Jeremy, if you could go a bit of an intro and then we'll hand to Cathy. Yeah, absolutely. So. Hi, everyone. My name is Jeremy Dalton, the head of immersive technologies at PWC. And, ultimately help companies and different industries to use virtual reality, augmented reality and virtual world technologies previously called Metaverse. But we prefer not to use the M-word these days. And, Yeah. So ultimately, I've. I've helped author books in this space, talk all over the world, work with local governments, startups and large companies. But everything I do is related to the business usage of virtual reality, augmented reality and virtual worlds. At the end of the day. Awesome. And Cathy. Yeah. So Cathy Moya, I am a senior customer experience program manager on the Mesh team. I've been at Microsoft 22 years and this is the best job I've ever had because I found virtual reality during Covid and went, oh my gosh, why is it everybody using this? And so now my job is to make sure that everybody is using this. So I work with our customers internal, external. I've been running some great events at Microsoft and trying to make more people run great events. Awesome. so yeah. So maybe I can start with you, Cathy. Maybe you can ground us in this. This audience, you know, knows probably Teams M365. Really? Well, I think like coming into Mesh like the avatar three world. Maybe you give us a bit of a grounding in the basics of what Mesh is, what the different options are. Okay. Yeah. And you know, when I saw the strategy, when I joined the team, it was already in place. And I thought, okay, this is brilliant because the first piece of it is just avatars like instead of being here on camera, I could be in my avatar form. It's less taxing in the morning. I have like VR face for my supernatural workout so I can just be on my avatar instead of all sweaty and and I can still interact. I can send emojis or, you know, different kinds of movements during the meeting so people know I'm there. I'm listening. And then you can take that same avatar directly into a Teams meeting just by going up to the view menu and saying immersive space. So the onboarding I mean, there isn't onboarding. You're already in Teams. It's two clicks. And then you go into the space where you can walk around in that avatar and you can roast marshmallows and you can throw beanbags. And we have an icebreaker activity. So it's all right where people already are meeting in Teams. Yeah. And that's just all included with your basic Teams kind of commercial license. You know, you may already have it. There and you don't necessarily need to have like this special gear and headsets, obviously you can do that, but you can do this with keyboard and mouse and kind of navigate yourself as well. Absolutely. Yeah. So just the regular PC that you're on, Mac coming soon, you know, you can just walk around and have all of that kind of free range conversation and interaction. But then if you want to get more special, we have a separate standalone app that you can install. It's in the store. And then once you install that, you can have all the custom worlds you can meet on Venus or, you know, under the sea or in a digital twin. We have some great digital twins. A couple of people have built here and and really just feel all that, that connectivity in, in totally customized environments. And you can customize in the app. So for someone like me who could barely spell Unity on a good day, I can just go in the app and I can do all my customization. If you want to go into Unity and do all the work there, sky's the limit. There's a ton of AI potential. I'm going to pause on that. I really want to ask that question, but I going to loop back around to that, Jeremy, maybe like you've obviously got a lot of customer experience was how we started off our conversation. What are you seeing in terms of adoption use cases? I think that that's really interesting to me is I think I've used Mesh a bit. It's cool, but I want to know what people are actually using it for in kind of, you know, different settings. Yeah. So that's Cathy mentioned Mesh is not simply one product. So it's not just virtual reality. And it's not just virtual worlds on desktop computers. So really I think it's important to distinguish between those two concepts. Firstly, because both virtual reality as a head mounted display, a headset you put on your face, as well as a virtual world in which you navigate to your avatar, your virtual representation in a digital world on your 2D computer screen. These are these are completely different products. And, they have different applications. But if we start with virtual reality, this is this is being used and has been used for many years now in all sorts of different industries. There is not a there is not a single industry out there that has not at some point used and, and in many cases continues to use virtual reality technology. One of the main applications that many will have heard about is in training. And within training. There are a lot of subsections. There. So soft skills training, anything to do with human skills, how you interact with each other. and then there's hard skills or practical skills wherever you have to use your hands to perform a certain action. Virtual reality tends to be very good at that, creating that, that sense of, memory muscle connection. and so those tend to be really good areas for virtual reality. Then there is remote collaboration. This is where we start to see a little bit of an intersection between VR and virtual world applications. So remote collaboration and the ability to connect together with others in different parts of the world, this is really powerful for the technology to take advantage of, because in VR, when it's done really well, you can make eye contact with other individuals, or at least with other with representations of other individuals. In these virtual worlds, you can turn together and write on a digital whiteboard in the in the corner of the room. You can split off and have separate conversations. Now in in virtual worlds where you're sitting on a laptop, perhaps, and you're navigating your avatar in these, in these 3D worlds. That application is also valid to so you can split off into different directions and have different conversations. And many ask or challenge, why would you do this? You know, why do you need to have a 3D world? And on a desktop computer where you could simply run a video conferencing call and, and Cathy's got her own view on this, I know, and I'll let her chat about it, but from my perspective, it's just far more powerful when you're able to give someone that sense of empowerment. It's far more liberating when you're able to, to allow someone that freedom to to go in the direction they want to go, to go to another side of the room and have a chat, have a private chat with someone similar to how they would do in the real world. So this is whether it's VR or virtual worlds. This is all about emulating real world behavior and making the the unfamiliar more familiar by, allowing similar mechanisms to what we have in the real world. And I'll just end by saying we also conducted a study with, the University of Munster in Germany and the Ministry of Internal Affairs in Lower Saxony in Germany. And this was all about comparing virtual reality. So when you've got a headset on, you're immersed in a completely different world to video conferencing for a deep remote collaboration. So agile planning meetings and, and events like that. And what we found is we found some real great quantitative data, supporting the value of virtual reality for working together in different parts of the world. And we found there was a 16% increase in communication. We found this a 58%, greater sense of of togetherness or closeness to colleagues, 11% increase in, satisfaction with workshop results and, and a 15%, greater comfort in a virtual reality meeting. And that comfort was not physical because it is certainly more uncomfortable to wear a headset than it is to use a computer screen. But that was to do with psychological comfort. So being represented by a digital avatar, not feeling that sense of being watched all the time and judged constantly. So some really interesting stuff there. We did also find some negative results too. So I think it's important to balance this out. But there was greater fatigue when using virtual reality. But given all the the massive benefits, we feel that the good outweighs the bad here. And as long as you use it in the right way, of course, because there's no point in us in having a deep, using virtual reality for, just a simple presentation purpose. It's useful for deep collaboration. as usual, it certainly seems I've played with some of the headset scenarios with with friends, and you're you're 100% engaged. You can't not be engaged. But I think with the remote working is really interesting because, I'm, I'm certainly guilty of it I'm sure we all are at times that you're multitasking constantly, right? You can't not look at Teams and Outlook while you're engaged. The nice thing about having the headset is you're you're 100% committed to that scenario. It's really interesting. Cathy, what's your your perspective on this? I'd love to hear you. you're working with some customers on it. What what were the use cases you've seen that have kind of popped for them? Well, so one that we've talked about publicly already is Takeda, so, you know, biopharmaceuticals, they, they're doing some really fantastic things in just some of the earliest stages. also another fascinating one is Mercy Ships is a charity. And so they worked with link to VR and Dell to create a digital twin of their flagship, which they take around to places that don't have medical care. And it becomes a hospital ship for those people. But you can imagine there are a lot of doctors, a lot of staff going in and out of these ships, and so they're able to just take people on to the digital twin and give them training. They can use the digital twin for fundraising. you can get in places like, you know, the bridge of the ship, which you wouldn't be able to go to in real life because it's a secured area. Yeah. You know, so just fascinating external and then internal. We've done Lunar New Year, for International Women's Day, we did a women's history museum inside of Microsoft. We had all of the different employee resource groups contribute content to it. we have Juneteenth coming up next week. So, you know, the cultural pieces are good. And then also for learning and development, we've been doing a lot with some of our internal onboarding Teams, like, m caps for sales, new employee orientation. That's another interesting area as well is is student recruitment. and we've been using student recruitment internally or we've been using virtual worlds for student recruitment internally for many years now. And it actually came out of the pandemic. I know you mentioned, Cathy, you got into virtual reality in the pandemic. We as a company got into virtual worlds in quite a major way during the pandemic. So we started using these types of environments to engage with with students all over the UK. And, it was just a fantastic way to, to firstly enable access and improve accessibility, because not every student is capable of being physically present on campus, where we are hosting a PWC tent for example. but, you know, we can we can reach anyone through, through an internet connection and a laptop system. So it was it was great from that sense. And it was also great from the perspective of psychological comfort. This benefit that I was speaking to to both of you about, and we got feedback directly from the students that they did feel really comfortable being in this virtual world because they could represent their avatar exactly how they wanted to be represented, and they didn't have that same level of scrutiny as they would do in, in the, in the physical world or on a video conference call. Yeah, I really like it for, I work with a pharmaceutical who were experimenting with for like, their grad program and onboarding, and they brought people together. And then they were doing a mix. They were doing a mix of in-person and virtual kind of alternating. And they said that it worked really well for, you know, like the they kind of digital twin concept, you know, they had the same space and it was like people got comfortable being able to have side conversations and breakout and those kind of things. that's, that's really interesting. Right? Cathy? I wouldn't be doing my job if I didn't ask, because this is the number one question I get is like, where is the line in Mesh between what's in core and or in, you know, Teams core and and then Mesh premium? What was where is that line. So the things that are built into the Teams experience. So that would be your avatars and that is being able to go to view Immersive Space and just walk in. That's all in the core license. it is core commercial right now we don't have edu or gov at this point. But you know, hey, we just released in January. So you know, give us time there. and that frontline worker because we don't see this as being something that frontline workers are necessarily doing. But you know hey if the market. Is busy in the real world, they're typically. Exactly. Yeah. But but even, you know, like Business Basic or your E3, your E5, it's all just included. And so it's really just up to the admins to make sure that, that the apps are available so you don't have to install the separate app for the immersive space, as long as the admin hasn't blocked it, as long as you have the ports open, you know it should just be there and working. Yeah. So it's when you go in, installed a separate Mesh app and use that, to enter into the virtual world. And being able to do all of the customization, it gets a little trickier with the headset because the headset, there's just one app. And so assuming you have the licensing, you can go to the Teams meetings or you can go into the standalone. You know, you don't have the licensing for one. It'll it'll block you on that one. But that's I think, where sometimes people get a little bit confused in the crossover. But doesn't that and like if it's in built into Teams then it's Teams core. If it's the standalone app, it's Teams premium. Okay. That's quite an easy way to understand actually. And you touched on like virtual world creation there and like yeah again not not an area I really understand. But like what how that work. Where do you start with that. Or you know, is that something that customers can do themselves. Do they bring in, you know, people like Jeremy's team? Is it how's that work? So in college, the most useful thing I learned was good, fast, cheap. You can pick two. And, you know, in Mesh. I've almost kind of blurred that line a little bit because you can get a lot of stuff that is pretty good and cheap and fast, just with the stuff that's out of the box. So we have a couple default environments that we ship with, and then we also have our tool kit. Now to use the tool kit, you need to have Unity. You need to have a license and you need to know your way around Unity. Like I said, I can't spell it. I'm not a programmer, but I was able to just get into Unity and upload our toolkit environments and do just like a little customizations in there. So I would call that kind of a like a low code solution. Yeah. And then I can just upload those using the tool kit from Unity. That's how you get it into Mesh. And so then it just sits as an environment. And then everybody who you add to a collection in Mesh, which is just an M365 group, can then go in and create events and use that and, you know, customize the environments in the app. If you want to do more than that, this is where, you know, you may want to hire Jeremy to come in and help build some some really amazing stuff because, you know, we've seen incredible things like, like the, the Global Mercy Ship. but but then it's going to take some pretty significant Unity skills and, and coding. You know, we we have things in our toolkit that will let you bring your data into the environment and, and do different things with it. The wonderful thing about using game engines is being able to create a completely a completely different world. That is impossible physically. So this is actually one of the benefits people talk about. Yeah. The, the advantages of, of, of the physical world over the virtual world. And it's very clear there are many advantages of the physical world over the virtual. But this is an area where actually there is an advantage to going virtual in that you are not constrained by the rules of the physical world. So you can you can create a world that is that is completely open to your imagination. That would be impossible to to create, in the physical world, or let's say very time consuming and full of hassle, complexity. Whereas if you're coding it and you've got a skilled team, you can create some really wonderful awe inspiring worlds that, that motivate and inspire. Yeah. It's interesting is and so I think. I was going to say two other things that I will add as an event organizer that we're virtual completely has it over physical. I have worn out shoes in the Orlando Conference Center in Mesh. I just teleport. And also when I'm in a Mesh event I can see everybody's name tag. I'm not doing that awkward. Kind of like, yeah, yeah, I see your name from here and it's all tied into graph, so I can even click that. In in the virtual worlds. We haven't spoken about this, but I know we were chatting about Cathy before. The real advantage that is similar to the physical world is that sort of that watercooler meeting where you bump into someone, you know, when you're in the office and, you're traveling from one meeting room to another and you bump into a colleague and you stop and you say hi and you have a chat and you catch up. It's impossible to do that in video conferencing. I've seen some people attempt it by pushing random touch point meetings with random people at random times, but it it doesn't. It feels quite forced. It's not as natural as bumping into someone in the corridor, and you can bump into someone virtually in a in a virtual corridor of one of these virtual worlds, and that does produce and elicit the same sort of reaction, the same sort of desire to catch up and see what that other person is up to. So, you know, you in summary, you really can use these virtual environments to drive greater connectivity in the workplace. Yeah, I had an interesting use case. We, we did a hands up at our last user group of who's using Mesh in a few hands, went up, and one of them was a big energy company, and they built a virtual operation center. So kind of my head automatically Mesh goes to kind of where you said Jeremy, like, I can build a, crazy virtual world and do all sorts of clever things. And they were like, actually, it's not it's not crazy. So it's very practical, but they've got, you know, tons of virtual screens. And they were like, this is like a quarter of 1 million pound build out to build in person. But we could build it all virtually. And people from all around the world in operations could jump into the same space and work on issues. So I thought that was really interesting, cause my mind didn't immediately go to a very, you know, kind of practical use case, but I've got tons of value out of it. I just if you if you look at the pure investment of building it, not, not the least that people around the world couldn't be in the same space. But that was a really interesting use case. I thought. And that's a great advantage of, or example of how it doesn't have to be sort of magical, wishy washy, you know, fun stuff. You know, it can actually be hard core, you know, down to earth, just business cost, a case, you know, here where we can't we can't ship this massive asset that cost millions of dollars over to this location because we want to try and sell it to a bunch of customers. It's just doesn't make sense. But we can recreate it in a virtual world to give some sense of it to customers all over the world. Awesome. So I put a pin in that question earlier, but now AI like like it feels like potentially that is a a huge unlock to that idea of creating the virtual worlds, engaging in new ways. I guess a question for both of you like is that is that coming up in conversation? Is that starting to be thought about because that potentially you see this image generation of video generation, I'm assuming, world generation is a possibility. Yeah. Well generation you can't. There's a video that we put up around Ignite last fall and it is not product truth. But if you go to Mesh.com, you know, there's a video up there that shows the kind of things that we're thinking about, future words of, things you might be able to do, like be speaking your avatar, be speaking your environment. So, yes. Are we thinking about it? Every single person at Microsoft is incredibly focused on AI I promise you, there's not a leaf unturned. And it's not AI focused. From, from my perspective, you know, as Cathy said, I think everyone's thinking about AI at the moment. And but more than that, I would say we go one level up, because right now we're talking about the connection or integration of, of AI and, and immersive technologies. but you know, what? If it's a AI and augmented reality or, you know, AI and the internet of things and augmented reality, it's really the scene we're talking about here is the convergence of emerging technologies. And that's where we see the most value being provided. It's when you start to see where the linkages are between all of these emerging technologies, to create a solution that adds the the ultimate value to organizations, that is, that is the, the framework with with which we're analyzing business problems. And I think is, is a useful one to consider, not only for artificial intelligence, but for every other emerging technology out there that we're all currently exploring right now. Awesome. Yeah, just being new and shiny isn't good. It's got to provide something to what you currently have. Yeah, definitely. Oh, Jeremy, if people want to find out more about you or PWC and your team, what they do, what's the what's the best thing to do? Probably best thing. You can visit my website, JeremyDaltonXr.com and then you'll have my LinkedIn details. some of the work I've done, the reports I've written. yeah, that'd be good place. Awesome. Thanks. And, Cathy, you've shared some great AKA links with me. I'm going to put that in the show notes for everybody, but if people want to reach out to you will find out more about Mesh. so I'm on LinkedIn. you know, Cathy with the C and, in the URL, it's Cathy loves tech is the, the handle that I have there and, you know, also the, the mesh.com site. And then we have a page called, we have adoption.Microsoft.com, and there is a Mesh adoption page. You can get to it, aka ms. Mesh adoption page. And that has a bunch of material that we put together about events we've actually run. It's it's all of our collective experience. We're trying to just, you know, distill it down for you. And if anyone wants to read that study that I mentioned, Tom, shall I give you the link to put in the show notes? Yeah. Please. Please. Yeah. we’ll add that to the show notes for sure. Great. Well thanks both. Appreciate you joining us sharing the stories. And, yeah, Hope everyone enjoyed that. If you've got any Mesh questions. Feel free to drop them in the comments and we'll relay them as well. Thanks a lot. Thanks Tom Thanks Guys, See ya