The Marketing Perspective of LiveOps w/Alex Hunnisett: The Art of LiveOps S3E4

Alex Hunnisett (@Blondagles) has been serving as Head of Live Service at Rare, supporting Sea of Thieves on its continued voyage into Live Operations. Having originally cut his teeth in digital and product marketing before taking the leap into game development, Alex has worked with several studios supporting their Games through Communications, Community, CRM and Live Operations.

Alex will give us some perspective on his history of running LiveOps on “Sea of Thieves” coming from a marketing community perspective. He uses an analogy of “The Hobbit” to illustrate the transition from boxed products to LiveOps.

This episode brought to you by Azure PlayFab: https://www.playfab.com

Transcript

Copy Transcript Copied

00;00;05;00 - 00;00;06;14
James Gwertzman
Hello. I'm James Gwertzman.

00;00;06;14 - 00;00;09;23
Crystin Cox
I'm Crystin Cox. Welcome to The Art of LiveOps podcast.

00;00;15;07 - 00;00;25;01
Crystin Cox
Everyone, Crystin here. I'm on my own today. So. So, no, James but I am joined by our awesome guest Alex Hunnisett.

00;00;25;01 - 00;01;00;21
Alex Hunnisett
I'm Head of Live Service here at Rare, currently working on "Sea of Thieves" and "Everwild". Been here for five years now. So looking after kind of the prerelease of "Sea of Theives" now the launch and kind of continuing to work on the live service kind of management and delivery of everything pirate over here. Before that, I've worked a few studios across the industry working on a variety of live service games, but I actually cut my teeth on kind of live service over at Xbox on the console side of things, looking specifically around kind of the dashboard and really kind of that as a home and a community and something that we're optimizing and managing.

00;01;00;21 - 00;01;24;12
Alex Hunnisett
So very still, still much, very much in the kind of livert service sphere, how we were managing, looking after stuff, but not a kind of traditional game product, actually looking after the dashboard itself before kind of taking that experience and moving it into a studio side. More of the kind of experiential games have side of things. As I went and did that, I actually moved into community and marketing as opposed to traditional live service liveops, live development, which has been super, super interesting.

00;01;24;15 - 00;01;41;29
Alex Hunnisett
Rare has such a kind of built in an embedded community marketing team that I ended up so embedded in live service and LiveOps that I ended up having it up. It's a really interesting lens to have coming from that background as opposed to a traditional kind of background. So it's really helps us thinking about audience first and everything we do with live service and live ops, which has been a great journey.

00;01;41;29 - 00;02;10;10
Crystin Cox
So we're going to have a really interesting conversation about Rare's journey to LiveOps. You know, going from a really long history of sort of box product development to running this super successful game in "Sea of Thieves", which has seen incredible growth over its years of live operations. Alex has been there for all of that journey. I think it's going to give us some really interesting insights into that. He comes from a more like marketing community background as well.

00;02;10;10 - 00;02;53;17
Crystin Cox
So I think he has an interesting perspective on how LiveOps is done. It's very, very community focused. So let's dive right into that discussion. [music] So often we talk about LiveOps. We do sort of vaguely touch on this idea of services outside of game development, but we actually rarely see people in the industry who, like you actually came from doing sort of more platform, really deep services work and then moved in to doing game development.

00;02;53;26 - 00;03;08;02
Crystin Cox
How is that transition? What did you find to be sort of the biggest difference between doing a really big service like, you know, Xbox Live and Xbox accounts and the platform on Xbox to making a video game?

00;03;08;16 - 00;03;30;20
Alex Hunnisett
Sure. I mean, there's so many nuances between the two and also so, so many similarities as well. But I guess kind of the biggest kind of cultural shift that I went through was one of like obviously when we're talking about video games and we're talking about kind of a pick up a pad and plan on playing, really, you're talking about that being a design led experience where kind of the objectives are.

00;03;30;28 - 00;03;50;03
Alex Hunnisett
I want to I want to keep in this product for as long as possible and have the most compelling, enjoyable, delightful experience. Whereas coming from kind of a service and more of a platform side of things, ultimately we're trying to provide you the right service. We're trying to provide you the right experience. But ultimately, especially kind of when we're talking about Xbox as a platform is a transition right?

00;03;50;04 - 00;04;17;02
Alex Hunnisett
We don't expect you to spend hours upon hours deeply engaged on the dashboard, although some people do. Some people do. Some people really love what we were putting out, which is awesome. But really, it's kind of it gave us a much clearer set of objectives and a much clearer set of metrics. Whereas when you're going into especially kind of thinking about talking about kind of "Sea of Thieves" and Rare where there's a transition between the studio at Rare very much they were kind of focused on boxed products and all kind of traditional game development experience.

00;04;17;13 - 00;04;43;15
Alex Hunnisett
So it was almost going from one extreme to the other and figuring out where we met in the middle between services, all about kind of the objectives and the metrics and ultimately how are we getting giving players the kind of the consumer experience they want and where where do we need to get them? What are our objectives as a service versus a game which is more around what experience we are showing up and how we delighting players and curating these more kind of traditional game experiences?

00;04;43;15 - 00;05;00;10
Alex Hunnisett
And where do we meet the two as a live service? How do we move beyond that? Kind of like we built the thing, we ship the thing so I guess those are kind of like the contrast between the two culturally is one is very kind of objective driven, whereas one was very much kind of still an objective, but the objective wasn't what I'd call continuous.

00;05;00;10 - 00;05;19;23
Alex Hunnisett
It was more of a finite approach. And obviously as we've adapted "Sea of Thieves" and we've adapted kind of Rare's culture, we've moved from what I'd almost call kind of like that finite objective of we built the thing, the thing is done. We've ship it. Everybody good job to this kind of effectively this infinite goal of we're never going to be done.

00;05;19;23 - 00;05;32;14
Alex Hunnisett
We're never satisfied with what we delivered. It's all always about the optimization. It's all about how do we continue to move towards these kind of infinite metrics, if you like, as opposed to the we've wrapped up. We produced, we've shipped job done.

00;05;32;20 - 00;05;57;16
Crystin Cox
Yeah. And that traditionally we've talked to other developers. That's such a big change. Right. Like Rare has this long history of triple-A development, that's very sort of, as you say, box product focused like we have a vision, we go and execute at a high level. And then, yeah, we did it. We deliver it out into the world. We talked to other developers, and that transition is so challenging for the team.

00;05;57;23 - 00;06;10;15
Crystin Cox
And you sort of been there with them that whole time, as you said, starting more on the marketing community side, but deeply embedded, as you said. So really with the team. What has that been like for Rare?

00;06;11;27 - 00;06;35;20
Alex Hunnisett
So it's been interesting, and our studio director actually has the most amazing metaphor for this, which I'm going to steal. So and he compares kind of our transition to effectively forming a fellowship. And so we're going to go full Tolkien with it. But basically he's always described kind of we started off with hobbits quite happily in the shire making these making these features right.

00;06;35;20 - 00;07;02;27
Alex Hunnisett
And they've been making features for ages they're very, very happy with it. And we kind of sat there going like the core of what we're doing at the core of this adventure that we're going on. It's still all about those features. It's still all about that, that kind of fun experience right. But we're aware that it's not necessarily going to be a straight forward, is not going to be all kind of, you know, chilling in the shire, if you like, So we were aware that there were other, other roles that we needed in place and we needed to form a fellowship around that.

00;07;03;12 - 00;07;28;07
Alex Hunnisett
Now our studio director labors as metaphor on, on quite a while, but to kind of give you the comparison, Ultimately it is about kind of identifying what we didn't know and the spaces that we didn't have. So how do you bring in kind of your your high charisma characters, your PR, your brand, your player communications, your marketing, which is where I kind of came into the business and where kind of I came into a newly growing team at Rare which previously hadn't existed.

00;07;28;20 - 00;07;47;29
Alex Hunnisett
And alongside us, which other characters do you need involve? So stuff like business intelligence and data was a huge one, which we always compared to Gandalf and rightfully so because they are wizards, but again, kind of like how how do we then go to understand not just in terms of what we're delivering initially, but that continued service around that, which is another muscle that we didn't have.

00;07;48;17 - 00;08;11;01
Alex Hunnisett
And then the two others, which would be quite telling in terms of how this especially went for Rare, you had kind of your development operations that like how we're then kind of building that robustness into like the things we are building like continuous delivery, which is always likened to Gimli. And then when you're talking about LiveOps, kind of the thing we're talking about today and kind of like how were you, how we react, how we're proactive and how we continuously deliver these experiences to the space.

00;08;11;12 - 00;08;34;12
Alex Hunnisett
We went and compared that to Sean Bean to the Boromir, which is pretty telling of how that then went for us and we went ahead and launched. But I've always loved that analogy. I've always loved that kind of the comparison. So I think it really speaks to how we actually went and formed that team around something that was well known, well-defined, and tried to effectively shepherd these these hobbits into Mordor and how that went.

00;08;34;12 - 00;09;00;22
Alex Hunnisett
So, yeah, obviously a very difficult first act for LiveOps. But again, it's how we adapt and we forge forward with, with the team that we have, right? So that's been really, really interesting. And I can't call that enough. Yeah. The data team that we're working with who again kind of came in alongside player communications and brand again with that kind of traditional approach and kind of how they culturally fit it into that because everyone perceives them to be wizards, but obviously then therefore it's magic.

00;09;00;25 - 00;09;20;01
Alex Hunnisett
How do you understand that? And I think culturally that was probably the biggest challenge was how we how we started to leverage data and how we started embedding that into into our approaches to a team that's traditionally you might get some user research around kind of level design. You might kind of go validate. Yeah. How does this feel experientially get those play testings in there.

00;09;20;01 - 00;09;47;17
Alex Hunnisett
But in terms of like everything from getting quantitative hooks in early on and not making that part of your production and dev process right through to how you're then blending well-understood kind of quality feedback that a lot of kind of designers, especially with kind of the 30 year pedigree that Rare has, how we then combine that with the quantitative in terms of here the numbers and what these means and how we make sure we're getting the right data to the right people was a real cultural transition for us and one we've got really good at.

00;09;47;17 - 00;10;03;20
Alex Hunnisett
We've got an amazing player intelligence team as we call them here at Rare. So it's been I'd say that's probably the biggest challenge and the biggest kind of shift that I've seen from where I started through to where we are now, which obviously doesn't actually traditionally set my discipline, but blends really nicely with kind of where we've been.

00;10;03;20 - 00;10;27;00
Crystin Cox
Yeah. No, I would say a lot of times when we talk to people about their like actual functional LiveOps teams, there is this question of data because so often the people on the team who are using the data are not data experts. Right. Or they're not the wizards. There is there is often this sort of group of wizards who are going out and like getting the data or analyzing the data.

00;10;27;00 - 00;10;47;07
Crystin Cox
And then someone has to take that data and do something with it, right? Like in order to actually make it functional to complete the LiveOps loop and actually use the data to make decisions. And so sometimes it's design, sometimes it's production a lot of times for teams, I think are more where you guys are at a little bit more sophisticated.

00;10;47;07 - 00;11;07;15
Crystin Cox
It ends up being someone like you who is running a LiveOps group, right? Who has got the more holistic view. I am curious to hear more about for you to talk about now. Yeah. What do you do with this data when the when the Wizards come and they deliver it to you after, you know, many years at this point, you know, "Sea of Thieves" is several years in.

00;11;07;26 - 00;11;11;09
Crystin Cox
What do you find yourself actually using the data for?

00;11;11;09 - 00;11;33;02
Alex Hunnisett
oh. I mean, the good news in the success story here is all sorts. It really is ranging. And I think where we've really succeeded is creating an appetite for data across teams, like you say, depending on kind of the orgs you work in or how the studio sets up, you could find it's sometimes it's just individuals within a variety of disciplines.

00;11;33;02 - 00;11;48;22
Alex Hunnisett
You haven't have a particular appetite for data. So like early on, we saw that, you know, we'd call out specific producers or designers. You kind of just we're a little bit more in tune with that. It might be their background. It might just be like how they think about design would be the ones of the appetite for that.

00;11;48;22 - 00;12;05;24
Alex Hunnisett
So you kind of see these kind of organic blending between the wizards, if you like. I mean, yeah, I'm going to keep on calling them. They deserve to be called that, so I'm going to keep on calling them that. So the Wizards and these individuals who are looking for answers, I think really how we're using it today is kind of macro in a micro level, if you like.

00;12;06;04 - 00;12;31;07
Alex Hunnisett
So we use a macro level, which is typically where I show up is kind of looking at actually we have our idea of what product health looks like. We have our idea of like what success for "Sea of Thieves" looks like. And that's a combination of kind of your traditional live service data alongside like how we look to understand the player experience. Are people having a good time? Which is often conducive of the kind of traditional service stats that we look like.

00;12;31;07 - 00;12;49;04
Alex Hunnisett
That really is kind of taking that macro level and looking to kind of proclaim that, if you like, across the studio as just a hey here's a rally cry, here's how we're doing this, you know, gets people kind of bought in and understanding this is what's important. But then also kind of looking to define with the rest of kind of the product leadership team.

00;12;49;04 - 00;13;18;07
Alex Hunnisett
Okay, if this is what the data is telling us, how does that then affect our road map? Where are the opportunities for us longer term? So that's where you kind of see a macro level is ultimately like what's the health of our product? Where is the experience looking great? Where is the experience not looking so great and then how can we use that to inform kind of how we look to write about at a more micro level and what's been awesome is to see anecdotally the last kind of six months, typically what would happen is I'd have to or, you know, myself or want to kind of the other kind of what I call kind of data

00;13;18;07 - 00;13;32;17
Alex Hunnisett
leaders or data advocates would be trying to plug these folks together. And what we're seeing more and more often is from a cultural point of view, that you turn up in the conversation, they're like, oh yeah, we were very spoken to the Wizards. We've already had these conversations and then, you know, you've won, then you know, you like, right, awesome.

00;13;32;17 - 00;13;50;20
Alex Hunnisett
What we're, we're in the right space now where there's this active curiosity around I'm designing this thing, and I really want to know what the player's journey is through it, be that from kind of traditional kind of design that will funnel in terms of like, cool, I know the Player is going to go from A to B and a C D in the middle.

00;13;50;21 - 00;14;11;01
Alex Hunnisett
Like, what does that interaction look like? Right through to a Hey, I think there might be this gap anecdotally from either knowing the game or playing the game. Can I have a conversation with you about it? So we're seeing more kind of micro level bespoke conversations happening with the with the data team, but at a high level, it's kind of that consistent.

00;14;11;12 - 00;14;33;28
Alex Hunnisett
The consistent trends across kind of what we've identified as a product health metrics is kind of really where I look to steer with the rest of the health team and go like, hey, here are the big rallying points. And then within those kind of better understanding those micro opportunities around particular features or particular audiences or particular events that we could go here and using that to kind of really whittle down and shape and refine what it is we're looking to do.

00;14;33;28 - 00;14;53;06
Alex Hunnisett
So it shows up in kind of those two main areas but yeah, the big the big shift for Rare has been that kind of like going from a understand the data's there, the telemetry or getting that even from an engineering point of view, kind of getting those telemetry hooks in was kind of ...I understand that we've got to do it.

00;14;53;06 - 00;15;11;09
Alex Hunnisett
But it almost feels like if we're trying to, especially when we first launched you're talking about us. We've come out, we've come out big, we've come out strong, probably bigger and stronger than we realized we were going to do. So then it's okay. You know, traditional liveops game story, a victim of your own success, which is awesome. But then it's like right when he's go small stuff, we need to keep the players happy.

00;15;11;24 - 00;15;24;04
Alex Hunnisett
But then that's like another ask, so understanding the value of why you have that telemetry and from a first place has gone from a case of like you having to sell that and every time to a point where it's almost like the first thing on the the breakdown list is right. So we need to understand what's going on here.

00;15;24;05 - 00;15;25;08
Alex Hunnisett
So that's just been awesome.

00;15;25;23 - 00;15;54;04
Crystin Cox
That's great. Yeah. Hearing that, that building the team's confidence in data and you're really building a culture of data, right? And it takes time but that's awesome to hear. And I know for you, especially coming to the background to come back, you come from so important the community side of it, right? Like how do you roll in player feedback into that pipeline?

00;15;54;04 - 00;16;07;08
Crystin Cox
Right. This is a I think this is an ongoing question for everyone in LiveOps, right? Like how do you take the qualitative and the quantitative and make them, you know, hopefully they work together, but sometimes they're actually opposed to each other.

00;16;07;26 - 00;16;30;29
Alex Hunnisett
Totally. And it's it's a real challenge that we've kind of faced is almost kind of like how do we complement or contrast these? And like you say, sometimes they give you different indicators. So at a high level, there was an certain experience we were looking at and what we were getting back from a from our community team. So kind of I've got a regular catch over head of community.

00;16;30;29 - 00;17;01;11
Alex Hunnisett
And what she was saying was across kind of like her cool partners. So just for context, we kind of solicited, solicit quality feedback across a bunch of cohorts with Rare, so we've kind of got our, our partners who are kind of deeply engaged "Sea of Thieves" players you'll advocate for us, we've got our insiders folks that we've had since day dot before that who kind of support us through PTR testing and kind of like we kind of put out early builds to them so your traditional insiders program.

00;17;01;11 - 00;17;33;29
Alex Hunnisett
So obviously we get feedback from those guys well across just kind of general social listening and kind of engaging with our community. Right? But ultimately like that community, although incredibly important part of our audience are effectively a subset of our audience. Right? They are they are a section and not necessarily representative of the whole. And what we found was this particular experience we were getting a lot of feedback saying like, hey, doesn't feel like this particular experience is rewarding enough, doesn't feel like we're actually getting enough bang for our buck here,

00;17;33;29 - 00;17;50;13
Alex Hunnisett
so it it really worth our while? So that got bubbled up to us and we're like, Ha, we thought we had that well balanced. That's interesting. Let's go have a look at the engagement have a look at how it's delivering and what we actually found when we looked at the distribution of rates there was although yeah, absolutely.

00;17;50;13 - 00;18;12;17
Alex Hunnisett
For that top 10% you're seeing it's effectively inconsequential for those folks who are the ones who are then going engaging kind of the forefront of the community and therefore kind of taste mazing when you look at the actual distribution across the breadth of it, we'd gone too far the other way in some regards. So it was almost like too generous. And what we were seeing was kind of two contrasting behaviors where like ultimately if... you can't serve everyone naturally.

00;18;12;17 - 00;18;38;10
Alex Hunnisett
And what we were seeing was, okay, so we're missing we're missing the sweet spot here to the point we're giving them. We're progressing them so much that they're losing interest. And conversely, for the folks had already kind of progressed this at they're going, ha, this is, this isn't quite enough for us. So then and that's how you kind of like both of those how we looked at the quant first initially and then gone to the qual, we probably would have found the same story.

00;18;38;15 - 00;19;00;12
Alex Hunnisett
So I've wanted a really clear signal for this something to go deep on here. But it's always about adding those lens and ultimately the steer there is okay. So what we're hearing there is like this experience of what we're looking at it is there a tool that we can develop that actually goes if this is a universal experience that plays across a spectrum of engagement are going to go hit, do we need to be more intelligent about how we're rewarding it?

00;19;00;12 - 00;19;18;17
Alex Hunnisett
And is that actually the opportunity here is to give the right message or the right rewards at the right place at the right time is, you know, ultimately this we're talking about a progression system here. And that depends where you can end up in a space where if you're talking about a specific feature and the feature or the experience or the events, intent is to go hit a specific cohort and you can be more refined around.

00;19;18;20 - 00;19;38;28
Alex Hunnisett
It's not hitting that group of the audience, but that's okay. That's cool. We're good and we're comfortable with that. But in this specific instance, that was a really good recent example of where we kind of we got the qual signal and then combine it with the quant and went well, no matter which way you cut it, we're not doing the right thing or we're missing where we are in original intent.

00;19;38;28 - 00;19;57;02
Alex Hunnisett
So how do we refine that? But I think it's always about kind of taking that prism of, yeah, the quant and the quall, and then seeing where you land is challenging, but kind of taking Ivory's gospel can can lead you to making some, some decisions that maybe out the back of your like huh, that didn't do exactly what I thought it did.

00;19;57;02 - 00;20;05;20
Alex Hunnisett
But that's optimization, right? Like ultimately it's a case of you make your hypothesis and you just get smarter and smart with very shot that you take. [ad break music]

00;20;10;29 - 00;20;45;20
Crystin Cox
[ad] The Art of LiveOps is presented by Azure PlayFab. PlayFab is a full featured back end solution that powers liveop games of all sizes. Whether you need multiplayer services, analytics, LiveOps tools, PlayFab can help you reach all of your retention, engagement and monetization goals. Learn more and you can get started for free at PlayFab.com. That's PlayFab.com P-L-A-Y-F-A-B dot com and you can get started for free. [ad break music]

00;20;45;20 - 00;21;19;29
Crystin Cox
You're listening to The Art of LiveOps podcast and today we're chatting with Alex Hunnisett. We're talking about all sorts of interesting stories about "Sea of Thieves" and his time running the LiveOps team there. So let's dive back in. You talk a little bit about it, but a lot of times we see teams that do get up and running with a large community have to really start thinking about them as many different communities, right?

00;21;20;02 - 00;21;34;20
Crystin Cox
Have to start thinking about them as different communities that interact with each other. Do you guys, you know, use any like really formal cohorting or like psychographics or anything? Is that something you guys have found to be useful.

00;21;35;09 - 00;22;00;04
Alex Hunnisett
So we when we when we first started out with "Sea of Thieves" and we're kind of initially looking at our insider program, there was an element of you know, let's go do some psychographic we actually shot the kind of traditional Bartol types and went Okay. Let's look to understand ultimately, both for us understanding we went to E3, we announced the game, let's see who's registering an interest and where they sit and terms of player motivations.

00;22;00;04 - 00;22;28;02
Alex Hunnisett
Does this tell us from a marketing point of view or at least kind of the gameplay like are our assumptions correct? And the kind of audiences and the tastemakers that are being attracted to us and those are ultimately already engages, right? So understanding who's in that pool. So we did start with that. And as we've gone broader and broader, we found that ultimately when we look to kind of traditional psychographics and how we were looking at our player groups, they weren't necessarily reflective.

00;22;28;02 - 00;22;43;00
Alex Hunnisett
It was almost a case of, okay, I'm going to cohort you to this behavior and then find you're almost doing a different set of things when you go and introduce that. Right? So it's like, okay, so the world is a wonderfully chaotic place where you can try and be as predictive as possible and it doesn't always land that way.

00;22;43;22 - 00;22;59;10
Alex Hunnisett
But what we did find in terms of kind of talking about managing those different communities or setting those different expectations is there are things that are going to appeal to certain player types and things that aren't going to appeal to others who are always quite intentful and mindful about that when we're thinking about feature sets, like, who is this for?

00;22;59;12 - 00;23;22;22
Alex Hunnisett
How do we manage the message for like ultimately who's a group of folks who are going to love this? How do we engage with those folks and how do we set expectations for others? Like, Hey, if you want this to be a super aggressive, kind of combative experience, that's not what this is going to be. Or for folks who are look, hey, if you want something that's super chill and relaxing, that's not what this is going to be.

00;23;22;22 - 00;23;40;24
Alex Hunnisett
So it's all about kind of setting those expectations and making sure that you understand the different groups, your communities, and anticipating how they will react to stuff. And our community team do an amazing job of kind of like spinning those plates, which is community management for me. And you can my two favorite analogies are herding cats or spinning plates, but either one of them indicate, right?

00;23;40;24 - 00;24;03;21
Alex Hunnisett
There's a whole bunch of different facets and groups who all have, and as your service grows and your product grows especially and this has been the story for "Sea of Thieves", we started out effectively as an open world adventure game. Then we added more kind of naturally combative elements, then we and introduced more kind of narrative elements. And you're starting to appeal to these these different groups who then all kind of munge together in this world.

00;24;03;21 - 00;24;25;29
Alex Hunnisett
Right, which has been super interesting. But yeah, so it's all about kind of understanding those different groups ultimately where they live and where they sit and then just making sure those messages match with them. But we did kind of get to round off that question. We did start off with that, but we found our player base kept on surprising us irregardless of how we try to how we try to bucket them. Yeah.

00;24;25;29 - 00;24;48;28
Crystin Cox
You know, it's I feel like it's a classic story. Like I actually joke sometimes with my teams that we should definitely do the work of trying to understand who our players are going to be and, and create an idea of who this is for but I don't know that we've ever worked on a game where that didn't that survived sort of contact with the community.

00;24;48;28 - 00;25;05;00
Crystin Cox
It's important work anyway. A lot of times that does help focus you. But it is so rare for the psychographics or group or cohorts or whatever you call them to actually bare out in predicting behavior.

00;25;05;03 - 00;25;29;14
Alex Hunnisett
Totally, totally. And scale's a factor too, right? Like we started off with, you know, and our community continues to be awesome and super collaborative and it's grown beautifully. But when we started out that was, you know, attracted and then grew in specific kind of cohort. And then going into beta, we saw, okay, there's these new layers of community that came in that we didn't, you know, after years of managing this community, didn't necessarily know how to engage with.

00;25;29;14 - 00;25;47;09
Alex Hunnisett
And like you say, as you hit scale and you choose these different spaces, okay, well it's not going to survive. Your client isn't going to solve contact with if you play base effectively, they are going to surprise you and it's more a case of I think going back to that data point, understanding what that behavior is and then kind of then seeing seeking out the opportunities are there,

00;25;47;09 - 00;26;03;15
Alex Hunnisett
Right. Hey, this particular feature didn't really hit with the audience we think it did, are there reactive opportunities we have here? Are there proactive opportunities that we have here? How do we feed that back to the dev team so they can then and our design team to sit there and go like, cool, I better understand how this has been delivered?

00;26;03;15 - 00;26;27;00
Alex Hunnisett
And it's all about feeding that back. And, you know, live service for me is, is that constant optimization of various loops of scale be it for events, be it for kind of your micro events, your messaging or your features. It's all about that kind of closing the loop and going like here's ultimately what we learn as opposed to sitting there going, no, this particular audience didn't engage with it, therefore it's a failure.

00;26;27;00 - 00;26;42;02
Alex Hunnisett
And I think moving people away from that kind of mindset of it didn't do what we thought it was going to do, therefore, it's not good, has been something that, you know, a lot of a lot of teams have apprehension around, especially when you move to a live service game. It's not a case of you can box it and kind of walk away.

00;26;42;11 - 00;26;53;02
Alex Hunnisett
It's a case of you box it and then you're exposing the dev teams to the communities. Right. And you know, the Internet can be a scary place so being able to feed that back is so important.

00;26;53;05 - 00;27;24;21
Crystin Cox
Absolutely. Can be. I will say, though, I think that to me that is one of the biggest indicators of whether a team can be successful in LiveOps is can the team find a way to feel like that process of having your expectations sort of shattered by the community is an opportunity? And it's met with a lot of curiosity and a desire to go find out what we're doing next and where the game is going to grow or if a team kind of can't get there.

00;27;24;21 - 00;27;40;26
Crystin Cox
If they're like, I can't kind of accept a world where the thing that I wanted to happen didn't quite happen, or the community ended up growing in a different way than I expected, then I think it is so challenging for them to really make the transition to successful LiveOps.

00;27;41;05 - 00;28;04;04
Alex Hunnisett
Yeah, yeah. It is difficult again and again, talking about kind of shifting culture being exposed to that, like in terms of how we look to frame it, there's always a case of like, you know, we work in a profession, especially the community marketing team are best in class and the kind of tools they're using and how they're looking to understand the broader set of sentiment across the broadest audience.

00;28;04;04 - 00;28;21;15
Alex Hunnisett
And then combining that with data and working with our wizards, to kind of get that output and then feed that back in the right way is important. And you've got, you know, and once that that data has been munged and pulled together and kind of framed in a place of like, here's where the opportunities are, it's often a lot easier for our teams to digest it.

00;28;21;15 - 00;28;43;13
Alex Hunnisett
Right. However, like you do fall into the trap of we do, we do live in a world where social media and you know, a lot of our devs are in these community spaces, right? So they're getting effectively raw data. And in the same way that I wouldn't expect one of our data developers to go through raw economy data line by line and sit there and go "well, this player spent this much on this item.

00;28;43;13 - 00;28;57;06
Alex Hunnisett
"and the player spent this..." like you wouldn't expect them to do that. However, when you think about that from a qual point of view where if you kind of view community spaces, that's effectively what they're doing and that can be really, really bruising when you don't have that kind of summarized in a frame to in terms of an opportunity.

00;28;57;06 - 00;29;04;08
Alex Hunnisett
But just, you know, a Bob124 on the internet going "this didn't meet my expectations", which is what normally.

00;29;04;08 - 00;29;06;02
Crystin Cox
If only they said at that politely.

00;29;06;11 - 00;29;07;23
Alex Hunnisett
Yeah, exactly. Exactly.

00;29;09;21 - 00;29;35;16
Crystin Cox
It is so true though I think that it is something that we've touched on before on the show, but especially when we talk to people with a community background of any kind working in this space it really does demand quite a lot of developers and sometimes they're not quite ready for it. You know, you have to have these conversations where this is going to change your relationship to social media, this is going to change your relationship with the community.

00;29;35;21 - 00;30;06;14
Crystin Cox
And that's not to be scary. It's not necessarily always a negative thing. But as you said, if you're not ready for that and you just sort of go out there and consume raw feedback, you know, it's a it's a cliche to like not read the comments, but it can be something you do have to work with the team on actually to be like, no, really like you, for your own mental health, for your ability to think objectively about your goals, you kind of need to develop a different relationship with it.

00;30;06;20 - 00;30;29;23
Alex Hunnisett
Absolutely. Absolutely. It's it's something that I've and again, I think it's something where you can tell if someone's got the live service experience almost or you haven't based on kind of like how they interact with that as well. I've seen a lot of kind of what I would consider more traditional boxed prodct developers kind of come into that space and go, "Oh my God, this is this is what is this"?

00;30;29;23 - 00;31;03;29
Alex Hunnisett
And then it's a case of, of like, "okay, cool. Sit down and talk about what this actually means". But yeah, it is it's definitely once the thing is once you do get it in the right space, it can be so galvanizing, it can be so informative, and it can really, really help you actually go and, you know, delight players, which is ultimately what we're all setting out to do is create these awesome experiences for players but focusing on kind of yeah, that kind of and ultimately like as we all know from kind of like the way that folks naturally receive feedback is the more negative it is, the more that's going to stand out

00;31;03;29 - 00;31;17;29
Alex Hunnisett
and resonate with you. And you could see know 50 tweets saying this is awesome versus the one that says I didn't care for it and that's the one that's going to keep you up at night. So yeah, learning how to manage that's been an interesting one for the team.

00;31;17;29 - 00;31;31;21
Crystin Cox
It's fun. My favorite analogy and when I used to tell my teams is games, live games are a safe thing to be angry about. It's one of the services we provide. It's a safe to get angry about.

00;31;32;06 - 00;31;33;23
Alex Hunnisett
I love that. That's awesome.

00;31;34;07 - 00;31;50;17
Crystin Cox
It's ultimately low stakes for the players right like you can you need to blow off some steam stuff not going great. You can go on to a forum and complain about a balance change that's happened in a game that you love. It doesn't mean you don't love the game. It just means that it's a nice, safe release valve for you.

00;31;50;26 - 00;31;56;24
Crystin Cox
So trying to get it framed like that can help sometimes the teams sort of absorb why that stuff happens you.

00;31;56;24 - 00;32;18;22
Alex Hunnisett
You know what that is... I was having this conversation the other day. I was talking to one of my designers and it was a similar thing and it goes back. So I obviously I have a marketing kind of community background, and I was kind of like on to your point, like, I know it's painful that so-and-so may have not liked this feature, but think about it they care enough that that's their feedback.

00;32;18;22 - 00;32;44;13
Alex Hunnisett
Like, you're not trying to overcome anger in these spaces. You try to overcome apathy, right? And if you sat there going, like, people really love it or people dislike the thing, ultimately they're passionate about it. And like you, like you say, that's such a good way of thinking about it. Like it's a safe thing for them to become passionate about and find some dislike for but like ultimately when you look at it and you actually break down the number of instances why kind of adding the quant to the qual is so important.

00;32;44;13 - 00;33;05;10
Alex Hunnisett
Like something I tend to look back to is, yeah, a lot of folks weren't necessarily enjoying this feature. They still came back next week though, like it's not like people are throwing down and run away. It's like this wasn't the best experience for them maybe or maybe it was the best experience for them ever. But ultimately they care enough and they're still coming back and they're loving this experience and I think

00;33;05;10 - 00;33;12;22
Alex Hunnisett
that's such a good way of looking at it is, yeah. It's, not as simple as just Red Arrow/Green Arrow.

00;33;13;08 - 00;33;37;22
Crystin Cox
Yeah, it really isn't. And then it becomes an important part of community management to in some ways protect it being a safe space to get angry about things, protect it remaining something that people can engage with and be happy and sad about it at times, complain, you know, share joy and not become something that is fraught with danger.

00;33;37;22 - 00;33;44;03
Crystin Cox
Right? Like and that is, to me, a guiding light, too, for, you know, how do you tend to community right?

00;33;44;11 - 00;34;05;10
Alex Hunnisett
Absolutely. And it's something we were really keen about early on. So we actually had our kind of our community, your kind of early community write our community charter pirate code, as we called it, so very on brand. But that was very much about kind of like we had very clear ideas of the space we wanted to create to our community.

00;34;05;27 - 00;34;20;02
Alex Hunnisett
But then it was a case of let's add the layer and effectively let let our players define what that is. Like you say, how do we create a space that's right for you? This is ultimately you're the ones you're going to be spending time in this community, in this game, enjoying it. So what's important for you, for this product?

00;34;20;02 - 00;34;38;19
Alex Hunnisett
And that's something that we very much kind of used as our rather in a kind of austere when we're thinking about these community spaces and using that for community managers to to to help protect that space for our community and make sure that it continues to abide to effectively what they asked for. They got, right. This is the space that we'd like around the game.

00;34;38;19 - 00;34;42;16
Alex Hunnisett
And it's so like you say, it's so, so important to make sure that's maintained and that's protected.

00;34;43;01 - 00;35;11;08
Crystin Cox
So you guys been running a "Sea of Thieves" for a while now and you know, we know you're working on another game. You know, you guys have announced that you're working on another title a lot of times it feels like no matter how prepared you are, there's a certain amount of just adjustment and getting the wheels turning once a game goes live.

00;35;12;06 - 00;35;23;28
Crystin Cox
I'm curious to hear like how long looking back now, how long do you think you're in that state for "Sea of Thieves"? And how are you feeling about doing it again? And like, where are you going to be with the next title?

00;35;24;11 - 00;35;44;14
Alex Hunnisett
So I guess what's been really interesting about "Sea of Thieves", when you talk about like obviously like you say, we've been out for awhile, but we've kind of although we've moved to the live service, we have kept one kind of aspect of how we used to deliver box products, which is we always go for a big old beat and we try and hit a big beat every year.

00;35;44;24 - 00;35;57;18
Alex Hunnisett
The most recent, obviously, being, if I do say so myself, the amazing collaboration we did with Pirates of the Caribbean with a Pirate's Life, and that again, it almost felt like launch day again.

00;35;58;03 - 00;35;58;12
Crystin Cox
Right.

00;35;58;24 - 00;36;20;00
Alex Hunnisett
So that for us was almost to your point, kind of like how do we feel going and doing that again and again and again as a studio, that was almost kind of like, right. Well, let's think back to the day of launch where our struggles, where were the challenges? And then what do we anticipate that looking like when we go do it again?

00;36;20;00 - 00;36;41;22
Alex Hunnisett
So again, it's much, much better understood now than it was back at launch day. And my favorite quote from our technical director after we launched was like, that was delightfully boring. It was is and that's when, you know, you are in a good space. Yeah. You're always going to be. And that's not to say we didn't bump into challenges we didn't anticipate or new things.

00;36;41;22 - 00;37;05;12
Alex Hunnisett
And this is the thing like you think you've got all the spaces covered and you'll always have a blind spot. There'll always be something else to catch you out. That that's the nature of managing huge audiences on digital products. Right? But in terms of how confident we are about kind of releasing and doing big releases, I'd say that kind of each one of these we do we seem to be getting into a nice rhythm of doing one every year or so,

00;37;05;16 - 00;37;29;19
Alex Hunnisett
is with each of those, we're getting increasingly more confident and increasingly better oiled in terms of how we're delivering that. So it's yeah. And and the fun one is always kind of like knowing who's been there the year, who's been there, because obviously you like, as you say, "Sea of Thieves" has been out for a while. So there's folks who joined the team midway through development or kind of as the service has been delivered, There's folks who've been there since launch.

00;37;30;14 - 00;37;47;22
Alex Hunnisett
And you can almost see just from the reactions in the room on launch day when something does go wrong kind of like who was here last year, who was here the year before, who is here at launch? So I think it's as a studio we definitely got more and more folks in the business and better processes, etcetera, to handle that stuff.

00;37;47;22 - 00;37;49;26
Alex Hunnisett
But it's it's always a surprise for someone.

00;37;50;01 - 00;37;55;02
Crystin Cox
Yeah, you've got those veterans now who can put their arm around the new people and say, "it's going to be ok".

00;37;55;03 - 00;38;01;00
Alex Hunnisett
Yeah, exactly. Like in the grand scheme of things, let's offer some perspective because it could always be worse.

00;38;01;04 - 00;38;25;10
Crystin Cox
Yeah, absolutely. Oh, you know, I'm sort of curious. The lot of teams I think are either in the middle of making a transition like you like Rare did to boxed product to live service or would like to. I'm curious if you have advice, right? Like what would you say to a team that's about to embark on this journey?

00;38;25;10 - 00;38;37;29
Alex Hunnisett
Oh, wow. This is a great question. And it's almost like I guess another version of that question is almost like if I could go back in back four or five years time and talk to.

00;38;38;13 - 00;38;39;25
Crystin Cox
Yeah, give yourself the advice.

00;38;40;14 - 00;39;15;02
Alex Hunnisett
It's always the way I'm framing it in my mind. Is, Yeah, what would we tell ourselves? I guess really the interesting thing that we found is first and foremost, making sure that you've got the right data in the right place and you understand you have the best view possible to go affect, I think some of the biggest challenges I've seen not just with "Sea of Thieves", but just generally across kind of live service products that I've worked on is not having a clear enough view on what the problem you're actually talking about is.

00;39;15;10 - 00;39;37;03
Alex Hunnisett
So you may see this bubble up through, you know, qual data on on social media. You may see a red signal somewhere but ultimately not understanding kind of the scale of the problem can often lead you to make some some misled decisions effectively. So making sure that kind of your player insights and your data team both exist in some form.

00;39;37;03 - 00;40;00;14
Alex Hunnisett
But is there from the start, understand what it is you're trying to achieve and then how you go and measure that. So and we learned that lesson with seasons which we introduced to "Sea of Thieves" kind of lost last this time last year, which was effectively introducing an entire new progression system to the game. Right, right. Again, going back to launch, we were like, well, we didn't have the data to sit there and understand where our experiences might have been going on.

00;40;00;14 - 00;40;22;01
Alex Hunnisett
And we went and delivered that and we got better. So by the time seasons came around, we, we to a degree of certainty, you could sit there and go kind of like, Cool, here's how people are progressing. Here's where people aren't progressing. Right down to kind of like how many skeletons people are killing in-game and then what that looks like in terms of progression and where players may or may not have been killing too many skeletons and then figuring out how that was even possible.

00;40;22;11 - 00;40;41;00
Alex Hunnisett
But again, having that view allows you to zero in on that. And also the scale of the problem is this 100,000 players doing this? Is it one player doing this? And then how do we, you know, deliver a solution accordingly? So I think understanding the scale and the space of what it is when you do go, live is kind of step one.

00;40;41;19 - 00;41;00;16
Alex Hunnisett
And then step two is having the agency to effect that. So sounds like a no brainer. But if you're going to work a live service product, it's good if you can affect the meaningful things in a live way. So certainly coming from kind of a more kind of traditional background, we were quite reliant on production to begin with.

00;41;00;16 - 00;41;21;02
Alex Hunnisett
And we've got a really good space now, kind of service team are again best in class of what they do. And they've done an amazing job of introducing more and more tools that allow us to go affect that. So going back to seasons again, if we said, "huh, that's a lot of skeletons being killed", we can sit there and go, Well, we have a number of ways of then tackling this, right? So we have the ops way.

00;41;21;06 - 00;41;40;07
Alex Hunnisett
We can even bring a designer into a room and go like from an experience point of view, but also from a fun point of view, how do we want to go tackle this for the short and the long term? So I guess it's all about the visibility and then the agency, the two things that we I guess looking back, looking back on it, we wish we'd had day dop. It didn't mean that we couldn't do anything.

00;41;40;07 - 00;42;00;05
Alex Hunnisett
It just meant we were a lot more slow moving. So as opposed to delivering within hours, it was days, weeks, months, etc. before we could go affect things in the way that we wanted to and if I could go back into it all again, it would be having those views and those tools day one and day dop. But yeah, and again, you won't always know exactly what you want to effect until it's out there as well.

00;42;00;13 - 00;42;25;02
Crystin Cox
I think that was the challenge. There are so many teams doing it for the first time. Even teams that I worked with sort of agonized over like, well, what should we measure, right? How do, how do we conceive of a future where we know what we're going to want to know? And there's I think there's a lot of work you can do to help narrow that down, but it can be quite intimidating, I think for teams.

00;42;25;02 - 00;42;25;18
Alex Hunnisett
For sure.

00;42;26;02 - 00;42;34;26
Crystin Cox
So we're getting close to running out of time. Here, unfortunately. But there's one question that we like to ask everybody that's on the show.

00;42;35;06 - 00;42;36;11
Alex Hunnisett
I know what's coming?

00;42;36;11 - 00;42;40;07
Crystin Cox
Yeah. Can you share a liveops disaster with us?

00;42;41;04 - 00;43;01;00
Alex Hunnisett
You know, I love this podcast and this is my favorite bit of the show is always sitting there. Go, yeah, tell me, tell me give me some perspective. That's what I need this week when listening to the podcast, someone else offer me some perspective, and then when you invited me on, it's like, oh, no, that was the, the dawning realization of What are you going to say? I know all this...

00;43;01;00 - 00;43;01;09
Crystin Cox
It's your turn.

00;43;01;25 - 00;43;35;15
Alex Hunnisett
It's my turn is there's so many things I could say. I really have to wrap my brains, but I think honestly, hopefully not sounding like too much of a copout. I think the biggest liveops disaster that I've experienced with "Sea of Thieves" was launch. That was, it was an interesting one. So I guess ultimately the way we were going into launch was one of to make a very long story short, we had this cool pirate game that we thought, you know, we had an audience for and we thought was going to be great and everyone was going to enjoy it.

00;43;35;29 - 00;44;00;07
Alex Hunnisett
And not too long before we went to launch, we were the kind of the offering for Xbox Game Pass changed and all of a sudden it went from effectively a boxed product with a live service continuing to we will be day and day on Game Pass, which is obviously this amazing service but increased our scale by infinitum.

00;44;00;07 - 00;44;19;11
Alex Hunnisett
And so all of a sudden it was kind of like everything that we thought we were scaled for was blown out of the water. So then it's a case of yeah. In that space of going, okay, cool, we thought we could handle as many players. We cannot we are down. And then just that was, that was a real, real fun, fun couple of weeks.

00;44;20;22 - 00;44;35;20
Alex Hunnisett
But ultimately, I guess I guess the big lesson that we had from that was yeah, the communication that we had off the back of this. Yeah, things went down. It was very much like the siege. We were going to go back to a Lord of the Rings analogies, but then it's like, how did you break that siege pattern?

00;44;35;20 - 00;44;58;24
Alex Hunnisett
Which we actually do through that communication. We had our creative director, head of studio and our executive producer around a whiteboard. Just hitting YouTube videos every day going like "here's where we're at, folks. There are more of you than we thought there would be. So here's how it's going". So it wasn't all doom and gloom, but it was definitely kind of if you imagine going day and day and going like," Cool, we're up and we're back down again."

00;44;58;24 - 00;44;59;00
Crystin Cox
Yeah.

00;44;59;17 - 00;45;01;27
Alex Hunnisett
Was it was it was a fun one.

00;45;02;04 - 00;45;13;27
Crystin Cox
You know, there's it's there's nothing like being live too, like you have problems that you think I have no idea how we would solve that. And then you go live and you have it and you just work it.

00;45;14;03 - 00;45;14;12
Alex Hunnisett
Yeah.

00;45;14;23 - 00;45;45;12
Crystin Cox
You just work it until you solve it. And at a similar launch, you know, even sometimes things happen and you know, you it seems wrong sometimes to complain about success that way. Like, Oh, I'm so sorry. Too many people showed up. But it can actually be quite challenging for a live game, you know, sometimes. Sometimes it will. Oh, I like I look over and some teams that have been able to start very small and then just grow organically and I'm a little jealous.

00;45;45;12 - 00;45;49;12
Crystin Cox
Like, I'm like, Oh, you know that. But then they have their own problems.

00;45;49;12 - 00;45;50;26
Alex Hunnisett
Absolutely.

00;45;50;26 - 00;46;09;10
Crystin Cox
But that problem of, oh man, too many people showed up can be can be quite daunting. And sometimes, you know, people there's a lot of talk, I think, in the community about it, like why can't you just know every every game falls over at launch. Can't you just plan for people? And I wish it was that simple.

00;46;09;10 - 00;46;11;04
Crystin Cox
Like every. I really do.

00;46;11;20 - 00;46;30;08
Alex Hunnisett
Yeah, likewise. And it was honestly one of the most hit posts because we were having this discussion as the community team is like obviously we're getting that feedback from the community. Why isn't as simple as just so our technical director went and just wrote this super technical blog post and went like, "here you go". And obviously that didn't land with the entire of the community.

00;46;31;05 - 00;46;46;07
Alex Hunnisett
But what that did lead to is a really interesting advocacy of people who just wanted the game to be up who honestly had had a bit of time to kill because it wasn't when I read this article and then started saying, Hey guys, I think I understand it now. And like that, that kind of then rippled out through the community, which is pretty nice.

00;46;46;07 - 00;46;55;23
Alex Hunnisett
But yeah, I think if you're starting big or starting small, you're always going to have your own problems and your own kind of routes forward. But yeah, it was painful, but got there in the end.

00;46;55;28 - 00;47;12;20
Crystin Cox
Radical transparency can also be so helpful. I do you try to tell teams to try not to be afraid of the player base and try to try not to be afraid of your community? You know, you can be a little trusting, so you know that they won't always return it, but more times than you would think, they might surprise you.

00;47;13;10 - 00;47;28;08
Alex Hunnisett
100% never underestimate your community. Ultimately, you all share a love. You all share a passion for your game and like that, that should be unifying folks. And they do have the interest rates of being honest and being transparent can normally yield goodness for both sides.

00;47;28;20 - 00;48;00;28
Crystin Cox
Absolutely, also I will say, like just I mean, I know the story of "Sea of Thieves", but I do just want to call out how it is so hard to go do something that hasn't been done before. And when you guys were watching it was really like, well, this is the first time we're going to be bringing something out like this into day and date on Game Pass and you don't really know what's going to happen. You know, you can make a lot of predictions, but that is truly one of the harder things to do to go do something that no one else has done before.

00;48;01;00 - 00;48;20;05
Alex Hunnisett
Call it masochistic. But it's something that Rare loves doing. So I think that's going to continue to happen. Yeah. And we've yeah, it's every day is a new challenge, a unique challenge in being able to sit there and kind of figure out, okay, how are we going to go tackle this thing and try things differently? Has been genuinely the best part about working at the studio.

00;48;20;09 - 00;48;21;10
Alex Hunnisett
So yeah.

00;48;21;20 - 00;48;33;15
Crystin Cox
Well, thank you so much for taking the time to have this conversation today. I appreciate you sharing your, your liveops disaster and all the things that you and the team have learned over the last couple of years.

00;48;33;29 - 00;48;36;18
Alex Hunnisett
It's been awesome to be on, Crystin, thank you so much for having me.

00;48;40;22 - 00;48;43;07
Crystin Cox
Thanks for listening to The Art of live Arts podcast.

00;48;43;07 - 00;48;48;13
James Gwertzman
If you liked what you heard, remember to rate, review and subscribe so others can find us.

00;48;48;13 - 00;48;53;02
Crystin Cox
And visit PlayFab.com for more information on solutions for all your LiveOps needs.

00;48;53;11 - 00;48;54;07
James Gwertzman
Thanks for tuning in.