David Nisshagen is an executive producer at Mojang Studio in Stockholm working on Minecraft Dungeons. He previously worked on other MMOs including Anarchy Online and Battlefield 3. He got his start as a video editor in the marketing industry which lead to his career in LiveOps Gaming.
David and Crystin discuss the difference between satisfying your players and attracting them to your game and how you can embrace the idea of a co-creative process with your player base by listening to your community and building relationships with them.
David also touches on how to establish a human-centered approach to your game from the beginning and to create a relaxed culture with your team. David also offers some valuable advice on tooling for your content management systems and the importance of stress testing your servers.
This episode is brought to you by Azure PlayFab.
Visit https://www.playfab.com for all your LiveOps needs.
David Nisshagen is an executive producer at Mojang Studio in Stockholm working on Minecraft Dungeons. He previously worked on other MMOs including Anarchy Online and Battlefield 3. He got his start as a video editor in the marketing industry which lead to his career in LiveOps Gaming.
David and Crystin discuss the difference between satisfying your players and attracting them to your game and how you can embrace the idea of a co-creative process with your player base by listening to your community and building relationships with them.
David also touches on how to establish a human-centered approach to your game from the beginning and to create a relaxed culture with your team. David also offers some valuable advice on tooling for your content management systems and the importance of stress testing your servers.
This episode is brought to you by Azure PlayFab.
Visit https://www.playfab.com for all your LiveOps needs.
00;00;05;01 - 00;00;06;16
James Gwertzman
Hello. I'm James Gwertzman.
00;00;06;16 - 00;00;09;24
Crystin Cox
I'm Crystin Cox. Welcome to The Art of LiveOps podcast.
00;00;14;10 - 00;00;21;23
Crystin Cox
Hello and welcome to the Art of LiveOps podcast. I'm your host, Crystin Cox. And today I'm going to be talking to David Nisshagen.
00;00;21;24 - 00;00;47;00
David Nisshagen
I'm an executive producer working out of Mojang in Stockholm. Mojang of Minecraft fame. But I've been in the gaming industry for closing on 20 years now. Always online. I started my career with MMOs. Worked on game called Anarchy Online. When MMO FPSs were all the rage, I moved to Sweden. I worked at DICE for Battlefield 3. From there on a wide range of online themed game studios.
00;00;47;00 - 00;00;49;02
David Nisshagen
And now I've been at Mojang the last three years.
00;00;49;04 - 00;01;11;16
Crystin Cox
He has a long history in LiveOps games stretching back to the early MMO days, and I'm excited to chat with him both about some of his background and the things that he's learned over the years, but also specifically about Minecraft Dungeons and the work they've done as a really, truly multi-platform LiveOps game. So let's dive in and start this conversation.
00;01;13;21 - 00;01;43;04
David Nisshagen
An interesting thing is that I spent my first decade while working in the marketing industry, so I was a video editor. I did commercials, music videos, and that has really helped me understand how important it is to do things that actually work, that have an effect. That combination of a bit of marketing background, a desire to make things that actually have an impact and some experience in gaming, that's why I so love the LiveOps aspect.
00;01;43;08 - 00;02;09;17
Crystin Cox
I think, yeah, you're hitting on something that's come up for me a lot over the years in LiveOps. When you're continually making content for a game and you're keeping it updated and you're trying to keep players engaged, I often feel like there's two really competing desires that you need to take care of. Like and I tend to like some of them up as like attract and then also satisfy.
00;02;10;02 - 00;02;27;05
Crystin Cox
So there's all this work you need to do to keep your players satisfied, which tends to be very much like focused on existing players and making sure they get what they want. It's very detailed. If you have a game that's all about like gear or something like that, you're, they're always like, what's the new gear and what does it do?
00;02;27;05 - 00;02;49;29
Crystin Cox
And it's very, very detailed. But you also have to take care of attracting people, whether they're people who are brand new or people who have lapsed. And that's a completely different set of things that might be interesting for achieving those goals and a different set of priorities. So I think having that that point of view can probably help quite a lot with that side of the equation.
00;02;50;09 - 00;03;17;10
David Nisshagen
Oh, absolutely. I think you're right on the money there. I think that so I've had the pleasure of working with a game called Minecraft Dungeons, and we set up to have a rather well maybe aggressive is not the term, but high cadence release pattern. We did lots of DLCs. We did lots of in-game events and things. The game wasn't quite architected for it, so it took quite a bit of effort to make it work.
00;03;17;10 - 00;03;38;07
David Nisshagen
But it did work and one of the things that we set up to do with these DLCs and the updates and the in-game events and things was to make sure to be able to have an answer to Is this marketable? Can we make this so people are interested? Because I think you bring that up. I mean, satisfying the player.
00;03;38;07 - 00;03;59;18
David Nisshagen
I love that phrasing sort of relates to retention as a KPI. Right. Happy, satisfied players, they stick around. But in today's reality, no matter how good you do the core game loops, no matter how fun it is, no matter how satisfied the players are, there is a new shiny coming up and they will abandon your great game and go there for a while.
00;03;59;28 - 00;04;21;28
David Nisshagen
So you need to get them back and that's the attract portion of it, or reacquisition or reactivation or whatever you want to call it. But then having that, how can we do this content that is so both hitting then the satisfaction of the current player base and then also getting back the players that left for something else.
00;04;22;05 - 00;04;38;22
Crystin Cox
It is, I think, something that a lot of teams struggle with because they really are too. I think there's this wonderful idea. We've both been in the industry long enough. I'm sure you've heard it. It's kind of died out now, but there used to be this really strong idea that like, well, if you make a great game, players will just come.
00;04;39;02 - 00;05;00;12
Crystin Cox
which I think we've mostly know that that's not true anymore. But it's a very tempting idea to say, Well, if you make something very satisfying, the rest of it will take care of itself so that you don't have to deal with these two totally different desires. Because unfortunately, a lot of times the things that really satisfy players are not the things that attract them.
00;05;00;12 - 00;05;18;04
Crystin Cox
They're actually two different sets of things. And you find yourself having to split and say, We have to do some things for these releases that are going to be attractive, even if we know they're not the exact same things that will then actually retain players and be very satisfying.
00;05;18;04 - 00;05;39;21
David Nisshagen
Exactly. No, I keep thinking of now that you mention it here, like if you do a game update, if you have a game that has long term progression or long persistence of some sort, right? What satisfies the current player base is more end game stuff, but it's more progression where they are today, which is probably at the end of current progression. So add more stuff there.
00;05;40;11 - 00;05;42;21
David Nisshagen
That won't help a new player at all.
00;05;42;24 - 00;05;43;05
Crystin Cox
Yep.
00;05;43;10 - 00;06;05;22
David Nisshagen
Improvements to the UX or the first time user experience that will help the new players. And then how can you package that in a nice bundle that becomes so attractive? So, you know, partners such as you know, Xbox or PlayStation or Nintendo or Steam would feel, Hey, this is really cool, we'd like to front this and show it so we can get new players in.
00;06;05;22 - 00;06;14;21
David Nisshagen
And trying to connect that whole cycle is... it's a challenge, but I think that it's the most interesting thing that's going on in the industry right now.
00;06;15;08 - 00;06;38;21
Crystin Cox
And I think it's worth asking, like even as a designer, I think it's really worth asking. Like when you're sitting down and making updates, it's something we talked about back when I was a Nexon. What is the headline for this update? Like what? What is it that's going to be interesting enough for people to talk about what we have to do, something that's going to actually generate interest.
00;06;39;14 - 00;06;58;09
Crystin Cox
You can't just put out a patch notes. That's you know, there are a lot of players who will love it if you just put a patch notes that are just like, we fix this thing you've been waiting for. We added this thing, this quality of life thing you know about, but that's very deep. What is it going to penetrate the like the cloud of noise out there in the world?
00;06;58;19 - 00;07;02;08
Crystin Cox
It's sort of a constant bit of work that has to be happening.
00;07;02;12 - 00;07;39;12
David Nisshagen
No, absolutely. And I think what strikes me now is that how important it is to have a relationship with your community. Right. And having... I can look at... I mean, I've been so fortunate in my career, I have always been able to work at companies that have a very warm and close relationship to their communities. I mean, I joined the game industry at Funcom in Norway at Anarchy Online because I was a super fan and I was so excited and I was on the forums all the time and I like deconstructed the game mechanics for, for whatever the class I was playing was.
00;07;39;12 - 00;08;03;23
David Nisshagen
And then it's like, you're wrong because it should be blah, blah, blah, okay, you're hired. And that's how I joined the industry as a super fan and that company and many other companies as well. If you have that listening to your community and you talk to them and you engage with them. Minecraft's a great example. We have a show called Minecraft Live every year.
00;08;03;24 - 00;08;28;06
David Nisshagen
It's like a show where we talk about things that have happened and things that will come in the future. And we, you know, start hinting or suggesting updates that'll come, you know, in eight, nine months, next year. And planting that seed in the community's mind, making them inspired, making them feel that they can participate in the journey of making that thing that feature, that piece of content.
00;08;29;04 - 00;08;43;13
David Nisshagen
Suddenly you have that. That works. And as a piece of acquisition, because they talk about it among themselves, it becomes viral in a natural way. And it's also obviously marketable. You know, people talk about it.
00;08;44;03 - 00;09;10;16
Crystin Cox
I mean, that's a part of marketing, right? Like that. That can I mean, you want if you're really operating a live game at a high level, you want that sort of organic marketing. But it's deceptive because there's, it takes a lot of work to generate the environment for that kind of organic marketing to happen. You're talking about these community building and like community relationships.
00;09;10;27 - 00;09;34;13
Crystin Cox
That takes a lot of effort. I'm actually curious, you've said like you've had, you've worked at a lot of places that had close and warm relationships with the community. What do you think the sort of steps are to getting there? Because especially right now, I think we're seeing out in the industry, developers sometimes struggle with their community and struggle with toxicity and abuse.
00;09;34;13 - 00;09;45;09
Crystin Cox
And then on the other side, communities being frustrated with developers who are like not open or they're disappointing them. So what do you think those elements are that led to those positive experiences?
00;09;45;16 - 00;10;07;29
David Nisshagen
I think that transparency is one. But as you say, that's a, that's such a sensitive thing. It can swing both ways, right? It can be great. And it can really invite into like a cesspool of toxicity. It can be horrible for developers. Right. But I think the key thing, the only thing that has this real massive impact is trust.
00;10;08;16 - 00;10;08;24
Crystin Cox
Mm hmm.
00;10;09;08 - 00;10;48;03
David Nisshagen
If you establish trust with your community, if you're consistent and honest, and you just keep doing that over time, that builds trust and trust then translates into this warm co-creative relationship. But it's also hard to build and very easy to break. You can you can break down trust so easily if you do things that are sneaky or surprising or unexpected cash grab or things that are perceived as negative. It really is like it breaks down the trust and then you get this whiplash back from the community, right?
00;10;48;03 - 00;10;51;01
David Nisshagen
They really turn into a negative.
00;10;51;04 - 00;11;12;21
Crystin Cox
I think for me, a lot of it ends up being a humanization on both sides. Right. I think a lot about, there's actually a ton of work that's been done in especially the last five or six years on how to help people make friends online. Right. How to actually in games or out of games.
00;11;12;29 - 00;11;29;19
Crystin Cox
How do you. Okay, we all know that players who play with friends, they stay around longer. So then we started thinking in our designer brains like how can we like get people to become friends? And there is a lot of thought about that, right? There's a lot of psychology research and there's a lot of thought about how it's done online.
00;11;29;28 - 00;11;51;13
Crystin Cox
And it does come down to trust is this huge part of it, right. Like and how do you build that trust? A lot of it is disclosure, right? You know, being honest with each other, sharing things that are important and meaningful. Reciprocity. Right. Giving and receiving of things. I actually more and more tend to think about that as like a way to set up a relationship with the community.
00;11;51;13 - 00;12;10;12
Crystin Cox
Like, how can I make this like I would develop trust with anyone? Can I humanize it in the hopes that it creates those feelings of humanization across both sides? Now, of course, there's going to be people in the community who are just there are they have an agenda. They're not looking to be friends. Right. They're looking to come in and be disruptive.
00;12;10;21 - 00;12;28;14
Crystin Cox
And there's I think you do have to you know, you can't just tolerate that. You know, you have to set firm ground rules. But I think you can get more support from the rest of your community if they see you, even if there is no specific person behind it is the developer as a collection of humans?
00;12;28;19 - 00;12;50;19
David Nisshagen
Yes. Just like anything from like, you know, a social media account, a Twitter handle or something, you sign that with a couple of initials rather than just having Here's the giant brand talking to you, or here's the initials of the person that's running the social media account. It humanizes it. Yeah, I think I think you're absolutely right there.
00;12;50;27 - 00;13;13;15
David Nisshagen
But because that builds trust both in a sort of predictable way that you know you can trust the dev team or the publisher, that they'll do what they say they'll do. And you'll trust that they'll communicate when they say they will communicate. But you can also... I mean, we're humans. We trust other humans. It's like I don't I don't trust chat bots.
00;13;13;15 - 00;13;29;21
David Nisshagen
Right. It doesn't it doesn't work. I try a little bit too, but I have a better time just searching the FAQ on the bank side or on a game site or rather and when I really want to have when I really want to have help, when I really want to have give feedback, I give them a call or I go to... I visit them.
00;13;30;09 - 00;13;32;25
David Nisshagen
I get a human to human interaction. No, you're right.
00;13;32;29 - 00;13;55;01
Crystin Cox
Yeah. And it is tough. I mean, it's a scaling problem for sure. But I do think when I see sort of communities that are functioning so well, they do tend to have more of that. I like the way you said, it's a warm feeling that exists sort of between the developers and the community. You know, it's, you're never going to get rid of...like I said
00;13;55;03 - 00;14;03;12
Crystin Cox
There's always going to be some disruptive element. There's an anti-social element right out there and everything, but I think it's definitely work that's worth doing.
00;14;03;21 - 00;14;04;17
David Nisshagen
Oh, absolutely.
00;14;04;17 - 00;14;19;09
Crystin Cox
I was actually going to roll back to something you said earlier. You said at the beginning when you talked a little bit about Minecraft Dungeons that you guys did a lot of updates, but the game wasn't exactly architected for it.
00;14;19;09 - 00;14;19;18
David Nisshagen
Yes.
00;14;19;18 - 00;14;36;23
Crystin Cox
This is, I think, a common issue for LiveOps games. I'd love to hear you talk a little bit more about what it was, like what are those edges you ran into? If there's like lessons that you guys have learned that you might take forward on how to do a better job of sort of content architecture?
00;14;37;03 - 00;15;01;12
David Nisshagen
Oh yeah. I guess Dungeons is to some extent then a victim of its own success. I mean, it was intended as a small scale fire and forget game and then it turned out to be really, really, really fun, really fun. And every like user research test we did and every, every time we put it into the hands of players, there was smiles and happiness.
00;15;02;03 - 00;15;21;10
David Nisshagen
It's like, okay, more, can we do more? Can we make it more platforms? Can we increase the scope, etc.? And but nothing was fundamentally architected for it. It was designed originally to be I mean, it started out from a design perspective to run on a Nintendo 3DS.
00;15;21;22 - 00;15;22;03
Crystin Cox
Right.
00;15;22;10 - 00;15;36;16
David Nisshagen
Right. And then it's like now it's a multiplatform crossplay cross-platform progression beast, right? But the fundamental architecture was like designed to be played offline. You should be able to play the game offline.
00;15;36;25 - 00;15;37;05
Crystin Cox
Right.
00;15;37;16 - 00;15;59;09
David Nisshagen
And that's hard to combine with LiveOps because, hey, if you play offline and you want to have some sort of progression, you trust the client and then your modded or hacked or whatever and the progression, you can just bypass it all. So that's the fundamental thing. Just make sure that you architect things for being online.
00;15;59;12 - 00;15;59;25
Crystin Cox
Yeah.
00;16;00;16 - 00;16;09;07
David Nisshagen
Yeah. It's a no brainer. Right. But. But you still need to make sure it's stated somewhere when you do the architectural work.
00;16;09;10 - 00;16;33;05
Crystin Cox
Yeah, that's a good point. I mean, like you say, it's a no brainer, but replication is something that I think a lot of game teams really struggle with. You can ship a lot of games on mobile, you ship a lot of games on console and never have to deal with, Wait how do I actually verify or sync up data between a server and a client?
00;16;33;05 - 00;16;53;29
Crystin Cox
Like, how do I actually do that? It's not always, I think, super straightforward for teams. And as you said, if you start from the assumption that you're not going to do that, right. That the game is going to have a lot of information stored in the client, a lot of. And the client is never going to have to, you know, validate itself or speak to itself.
00;16;54;10 - 00;17;12;13
Crystin Cox
Changing that can be quite hard since actually I'm actually even a little surprised. I did not realize that the Dungeons started from a place where you know, it was sort of maybe an earlier handheld sort of idea because now it is so much a complete like online multi-platform game.
00;17;12;25 - 00;17;36;02
David Nisshagen
I know. And there's like we've had so much, so much great talent in the team and help we've gotten to take it to where it is now. But it's... we did have a lot of conversations and thinking about like, hey, how about PvP? Let's do PvP. It's fun. We know it works. It's, it's great for it's very marketable, great for acquisition right. Great for durality.
00;17;36;13 - 00;18;04;12
David Nisshagen
But then it's like, how, how can we do this in a safe and secure way? How can we do PvP that players can trust? And the answer is we can't with the current architecture, we can't do it because we'd end up with like say that 99.9% of players would play and be happy. But there's always just like with the community, people who are not that like people who want to disrupt, players who're not playing nice to some extent.
00;18;05;00 - 00;18;17;25
David Nisshagen
And if we have vulnerabilities or like loopholes in how client data is stored, that would then lead to, I think our network engineers called it like a poisoned well.
00;18;18;05 - 00;18;18;16
Crystin Cox
Yeah.
00;18;18;19 - 00;18;33;18
David Nisshagen
So you just have one source of poison. I guess it just breaks the matchmaking for PvP or makes it a really horrible experience and then that just that poison spreads. And you can't do anything about it unless you architect for it from the start.
00;18;33;25 - 00;18;34;06
Crystin Cox
Right.
00;18;34;12 - 00;18;44;09
David Nisshagen
So, so that's, that's one of the things that I think is important. Another thing is also to be able to tweak and change content. Runtime on the fly.
00;18;44;19 - 00;18;44;27
Crystin Cox
Mm hmm.
00;18;44;29 - 00;19;06;29
David Nisshagen
Like Dungeons is a great example of, like, we hardcoded. Not hardcoded. It's a beautiful little config files, but still, it's in the game client. Yeah. How much damage does this sword do? What's the range of this bow? And if we make a mistake, which sometimes happens very rarely, I'd say we're were very fortunate with talent again. But sometimes things, things break or things change.
00;19;07;13 - 00;19;17;05
David Nisshagen
And the only way to, you know, bring down this incredibly overpowered weapon or fix the thing, the broken thing is to do a game update.
00;19;17;14 - 00;19;17;25
Crystin Cox
Yeah.
00;19;18;15 - 00;19;44;16
David Nisshagen
And that's like who? We're a multi-platform game that means lot check on the switch and XRs on Xbox on the Windows Store and Steam and PlayStation. Ugh, that's a significant effort, significant cost for just this. It can't just be for a broken weapon. It has to be a package of things that's worth the cost of the update to some extent.
00;19;44;18 - 00;20;02;20
David Nisshagen
And that's I so wish that we had architected that earlier that we were able to like tweak and turn things on and off on the fly. It's also like a no brainer. But it's good to call it out that please do consider that. Make these runtimes tweakable.
00;20;02;29 - 00;20;22;04
Crystin Cox
Yeah. It's and these days there's a lot of options for that, right. Because we have, there's all these clouds that you can connect to. And so it's a little bit easier. I definitely think we both remember the time when MMOs were Yeah. Had lots of config files, but they are stored on the physical servers that you run. Right.
00;20;22;04 - 00;20;46;19
Crystin Cox
Like so easy to change but to set that up required quite a lot of work. Like you're like we got to do all kinds of things in the early days to get ready to distribute this. Nowadays you absolutely can. Like even if you're a fairly small service, you can store a lot of configurable information in a secure cloud and then, you know, grab it and upload it at any time, which is really great.
00;20;46;29 - 00;20;56;04
Crystin Cox
But if you don't, even now, that is easier if you as you said, if you don't expose any of that, if you don't set up that pipeline, it's really tough.
00;20;56;12 - 00;21;25;00
David Nisshagen
Yeah. I mean, that's one of the things we did with the Minecraft Dungeons as well is like to give a couple of the... we have a couple of quite cool actually in game rotating like daily challenges and in-game events and things. We basically took content management system that was designed to do completely different things and I'd say we abused that into something that works-ish. It works.
00;21;25;01 - 00;21;46;09
David Nisshagen
Yeah, works. Obviously it works. But it's like, oh, when you look at it's like, this is not beautiful. This is not how we should have done it. Tooling making sure you have the tooling, making sure you have the pipelines and making things online and runtime tweak-able and you can exhale. Then it'll reduce so much stress, right?
00;21;46;21 - 00;22;22;14
Crystin Cox
Content management systems are an interesting topic. It's one of those spaces where, you know, back in the like pre cloud days, pre sort of consoles and stuff, being online days like through early or late nineties and early 2000s MMOs, you just made your own content management system, right? Like I've worked with a bunch of homegrown content management systems, but it's still one of those things where I'm surprised that no one has stepped in and made a just sort of out of the box content management system that really works for this kind of thing because it's still a struggle.
00;22;22;14 - 00;22;35;05
Crystin Cox
Like, I feel like every for every game it's still a struggle. And you're still as you said, taking something that was either taking something that was designed to do something completely different and then just sort of, you know, shoving it... square peg into a round hole.
00;22;35;05 - 00;22;35;16
David Nisshagen
Yeah.
00;22;35;16 - 00;22;38;12
Crystin Cox
Or your back to building your own from scratch.
00;22;38;20 - 00;22;56;17
David Nisshagen
No, I know. And it's so I mean, it's a big industry and there's a lot of games now, a lot of LiveOps games, a lot of games as a service. I know all of the games have their own unique properties. Obviously they do, but they should be able to be a bit more generalized, I think.
00;22;56;21 - 00;22;57;07
Crystin Cox
Yeah.
00;22;57;07 - 00;23;26;11
David Nisshagen
And it's like I'd much rather have super talented devs building fun game experiences than building a CMS, right? Whatever tooling there is that you can lean into. Yeah. I'd love to encourage people to look at existing tools, things that others have built, so you can focus on building the game rather than just, you know, trying to recreate what's already there.
00;23;26;22 - 00;23;55;22
Crystin Cox
Yeah, I mean, there's something to be said for building your own right. Like, obviously you would get a very it gets very, very tailored to what you're doing. But yeah I'm often struck by... I sometimes think I wonder how much time teams I've worked with have spent making the tool that just puts, you know, announcements into the game or just puts calendars into games or like, how much?
00;23;55;22 - 00;24;00;17
Crystin Cox
If I added it all up across like every team, it would be a massive amount of time.
00;24;00;18 - 00;24;25;06
David Nisshagen
Yes, it would be ridiculous. Right. But yeah, but then again, every game is different. Some use JSON, some use other things. It's like whatever the config files are. So I don't know. It's interesting to see there is at least I haven't seen any form of standardization of how the practices are fairly well established now are fairly common and somewhat predictable. You have your schedules.
00;24;25;16 - 00;24;28;26
David Nisshagen
You have your calendars, you have your in-game events and they are.
00;24;29;17 - 00;24;49;15
David Nisshagen
We're sort of starting to have a little bit of a library where you can pick things from the shelf and arrange something that is a pretty okay LiveOps experience. And then you do some custom innovation magic on top of that. But when it comes to the technical how it's all over the place, right? Everyone rolls their own solutions.
00;24;49;23 - 00;25;08;14
Crystin Cox
The tooling is so hard because even like, you know, so much advancement has happened in services, right? Like, I mean, PlayFab, listen, like this is one of our technologies is great like that's an amazing back end. Great you don't have to build your back end like you could do all this stuff. But the tooling is still yeah, there's still so much more.
00;25;08;14 - 00;25;30;23
Crystin Cox
That is, I think, an opportunity there for having more shared tooling across games. But it's I mean, there's been some work too, like Epic has done some work for sure. I think we're starting to see some more of that stuff, some more of that stuff get into Unreal. But yeah, still a long way away from there just being sort of like, Oh, hey, like you need to build an in-game store?
00;25;31;03 - 00;25;42;00
Crystin Cox
Here it is. It's got a good standard of display and highlighting and like searchability and scheduling and all of these things
00;25;42;00 - 00;25;56;16
Crystin Cox
It hurts so much when you say that. It's like, Ouch. I built a couple right. Well not myself. I've been part of several teams that have done it and it's like we know what it should look like. It's fairly standard, right? And you build a new one every single time.
00;25;56;24 - 00;26;22;07
Crystin Cox
Yes. Yeah. And I mean, the connections to the game are usually pretty specific, right? Like, for sure. But yeah, it is rough. Like, I think I've actually sort of sat down with the Minecraft team before and seen they're very robust at this point, you know, homegrown grown content management system for their store. And it's like, yep, there it is again.
00;26;22;14 - 00;26;43;17
Crystin Cox
And you can't and it's funny because in some ways it's gotten a lot more sophisticated and in other ways, I think back about doing it, you know, back in the day, doing it with, you know, .NET sort of web based stuff. And it's not, you know, in a lot of ways, it's basically the same stuff still.
00;26;43;29 - 00;27;09;08
David Nisshagen
Yes. No, I mean, the user experience, that's where it all lands. The technology is to some extent shouldn't be visible. It should just disappear. Right. It's the user experience. What is the player? How can we show exciting things to the player? That's what we want to do and present that in such a way that it's easy for them if they choose to pay for something to do so
00;27;09;08 - 00;27;09;29
Crystin Cox
For sure.
00;27;09;29 - 00;27;36;17
David Nisshagen
And then that hasn't really fundamentally changed. But the underlying technology for sure has many times and will continue to do so. Mostly for good, I suppose. I think it's a lot safer these days, a lot more secure, especially if you use like existing large scale services service providers. So you don't need to reinvent and create your own.
00;27;37;01 - 00;27;56;08
David Nisshagen
It's like this. There's this horror story from when I started my gaming career there with Anarchy Online. There's this guy who's going to build the subscription system. Yeah. And he they checked. And how are you doing? How are you doing? Because the devs were building the game, right. So they found this, this other person who can help them with that.
00;27;57;08 - 00;28;00;04
David Nisshagen
It's going great. I'll be done soon.
00;28;00;04 - 00;28;00;17
Crystin Cox
Uh huh.
00;28;00;18 - 00;28;07;25
David Nisshagen
Yeah. And then that just kept on going. And as they approached the release day, there was nothing.
00;28;08;18 - 00;28;09;28
Crystin Cox
ooh.
00;28;09;28 - 00;28;27;28
David Nisshagen
It didn't work. And it's like we're releasing soon and we, we were releasing a subscription based game and we don't have the subscription system in place. So panic, house on fire and there were a couple of bugs in the game as well. But they moved people over to fix it and get it all in. But I'm glad we're not there anymore.
00;28;27;28 - 00;28;39;20
David Nisshagen
Where we custom build these sort of payment solutions and that's sort of foundational structures. There i's so good to be able to lean into to strong, proven service providers for that.
00;28;39;23 - 00;28;48;25
Crystin Cox
Yes. Shout out to everyone who's ever builds their own payment system. That's hard and I'm glad I'll probably never have to do it again.
00;28;49;08 - 00;29;05;03
David Nisshagen
Yes. No, I mean, it's amazing. And sometimes you can pending on your product. I mean, you can reduce friction. You can make it so that it's you have a better user experience. It's more easy to for players to buy things they want, and that's great.
00;29;05;13 - 00;29;05;22
Crystin Cox
Mm hmm.
00;29;06;06 - 00;29;13;05
David Nisshagen
But. But, yeah, at the cost of then development effort and of, you know, risk. Risk of security.
00;29;13;13 - 00;29;44;23
Crystin Cox
Yeah. I mean, it's one of those things, too, that working in MMOs and especially the free to play space early earlier on you know like in the early 2000s definitely got exposed to the more extreme security issues and fraud issues that, you know, I think in some ways we've gotten so much better, right? Like there's so much more security now and so much more trust that can be built with players.
00;29;44;23 - 00;30;16;20
Crystin Cox
But it was kind of interesting having been through some of that, to then watch the rest of the Internet actually catch up to it. You start seeing stuff that you saw, you know, early 2000s happen to larger corporations in like the mid the in like the teens in the twenty tens this kind of interesting you know but I do think we've come a long way and it is you know, it is much more secure, although I think games are still kind of tip of the spear in a lot of ways.
00;30;16;20 - 00;30;20;10
Crystin Cox
You get a lot of attacks. We always have to be thinking about it.
00;30;21;04 - 00;30;45;10
David Nisshagen
I think that goes back to the community aspect that we are... the games industry have such a passionate fan base, such a passionate community. I can't quite think of anything that's as passionate. Well, sports obviously. Yeah, but if you're talking about business to consumer product or business to consumer service in some form, I don't think anything comes close.
00;30;45;24 - 00;31;10;05
David Nisshagen
I mean, maybe movies, Marvel superfans, maybe they become as excited and as passionate as gamers do for their favorite game, but not at that same scale and not. Yeah, it's really quite fascinating to be in this, as you say, tip of the spear where we create so much fun experiences and that we're able to touch people.
00;31;10;16 - 00;31;19;16
David Nisshagen
And even deeply, emotionally they genuinely care. That's wonderful. And just trying to mold that into something that is positive.
00;31;19;21 - 00;31;46;29
Crystin Cox
Yeah. I mean, I think you're hitting on something I think about quite a lot, which is when players invest in a game, they actively are participating in it and they put a lot of themselves into it. It can be tough sometimes and I absolutely have felt it too, when it can feel like your players might be quite demanding and you know, they don't think about your needs.
00;31;46;29 - 00;32;08;01
Crystin Cox
They they're always so worried. But if you think about the fact that you've made something that is now part of their identity, like this game is part of who they are, is part of their social experience, it's part of who they are as a person. And they're trusting you to take care of that. Right. And they have very little power like they are just sort of trusting that you're going to do right by them.
00;32;08;17 - 00;32;34;27
Crystin Cox
It can be a little bit more understanding of why sometimes they're a little they're maybe a little demanding or entitled or freaked out about things. But it's true like that deep connection is I do think it's special. I think it's very special in games because when you invest in a game, you are actively investing in it. You're not watching something, you're not consuming it, you're actively investing.
00;32;35;03 - 00;32;48;23
David Nisshagen
Oh definitely, definitely. And there's this part as well of co-creation. I think, that players often feel or want to feel that they are creating the game experience together with the developers.
00;32;49;04 - 00;32;49;28
Crystin Cox
Sure. Yeah.
00;32;50;10 - 00;33;21;26
David Nisshagen
And that's super powerful. I mean, it becomes again, looking back at MMOs or other online games, some of the most exciting ideas or suggestions actually originated from the community from this. Wouldn't it be cool if we could do this? Because again, I mean, they invest so much passion into the games, they often know it better than we do as developers, right?
00;33;22;10 - 00;33;23;08
Crystin Cox
Oh, for sure.
00;33;23;12 - 00;33;38;07
David Nisshagen
Yeah. Right. So there... we sort of create it and they consume it and I don't know, deconstruct it and internalize it in such a way that this is what you should do and you can look at that suggestion and say, Yeah, that's actually a great idea.
00;33;38;19 - 00;33;41;06
Crystin Cox
Yeah, well, they always outnumber us, I like to say.
00;33;41;06 - 00;33;41;16
David Nisshagen
Oh.
00;33;41;26 - 00;34;10;08
Crystin Cox
We're always outnumbered. You know. So it's true. Like, they're like they are going to know more than you in a lot of ways because you'll never compete with their sheer numbers, so you shouldn't try, you know. Nothing is more beautiful than an MMO wiki in my mind. And I think sometimes we would be embarrassed to say like if players knew how much we go to the wiki to find out.
00;34;10;15 - 00;34;14;02
Crystin Cox
Like wait, what? When did this get put in? What does this actually do?
00;34;14;02 - 00;34;26;20
David Nisshagen
I know, and it's so consistent as well. Right? It's like every time is how much is the internal documentation or the fan created documentation which one is better and more up to date? It's gonna be the fan.
00;34;26;21 - 00;34;47;13
Crystin Cox
Probably the fan documentation. Yeah, they have the numbers they got, they have a big advantage. I know that we're rolling into about a little bit of time left here, so I want to make sure I ask your favorite question. You kind of hinted at one, but when you talked about almost trying to launch Anarchy Online with no subscription service.
00;34;47;27 - 00;34;52;12
Crystin Cox
But I still ask, do you have a LiveOps disaster that you can share with us?
00;34;52;18 - 00;35;01;12
David Nisshagen
Oh, boy, do we have LiveOps disaster? Now, actually, I'll be very honest, I'll. I've been in the very fortunate position of having near disasters.
00;35;01;23 - 00;35;02;01
Crystin Cox
Mm hmm.
00;35;02;20 - 00;35;24;22
David Nisshagen
So super talented people that have seen this disaster and mitigated it or with the possibility that, hey, the thing that we were supposed to release now. Oh, my dear, it's not going to work. That's the beauty of LiveOps, right? You can just do it in the next release. You can do the next update and can move the timelines a bit so that you can reduce that pressure.
00;35;25;07 - 00;35;49;22
David Nisshagen
But there was there was one thing I was thinking of. I was working at a wonderful studio called Paradox Interactive back in my design days, so I was working on a free to play four versus four MOBA called Magicka Wizard Wars. It was amazing. It was so much fun. It had no monetization whatsoever, but it was a great, fun game, right?
00;35;50;29 - 00;36;26;01
David Nisshagen
It became very popular at launch, above expectations. Right. And again, we didn't quite architecture it for lots of players. So we had a bunch of login servers and I don't know if they capped out at 255 per login server, some sort of arbitrary technical decision done way back in the day. And all these players that wanted to play the game, they joined in and then one of the log in servers fell down and moved all the 250 people to the next.
00;36;26;01 - 00;36;38;29
David Nisshagen
That fell down. That fell down. That fell down. So just giant domino of login servers. You can just see we had a dashboard and you can just see like things go green, yellow, yellow, red, red, red, red, red, red...
00;36;38;29 - 00;36;39;19
Crystin Cox
Oh no.
00;36;39;20 - 00;37;04;10
David Nisshagen
Oh, and I, for some reason, I don't know. I used to play percussion when I was younger. Right? So for some reason, I. I had a knack for timing how and when to restart these login servers so you can actually try to counter it. So I was sitting there in the evenings, you know, not playing the game, but playing the login servers start trying to stabilize the game.
00;37;05;03 - 00;37;05;29
Crystin Cox
That's amazing.
00;37;07;09 - 00;37;12;00
David Nisshagen
Yeah, memorable experience. And the take away is do stress test your servers?
00;37;13;08 - 00;37;33;20
Crystin Cox
Yes, yes. Do please do stress test your servers. It's a common one, right? Like it's a common problem of how do you deal with load when it comes in an unexpected way. And I think your experience might be more common than people realize and how much manual intervention.
00;37;34;00 - 00;37;34;10
David Nisshagen
Yeah.
00;37;34;19 - 00;37;42;16
Crystin Cox
May have to be maybe happening in order to just try to keep a service that's falling over functional.
00;37;42;17 - 00;37;50;12
David Nisshagen
Yeah, yeah, yeah, absolutely. Like, yeah, there's load balancers. Yeah. Sometimes the load balancer is just like a very tired dude clicking buttons, you know?
00;37;50;21 - 00;38;17;01
Crystin Cox
Absolutely. I mean, I will say I have my own experience with at one point having to run a team on a 24 hour shift for a while because something needed to be manually cleared on a regular basis to make a very important service work while another group of people frantically tried to rewrite it so that it would stop requiring this manual attention.
00;38;17;01 - 00;38;25;12
Crystin Cox
And I do I think it's definitely a bit a invisible situation that happens more than people probably would like to believe.
00;38;26;02 - 00;38;36;04
David Nisshagen
And I think that if I tried to tie that back to the community aspect again, I mean, the players are so passionate about our games and I think that is only matched by the passion of the dev teams.
00;38;36;13 - 00;38;36;26
Crystin Cox
Yes.
00;38;37;24 - 00;39;10;17
David Nisshagen
There is so much effort, so much pure passion that goes into creating these experiences and devs are sitting yeah, really working hard, being very clever and also just putting in effort. That is almost mind boggling at times. Right. And that's okay if it goes for a little while and then you have time to recover, that's fine. But one of the things that I have seen in LiveOps environments is that you never get the recovery time.
00;39;10;20 - 00;39;22;07
David Nisshagen
You end up on this. You sprint and then the sprint turns into a marathon. You need to understand that it's a marathon from the get go and add that in there, but the passion of.
00;39;22;14 - 00;39;23;06
Crystin Cox
Because you have to pace.
00;39;23;06 - 00;39;48;25
David Nisshagen
Yes but you need to pace yourself. You need to even if the things done were very close to done, doesn't mean you have to ship it right now. I know the players would love it. I also love to see the things in the hands of the players, but it can wait a couple of weeks. It is fine. And that's, from my perspective, one of the most beautiful things about the sort of games as a service aspect or regular updates that you can.
00;39;49;18 - 00;40;13;28
David Nisshagen
It's not quite done. It'll come soon. And that, that helps de-stress the dev team. It helps set expectations and if you're then honest to the community of other players and just say hey it's not quite done yet and we're not going to have our developers working weekend. So you'll get it when it's done. And when you have that relationship with your players, they're like, okay, that sounds great. We trust you.
00;40;13;28 - 00;40;40;19
Crystin Cox
Yeah. And even though you might have a couple of people say, Get in there on the weekend and do it, you then also tend to get the rest of the community saying, Whoa, don't be like that. Come on, it's like, let's be nice to these people. You know, it's funny. Like, you know, sometimes we think about giving people advice for like doing LiveOps and they think about like sophisticated machine learning algorithms or like really like amazing, you know, data analysis stuff.
00;40;40;19 - 00;41;01;08
Crystin Cox
And a great technology. And a lot of the best advice is don't ship on Friday, don't ship after the first week of December. Like, you know, like this really basic. But a lot of it really is about team management and making sure your team is doing okay because you don't want to wear them all the way out.
00;41;01;26 - 00;41;27;04
David Nisshagen
If you have an invested development team that that feel empowered, they will save these disasters. They will stop them... will prevent these disasters from happening. And they will happen without control. They will just appear and you got to have your team or it's such a good thing to have your team happy and eager to take those challenges on and solve them with a smile and take a couple of days off.
00;41;27;04 - 00;41;36;00
David Nisshagen
Yeah. And celebrate. That's sort of if you can set up that sort of culture in your team, that is wonderful, that is super powerful.
00;41;36;00 - 00;41;51;16
Crystin Cox
That is that is some great advice. You know, to me, it's not just technology, it's not just design. It's also team management. And that's really important and it's a great place for us to close up then. Thank you so much for coming on. This was wonderful.
00;41;51;19 - 00;41;57;08
David Nisshagen
Thank you for having me. Such a pleasure.
00;41;57;08 - 00;41;59;22
Crystin Cox
Thanks for listening to The Art of LiveOps podcast.
00;41;59;22 - 00;42;05;01
James Gwertzman
If you liked what you heard remember to rate, review, and subscribe so others can find us
00;42;05;01 - 00;42;09;19
Crystin Cox
and visit PlayFab.com for more information on solutions for all your LiveOps needs.
00;42;09;28 - 00;42;12;22
James Gwertzman
Thanks for tuning in.