Mike Pagano, bringer of doom, wants you to know that if you’re not focusing on this, you’re gonna fail. What is it? Find out in this, our second episode of The Art of LiveOps. Mike Pagano, an early pioneer in the LiveOps mobile space, shares with us some lessons about LiveOps, as well as diving into how and why it developed in the first place.
Mike Pagano, bringer of doom, wants you to know that if you’re not focusing on this, you’re gonna fail. What is it? Find out in this, our second episode of The Art of LiveOps. Mike Pagano, an early pioneer in the LiveOps mobile space, shares with us some lessons about LiveOps, as well as diving into how and why it developed in the first place.
Hello, I'm James Gwertzman. I'm Crystin Cox. Welcome to the Art of LiveOps podcast.
Speaker 2:[inaudible].
Crystin Cox:I've been making games for 16 years as a designer and a game director focused on live ops.
James Gwertzman:I founded PlayFab five years ago because I saw a huge gap in the kinds of access to live ops technologies game studios needed to be successful. We've put this podcast together because a lot of the information on how to do live ops effectively just isn't out there. There's way more information about how to optimize your graphics pipeline or how to put together effective game design than there is on how to do effective live ops.
Crystin Cox:And since it's pretty tough to go around yourself and just find everyone that does live ops and ask them questions. We did it for you.
James Gwertzman:We've picked together a list of some of the top practitioners in the industry, folks who we think are really pushing the boundaries and doing some cutting edge work.
Crystin Cox:And we interview a different one every week and ask them about their experiences running live games, doing live ops and even having live ops disasters.
James Gwertzman:Yeah, train wrecks are the best. That's where you really learn how to do things effectively.
Crystin Cox:So tune in. We have another interview for you today.
James Gwertzman:On with the show.
Speaker 3:[inaudible]
Speaker 1:Hi James. Hey Crystin. We have a really good guest today.
Mike Pagano:Hey, I'm Mike Pagano doom. A lot of people know me as Pagano.
James Gwertzman:Or crazy Mike. Uh, I've known Mike for years. You and I first crossed paths at Electronic Cards.
Mike Pagano:We did. That was in, uh, in China.
James Gwertzman:I think he has the record for the single highest ROI game EA ever produced. It was a version of scrabble for the kindle and I think it was made for like a few thousand dollars and it went on to do millions of dollars of revenue. So I'd say it's like a crazy He's got a really crazy track record of doing really interesting projects.
Mike Pagano:I'm one of the original free to play guys in mobile
James Gwertzman:And so he's got some early lessons to share and I think he's also gonna talk about, uh, his, his, his new studio now and some of the new projects they'd been working on.
Crystin Cox:Yeah, it'll be really great to get some of that perspective. You know, he's worked very internationally, so he's worked both in Asia and he's also worked in the west. So I think it'll be really great to hear what he has to say about how live ops is maybe a different in different parts of the world.
James Gwertzman:He also, for a long time within EA, he was sort of the, the one of the wandering gurus when ea was first getting introduced to live ops broadly. And so he has a lot of experience kind of going around and teaching traditional studios how to incorporate a lot of the ideas in their games.
Crystin Cox:Awesome. Let's dive right in.
Speaker 3:[inaudible]
James Gwertzman:I have to say I credit you back then I was at popcap games. POPCAP was sort of making the transition over to free to play, you know, China, we were already kinda there in North America. We're just getting started. Um, we both ended up kind of becoming because of that experience in China live experts for the company. And you in particular, you really taught me a lot of what I know about live ops. And there was a time when you were traveling around all of the EA is sort of like the, the, the live ops. Yup. And I'd love to hear you talk about that time a little bit in in particular, what were you pedaling and what was the reaction from the studios and talking to.
Mike Pagano:Well, why don't we go on the way back machine and let's talk about, um, where I was and where the industry was. So this was 2011 of smartphone was the new thing. Oh Wow. Gaming on smartphone. And then there was a couple of games that really took the west by storm, like the world by storm. It was, um, uh, smurfs village member, that one. Uh, and at the time it was Zombie farm. So there was two farm games that were really drafting off of, uh, intelligence from the social game world. So social games, these are games on Facebook were really getting big, really getting big. EA acquired playfish there's two big players at the time. Playfish and Zynga, we acquired Playfish. And so I was in the EA mobile group in Los Angeles and we were talking about how do we learn free to play, where, what do we, what do we do to get ahead of that? Because we were about what does it, so smurf village had been out for about a year or two at that point and was really doing well, which was just dominating the charts. So it was my job to found a studio that was free to play and learn it and then teach the rest of the organization. I was in a group called future tech, which we would explore new platforms. So I was the guy, the shock troop, right, to go off and go find and figure it out. So at the time I was in Los Angeles. And how do you create new away from a large organization or in a large organization? I tried to do it away from the org. I'm try to keep away from the eye of Sauron so I'm able to take the time to design, to create. And so what I did was, um, we, we, we discussed where the studio should be. We explored South America, we explore Japan, we explored a few other places and where we really landed to China. And so it was a Beijing. I'd never been, never been right. Uh, now[chinese]. Right. That's going to sound terrible. I'm sorry. My Chinese listeners. Um, uh, I've, so we, I moved myself off to Beijing to go learn free to play. And so I created a game from scratch called Ghost harvest, which was, um, EAs, probably one of their most colorful games, I have to say, because you're a harvesting souls for money. Very Classic Classic, right? And so I created this group called eight pound gorilla. So eight pound gorilla was a play off of the 800 pound gorilla, which is what ea was. So we're a fast, nimble, sharp dynamic, right? It's Super Fun. And so we staffed up the team largely of um, uh, local talent. So it was all Chinese team and then me and we sat next to playfish. So playfish had a giant team out there, um, who had been doing social gaming and so they're like, Mike, go figure it out. So I sat up myself next to them and just asked why. So I come from a traditional gaming background. Um, I am educated at Digi Penn Institute of Technology, a wonderful, wonderful school. So I have a degree in real time, interactive simulation, oohh, video games. So I love the craft and I'm always about design first. Make, make a fun game and then you can figure everything else out. So I went to China, um, inside of Beijing with playfish and constantly asked why, why is this fun? Why, what's good about this? Um, and a lot of it came back with answers that we don't know why this works. Oh, this is interesting. So me as a scientist likes to come in and really figure it out. So what I did was I started from the ground up, built a game, very first free to play game for EA, right? Um, uh, at least mobile at that time, if I can remember correctly. Um, and really taught myself how to be, what is a pm, what is analytics? Right? Analytics didn't exist in our organization at the time. We helped to build it. So we learned a lot of really fundamental things of how do you, what is live ops? What do you do to do it? Like what's, how do you get from point A to point B? What's customer service? Right? Like I said, we were, we were looking through all of these different facets of live ops and really creating it. And so what we did was we created a presentation well I created several presentations, but, and then what I would do is I would go from studio to studio and go, this is free to play. It's not mysterious, right? Not at all. It's a service. You're here to think of it like you're a band. You're here to continually make cool stuff that your consumer wants over and over and over, your direct to consumer. And the real, real interesting thing that way I was really pushing was, oh my gosh, and we get to do it a real time. Whoa, wait, I can then get a thousand a thousand cohorts in that instance. I'm sorry. So cohort, that's a, it's a group of people that, uh, that come in and install on a certain day. Oh my gosh, I can look to see what are they doing, where, how long are they there for? And so I can really start to understand the consumer behavior. Oh my gosh, that is awesome. So I designed this thing that I can see if they actually like it through the time that they had invested in that one specific area. So from there I basically created these presentations and ran around the EA and said, hey, this is how you do live ops. These are some really core things with data. Here are the things that actually work, right? This will increase. What's our, what's ARPU cause ARPDAU that was a Zynga term. We used ARPU, right? So what's your art poon what's your, ARPUPU right? So we designing what are the KPIs, how do you achieve that and why? And that's how I met James.
Speaker 4:Um, uh, we were, we were creating just lots of free to play in just going through, uh, initially the mobile org. And so that was a popcap at the time. Um, and so we became partners of, oh, hey, this is some cool stuff. And then we just started talking about it more and more and wow. It's just, we really started creating what live ops is. Ah, so much fun. So much about the summit. Pause there. Yeah.
Crystin Cox:When you were, when you were running around ea sort of giving these presentations, this was like 2012 yeah, it was about 2012. What was the reactions like? What were, what were a little bit more specifically, what was the things that were the hardest for the developers to understand?
Speaker 4:That's a very good question. There's several different areas depending upon who they were. It was generally, there was a lot of hesitation at the time due to we're in mobile, it's small, it's weird and it's casual. We don't think there's a, that we don't think that what's, what's, what those, what you're doing there is directly applicable to the rest of the organizations. And so I always fighting an uphill battle with a lot of that. Um, and so that was more just the console/mobile debate, right? Like HD versus mobile games. Um, so that was a little bit of inroads, uh, that I had to go build it and say, hey, check it out. Really here's data, check it out. So I really had to fight for that. A few other avenues and topics. Let me think, that were really challenging at the time. Um, and it's just understanding games as a service, right? That was a really challenging thing. Um, people didn't know how to look at data. What is data? What should I look for? What is good? A lot of people really just fundamentally didn't understand because at the time a was what we now affectionally call onetime downloads premium games where you design it, you throw it over the fence, right? And then you're off to the next thing. A lot of folks weren't really under the, they didn't understand games as a service. So GAAS, GA, capital g, lowercase a lowercase a, capital s right at the time. GAAS. Ah, fun. Um, so that was a big challenging uphill battle that, um, a lot of folks just kinda checked out from what I was saying as a result, which is unfortunate. Um, but a lot of folks did. So a lot of folks really embraced it, including the sims free to play team. Right. Um, and also the Simpsons team, um, look at them today. Yep. Wow. They're not doing to say look bad. Right? Yeah. So yeah, it was generally just, it was, they didn't understand the different business model, which was really a big thing. And so it was challenging for them to really wrap their brain around it.
James Gwertzman:But one thing, I mean, one thing we found is we talk around Microsoft. I think there is this sense that the, the culture of a studio building for live ops is a very different mindset, a very different culture and a culture is building for, it's like a premium content. I agree. One thing we like to talk about is it's like, it's like being a movie director versus a cruise director or it's almost like you're designing your game to create opportunities for players to have really interesting interactions with your game and with each other versus, and I think you, you hit on that earlier, this notion of continually delivering basically what your consumers want.
Mike Pagano:Yeah. And it was really more of a treadmill of delivery, um, which they were not used to, get the game out there, test it, and tune that was a very different mentality because at the time we had disks and so they would not want to get the game out unless it was 120% tested, which takes time. Right. And so live ops, you did ship things more often. Sometimes they always more buggy. Mine were not. That's right.[inaudible] not, um, um, but yes.
James Gwertzman:So what, okay, so what is live ops? I mean, you, you, you what is, what does live ops?
Mike Pagano:So that should go well, that's a really good, that's a, that's a great[inaudible] question. Um so live operations, like what is it for us, the way that I look at and the way that I professed it to my team was, there's two different ways to look at it. One is service, right? It's a service, a consumer facing service. What are you getting in different time slices? So you can't think about the players going to play for x amount of time. They get to a level, hooray. This one is what is he usually doing day to day, week to week, month to month, year to year. Right? And so, and then building the experience around that and the pipeline around that, which is fundamentally different than creating the initial product. So this was more a television model as we like to look at it, which is what's, so you're always, you're always in a writer's room, right? If I may use the television, I've been in Los Angeles for a while, so you'll get some TV relations. Um, so we're, they're always designing. You're always designing, you're always about a month out. You're always testing, you're always releasing, you're always talking to your consumer. And one of the ones actually from live operations, the live operations from a, um, I started going rat holing down each one of the different ones. But live operations as a whole is the mentality of service. We're here to continually work with our consumer and work with our player to create an optimal and amazing experience for them, right. Over a period of time. So to me it was live operations is the real time interaction with my consumer.
Crystin Cox:Nice Pass. So you've been through this transition very, very much specifically this transition that a lot of the industry is now going through. What would you say if you go back and meet yourself? Yep. You're on the plane division thing and you'd be like, I'm going to tell you a couple of things.
Mike Pagano:Learn Chinese, learn Mandarin.
Crystin Cox:That's, I mean, that's good advice. So everyone learn mandarin.
James Gwertzman:Do, liveops learn Mandarin.
Crystin Cox:The key to liveops is Mandarin. You heard the time. It really was. Um, but if you could give yourself advice and the kind of advice, I think that would be very helpful today to a lot of teams who have now been together for a number of years, maybe shipped some games and are now saying we've got to make this transition. We have to care about engagement, we have to care about retention, we have to care about live ops. Yup.
Mike Pagano:Um, what I would recommend is, and what I've seen now through the industry and what I would recommend to my, my, uh, my younger self would be, it's less about. It's more about how you think about the consumer in real time and using that data to inform your decisions. Watch out for uh, pro product management. The reason why I say that is product managers at the time were operating also as producers and designers, right? Which is good and bad, right? But it's makes sure to constantly look at the fun, Eh, constantly look at engagement, look at retention, those, your primary metrics for success. You need to have a game that people want to play and people will come back to play after you have those two key ingredients, right? You can monetize and then you acquire more from there. Right? And so I recommend to my younger self, I would say, Mike, Think about your consumer that you're going to talk to them every day and you're going to make the best experience possible. Don't worry about the money initially, at least for the first year to two years. Wow. Um, at the time has changed of course at the time. Um, and really focused on making the best game possible with the audience that wants to play it right. Um, from there, make sure you document everything, make sure you have lots of analytics. But the other one really the other, and just as I was saying that, the one other one that really, really jumped up in my head is watch out for a[inaudible] metrics. Watch out. Really Watch out when, what do I mean by that? There's so much data out there. Like when you start looking at one game, um, of every interaction that's ever done with your consumer or with your player, I created terabytes of data from a 50,000 DAU game, terabytes in a day. How do you sift through that? What do you suppose, cause I always at the time, let's track everything. You don't need to track everything. You need to track the right things at the right time to inform the decisions that you want to make. The tools out there are fantastic and the tools that you will create, Mike in the past are fantastic or will fantastic. Right? But what you really want to look at is a very simple thing of what are your, do your consumers like it and do they come back to it? Do they want to spend money? Hmm. And do they want to tell their friends about it?
James Gwertzman:So let's, let's talk about analytics. If you're taking a game and maybe two game you're building now, you know, from scratch or maybe to gain that's been live for a while, but it doesn't have great analytics, where do you start to instrument again? Like what kind of telemetry does matter? I mean you, as you said, it's very easy to capture everything and then there was so much data, you never have a chance to look at it and it's almost meaningless. So what kinds of telemetry is important to actually help you later making the right decisions in, in live ops?
Mike Pagano:Really good question. It's different depending upon the organization and strategy by which you want, uh, by which you gauge success. Is, is success a, is it a dollar? Right? Is it users, is it a marketing campaign? Is it something? So you have to, from an organization level, we have to figure, you have to figure out what matters for you, what does good look like? And then from there we take a substantial amount of other data and start, um, uh, start reviewing that. So what does that look like? What does that data, how does it manifest itself? And so I was using those four different buckets and I'm going to consistently use engagement, retention cause I view those two very separate things. Um, uh, acquisition, I'm sorry, monetization acquisition. So what I recommend to do is first what is, what is your DAU? Right? How long and where are they playing? That is very instrumental. Whenever I approach a game, I want to know where my users at, what are they doing that's fundamental. And what does that mean? That means x axis level, y axis amount of players, right? Where are they at? Okay. And then from there, what is the progression of them? Where are they going, and how long does it take to get from point a to point B to level 30 to level 50, wherever you feel is the right place, where the hook is for your game, which offers optimal KPIs, key performance indicators as we like to call them. Um, and so we look and we focus on engagement. Where's the user at? Are they going to the places that we want them to go to? If not, is there any discoveries that we went, Oh wow, we didn't see that they were actually doing that. And Aha, that happens all of the time. Keep, keep your mind open. I think that's a really big thing because when you're looking at data and you're, and you're sifting through, you're going to find new things that you did not realize that your players were doing. And so you're looking at engagement metrics and so that's going to be time at certain location over a period of time and you go, okay, they're playing in the right areas that I want to play in. Oh, that's pretty neat. They're progressing as far as we want them to. Now we want to look at retention. Now what are you, where you start with retention. So you start with retention, where, what's the next day but coming back and they coming back and your, or are they people call d zero d- one d- two, depends on the company, right? So we go look at our, do you want to go? Okay, is this hitting the numbers that we want? And it's going to be different. So take for instance, a casino right is going to have a very um, uh, narrow funnel as we like to call it, which is a low day one retention, but a very high day, 30 Day, 90 retention that stays consistent. So depending upon category and your consumer is going to change how you're going to evaluate what good looks like, right? And so you look at your day one, your day seven, I like to go day 14 day, 30 day, 90 Day 120 day, 360. Right? So I can look at my, and we do it from a running perspective. That would be, um, so every a trailing, I forget what it was trailing or running, I forget what we were calling it. Um, every seven days we're looking at it and we're a trailing, ah, the trailing seven days[inaudible] 30 days. So we look at that and we take the team together and we look at those metrics. And so before jumping onto the other metrics, I think one, a really good process that I recommend to everyone is show all of your numbers to the Dev team. Show it every day. Have your project manager or your analyst or product manager, your analyst, reviewing numbers from every one of your KPIs and what you value is good. Are you hitting, are you not hitting it right? And then from there, don't panic when you're not hitting it. Why? Always ask why. Why is this number down? Is this number up? Is it just a cycle? Right? So take for instance, my installs dip, um, and my dau dips in my teletubbies game by 50%. I'm in the middle of the week. But what we found is over the course of six months, we found the player pattern is they play on the weekends. So what do we do from there? Aha. Why don't we do sales and promotions on Fridays, right? And so you start getting these, this, this interpretation of that analytics that help your business. And that's really what you're using. What you wanna do.
Crystin Cox:I do want to interject, just please, please as.
:we're on this topic. Yeah. How, this is something that I think has always been very challenging in my experience. How do you get a team not to chase the numbers off a cliff? That's a very, e specially short term. Like I think teams that are new to data, they want to, they want to look at things very quickly, they want to make decisions very quickly and they often want to just follow numbers that change quickly.
Mike Pagano:Yeah. Um, agreed in how do I think it's a delicate balance leadership, um, both from a product project, product management perspective, and a, um, a production perspective. And it's, Hey, look at longterm vision and know what good looks like for your category. So you're going to need to have research inside of there to say, okay, we just gotta you're gonna know when there's a problem. So take for instance, we had ghost harvest. My servers went down over the weekend, right? I didn't have a quality because quality of service did not exist. Um, uh, I lost 50% or 50% of my dau never came back. Yeah. Right. So that's where every, oh, watch out. So there is certain numbers that you do want to look at from a day to day basis, but from a longterm perspective, what is good look like? You have to understand that it's, it's more like statistics, right? Right. You don't want to look at a very narrow band of data. You're going to need to have a critical mass of data. And usually my rule of thumb was it's going to take 10,000 users right. Over the course of 30 days before you really start to understand the very basic behaviors of your players, right. Equilibrium. You need to receive all of it. You have to reach an equilibrium. Yeah. Um, and so that's setting out what does good look like at the beginning, see how you trend to it, but then change it. It's you should review and change what good looks like over a period of time to to match what your audience is doing that will hit your numbers. Hit Plan, right? Yeah. So having that longterm vision at the beginning and talking about it is usually the way that I was going about it.
James Gwertzman:So it takes 30 days to reach equilibrium before we can really understand what's going on inside your game. How do you, what's your approach to soft launch? Because I think a lot of, you know, games typically have this, modern games have this notion where you build your game, you soft launch it in some limited number of territories for awhile, could have dial things in and then you go go live globally. Not all games do that. That seems to be a common approach, you know it. Do you believe in that notion of like a soft launch? And if so, how long do you usually budget for a soft launch? And how do you know when you're ready to exit soft launch?
Mike Pagano:Yep. Yep. Um, so doing soft launch, so soft launch, I've done um, various different soft launches from territory Geo lock, soft launches to time restricted to, hey, I'm just going to shoot it live and see what happens. And then use that as my quote unquote solid like soft launch with no marketing. Right? Um, my rule of thumb is 10,000 people come in and play it. You'll start to understand your engagement. You won't understand retention, you won't understand. Um, um, um, monetization or acquisition that I do believe in. Uh, uh, I do believe in soft launch, I think it's going to take, it usually takes me about three months to really understand my basic system of retention. And then from there, monetization and acquisition takes me another six months. So for me, I like to have a soft launch that will really prove my KPIs. So if I'm a large business, I would, I generally recommend between a minimum three months. If you can squeeze a year, that's much, much better because you're going to come out with a much stronger product with a smaller audience who loves your game, who will weather the storm, who's the golden cohort. And we can go back to that one. You have the golden cohort who will love your game, whether they still give you the best feedback constable and you have a stronger product as a result. And so after you come out from soft launch, then you do acquisitions. So what have you soft launches is to prove my first three metrics. It's soft launch for me is to prove, um, first engagement. Are they playing where I want them to play too? Are they coming back with a month enough frequency in the timeframe that I want them to right? Um, and then three, are they spending money? Right? And how much are they spending? And then for me as the business, I can go, okay, is it ready yet? What is ready? So what is ready is you need to at the beginning to sit down and what does it look like for you to launch a product? What will make you confident in launching a product and creating your launch readiness? KPIs, right? And so what we generally do, so take for instance like casino, I'll, I'll use casino again cause we use that previously or kids, which actually have pretty similar kind of patterns, which are really interesting. I guess the older and younger you get right? With the whole belter. Thank God. Um, uh, so, so it's a lower da, it's a lower, um, retention at the beginning. So it's usually sub 30 and then you're getting into, probably for me it was a into the single digits when I started getting to the 30 to 90 days depending. Right. Um, uh, and I look at that and go, okay, are they playing where they're playing? Is that okay? And are they spending money where they should be spending money? Um, and so I'll come and then I'll, then we'll review as a team. So we look at it usually every, at the end of every sprint. So usually, one to two weeks. Um, we look at the doubt and say, okay, is the game ready? Okay, it's ready. Take a breath. Right? Because usually what happens is in another, another recommendation, should I say, your sprinting to finish, right? So you sprinting the get the game out. Time to market is king. You need to make that my gotta hit that deadline. You sprinting, you burnt the team, oh man. And then your launch. Oh man, it's feeling good. And then then the real sprint starts, right? This is when your operation changes to really go full tilt. And so you don't want to burn out your team initially getting that game out. So take a breath, look at the numbers. Are you going in the right direction? I'm really, I love how supercell does it. I really do. Because they take the time to look at it. Take a breath. Is this the direction? Because what I found, and I don't know if you all found the same thing, a lot of folks solely rely on a quantitative analysis, right? You need both quantitative and qualitative and design to really figure out what you should, what you really should be doing for your business. And so for you, that would be look to see what you think is get approved success with your financial models. Right, right. Review that with the team based upon the KPIs and the trend that you're seeing, do you think it will hold out? Do you think you'll keep getting your installs? Do you think that your users are going to keep staying in for as long as you need? Oh, you have all that data. Oh Nice. Sweet. Does it actually work? Yes. And then you can go to your boss and say, look, I hit all the numbers right. And if you don't, because 99% of the time you don't hit the exact numbers that you want because you have to understand that we, we, we're, we're estimating. And so you have to look at and continually refine your numbers. You see, are you okay with launching that and are you okay with really putting a lot of marketing because what does it mean to go to launch marketing dollars? You spend, you start spending, are you okay with spending? Right. And so there's actually a lot of intelligence that my marketing teams generally do for pre spend to make sure that they know end of the day, what does it turn into? What's your cost of acquiring your player? Right. Usually quite expensive. Um, uh, and then what's the lifetime value for your player? Does that equation make sense? Right. Oh it does based upon our numbers on our math crew. Awesome. Let's go through a whole bunch of money at it and right. And see to see if there was usually going to be quality enough to ensure that my retention stays where it needs to.
Crystin Cox:So you said something earlier though. Yeah, it's fascinating because I came mostly from an MMO background, which was much, oh wow. Bigger like really, really large, but really, really large scale downloads and like not considered casual. You said it takes nine to 12 months for monetization optimization for the kind of games that you've been working on. Yup, absolutely. The same experience in MMOs, even though you would think mobile games or casual games would be different. But actually my experience in MMOs is yes, it's about, it takes time. You have to actually spend time with the game before monetization can be optimized. It's, it's never really optimized right out of the gate. You have to actually look at behavior from players before you can start optimizing it.
Mike Pagano:Oh yeah. And that takes time. Yes. And it really does. And it's like you're going to have your initial cohort, we'll go show, here's another fun tidbit. I don't know if you felt the same thing, which was our golden cohort who had come in their numbers show your soft launch numbers will always be higher than your initial launch numbers because the degradation of users, the quality, the degradation of quality of players that come in. Not a bad thing. It's just think about it. It's uh, I come in, I play call of duty, then I get my friend who in the, a different type of game right into, oh, you got to play as, you'd be amazing thing ever. Right. You're not going to be as good and probably not stick around as long. Okay. So for us, I think it's more, it's really interesting you say that cause I kind of see, I see it, I saw it as the same way, which was you have real time data and your, your a service that you're, you're servicing a human. And a human has specific patterns that we need to as designers. Right? Catered to. And I think it's, it's the games that we were creating both had, even though they are casual, we used to, we used to joke, we called them, um, massively single player games, right? Where those big clicker games that you would come in where there's a single player, but you can go adventure off into other worlds. Um, I think yummy came up with that Yummy lace came up, um, a massively single player game, but it's a massive game, right? Complicated. Hundreds of thousands of millions of people. This is giant funnel of data coming in. You have to figure out what does it mean? What does it mean? What does good look like?
James Gwertzman:Let me ask about that. So what kind of mistakes can you make if you're trying to either optimize too soon or you're not looking at enough data, if you try to like over optimized too quickly, what kind of mistakes connect can that lead to?
Mike Pagano:Oh over optimized? That's an interesting, yeah. So it can take nine months to really understand. Yeah. So usually what I've seen, so I've been to a few different businesses now and played with a few different teams and it's interesting to see how they approach it. So I have an approach of it's a build and I've mentioned that before. Engagement, retention, monetization and acquisition. I've noticed a lot of folks, they don't focus so early social games, they were really fun games for the demographic. The demographic has now evolved, gotten bigger, better. They want more and more complex games. So at the beginning lots of folks took, took engagement engagement for uh, for granted for granted. And so we had a lot of folks who would come in and focus more on monetization focus more on economy model, right? So I guess that's monetization retention, right? Um, but doing it through abusive tactics. And so you probably saw that aligns that we still get echoes of that today. And that was one of the reasons why I believe, um, the HD games didn't like it so much, right? Because the, it was kind of a different mentality of how do we, and I've heard this several times and I'm like, it grates me as a game designer the wrong way. How do we extract the most money out of our whales? Right? I'm like, where do I start with that? I hate the term whale. Where are we? I know, I'm like, I call them VIP.
Crystin Cox:Yeah, it's not nice. It's not a nice thing to say.
Mike Pagano:No that it's not dolphin, middle whales, all that fun stuff. So I came from that world and so my whole thing was engagement, engagement, engagement, um, think fun, not grind think VIP is not. Um, so it's usually a fundamental different approach that they start squeezing their users. They squeeze their, users they think mentality of squeezing their users. So they said there, I think there's a fundamental shift that a lot of folk need to make, um, uh, that they need to focus on. It's a service. You're here for the consumer. Think of yourself as a band. Like Amazon is quite good with their consumer, right? If they come and they come, it's you. If you don't have your Dev team talking to your consumers, I guess that's another kind of big thing that we should kind of rattle down at some point. Um, um, but I found when you fundamentally look at the, the business from a different perspective of squeezing money out of them, um, how do we make people have, uh, bring more people in, right? Um, without focusing on making the game fun, right? You're gonna fail.
Crystin Cox:Oftentimes, when I talked to designers about this, I always say, you can't design to solve your problems because your problems are not your player's problems, right? So as soon as you're saying, well, how do we get money? How do we get them to bring us more people? You're maybe on the wrong road, right? You have to be thinking, well, what did they want? Yo, you know, what would they like to buy? Well, they want to do?
Mike Pagano:Here's an interesting thing now. So the parabola again. Yeah, right. Game designers came in and I'm a big champion of design. Love it, but I'm also a champion of PM, right? I'm a champion of analytics. Um, I like, I embraced the power of the, and I love it. I found a lot of resistance. Kinda jumps back to the previous conversation as well. I found a lot of resistance with um, uh, the game designers to look at data, right? Lots of folks don't want to look at data. They find that as a pencil push in a, um, or, or a Wall Street type mentality where you're just looking at numbers and they're not humans. I'm like gotta look at both. You really gotta look at both. So if there is any game designers listening to this, please, please, please go make best friends with your own analysts and go look at the data that you want to look at are players playing, what you want to play. How neat is that as a game designer, you can, I imagine you had this in MMOs where I was like, you look and you're like, Oh wow, they get to play my stuff. What are they doing?
Crystin Cox:Yeah, I mean I think you want to type it fall in love with data as a designer fall in love with data.
Mike Pagano:But don't let it be crippling.
Crystin Cox:No, I mean it shouldn't be the thing. It doesn't change your vision. It can never replace the goal. Yep. But it can help you so much, reach your goals and it can help you get real feedback. Like actual feedback that means something.
Mike Pagano:Right, right. Oh I love that. And it's a, and so there's a few things that we used to really push and it's all sort of coming back to me. Analysis leads to paralysis. Um, you can't always just constantly look at data. You got a eight, what is I? Or what does the ABS always be shipping? So you always, always hit your deadlines, you're a show, get the gains. So Travis Boatman was really pushing on this at the time and he was like, Hey, get the game out there. It's a live performance, right? Think of it, you are a live performance in the, in the customer expects a certain cadence. You got to get it out there. Yeah. Make the decision, don't rat hole on uh, on analytics. Yeah. Embrace it. But take that as an input point and also all of your stakeholders, which are also your consumers, listen to them and really push that forward. Ah,
Crystin Cox:Yeah, absolutely. I mean, I just say data's not predictive. Nope. It, no matter how much you look at it and you can look at it all day and you can cut it any way you want, it will not tell you the future. It won't, it can tell you how you did. It can validate whether or not your hypothesis was correct. But yes, it's very easy to get stuck in thinking, well, if I just have the right KPIs that I just put the right dashboards together, I'll always know. I'll become omnipotent.
Mike Pagano:That's where all these themes come from, I think. Yeah. Um, so a lot of folks come in and they chase, they think that they, and they usually do make some money. Right. Um, and they usually make these games where it's, it feels good, but theres something missing right there, they're not going after[inaudible]
Crystin Cox:It don't last long term. Right.
Mike Pagano:I mean that's the problem. It's a flash in the pan I think.
James Gwertzman:I mean you mentioned this now something, I think you've talked about the analytics data analysis paralysis thing. One of the reasons why people let themselves get paralyzed is something else you just said, which is it's a live performance. It's like doing a live show 24, seven. Yup. Where you screw up and you see the numbers right away. And so I think, so give me, can you maybe give you some examples of what mistakes you've made? Do you have any, any epic kind of train wreck stories of, of, of live ops disasters that either you narrowly averted or, or, or, or that happened and then, and then you had to dig out of?
Mike Pagano:Yeah, I think I so, I, I, I mean I went into the whole DAU of, of ghost harvest more. That just tanked. And we looked at, I went, Oh man, I'm glad this is not a major brand. Um, quality of service. Make sure you're looking at that. Um, one of the things that we really had a challenge with was, uh, customer support. We had a, we have mobile. Um, had one dude love him. He did a great job, but he did a, what is it, a hundred products, right? Um, and imagine you have a live service game where you have consumers who are very, uh, vocal about what they want because this is their game. They want it to keep going. So one of the things that we did to narrowly abort crisis, because I saw it time and time again a game would be great, and then it would go look at the forms and they just trashed the game, right? So what I did was I'm like, okay, cool. So Ea, you have a system where you have your own customer support. They dial in. I'm not going to do that. I'm going to do my own, which is a help@ eight pound gorilla that went to my inbox. Right, right. And so me as a, uh, I answered every one of the customer service myself. Yeah,
Crystin Cox:That sounds very scalable. Yeah. Yeah. Very.
Mike Pagano:Do you know what it really is? And so I disagree with a lot of folks who say it isn't, and because what I was doing was when you make a good game that isn't super buggy, that's usually the problem. When you have something that people really hate, you'll get flamed. But that's okay. What generally happens is from a customer service perspective is I came in every morning, I answered about 30, 60 minutes depending upon day of, it usually fell into like five different buckets, right. I can't remember the exact bug is something wrong with the game, this and this. Oh, I want this, I want this. And what I did was I was super liberal with it. Anytime somebody connected with me, I had their a, I had their information like, oh, hey, okay, cool. So I'm the Dev, Hey, what's going on? Talk to them like a human being. Yeah. Solved every one of them except for one where I guess this is, Oh, here you go. This kind of leads into it. So I saw every one of them except for one. So I had uh, had peak, I was at 80,000 DAU settled to about 40,000 DAU and I was the one customer service guy that did every one of them. Right. Every one of them. No problem. Was Not overworked. Not Terrible. Right. Insane to me. But that isn't, it isn't it? But, but, but what it turned into is just I love customer service. That's really my jam. It's like, so I've been in Indie bands, so I've been in death metal bands, grind core bands, right, where you have no audience and you just have to be friends with everybody. And I'm a friendly dude. I love people, I really love people. So when they reach out to me, I'm like, hey, cool, what's up? What's wrong with the game? It's my game. Yeah. Hey. Right. It's like, let's talk about what don't you like about it. Right. And so I use it as an interaction point because these are VIP consumers who are upset that your game isn't exactly what they want, but they want to like it. Yeah. Aw, that's so cool. So I can go to them and say, cool, what do you want? Oh, neat. Oh, what was the problem? So for me, that gives me a runtime analysis of what am I, what's my, uh, what, what do they like, what don't they like? What are the problems inside of the game from my quality of service, right. Because that didn't really exist at the time except in SAS. Um, and then, um, uh, being able to really modify that narrative so that I don't get negative reviews in a public form. Right. Right. And so they'll come to me and I'll solve everything that they want. And what I would do is I would give them a, usually like$50-$100 worth of stuff going, hey, thanks for reaching out. Yeah. Right. You're cool. Sweet. Thanks for playing. Here's a whole bunch of stuff. Have a good time. Right? And what happens is they stay longer and they spend more money. Yeah. Right. Because in love, right? Be Generous, be generous, be super, but keep an eye on it. Right? You do have people who come back and try to game the system, but it's really easy to find, right. Especially when you're not afraid to pick up the phone and call them. So what I would generally do, and this is actually one of the stories that didn't work, so I was doing ghost harvest, super proud about it. Right, really proud about my customer service there. But there was this one lady who reached out to us that had some problem like, oh man, that stinks. Like she had, there was some wrong where it was like an[inaudible] unit that there was something fundamentally wrong that, um, something was broken. I'm like, I can't figure it out. We don't have the device here. Um, can you, can I give you a call? Right. He was like, and she was just being super hostile at that point. Um, and all caps, do you think I'm an idiot? Duh.[inaudible] show the, the, no. So I did not solve that case. Like so out of, so I had a probably about, um, probably about a few hundred or a thousand over the course of a year. Um, uh, uh, customer service responses. And then one was you can't win them all. Yeah. Right. You just can't, I mean, you're gonna try, right. I would try like, I really think because they want to love your, they want to love it. Right? Make sure you put people on the front line who actually can talk to folk. Right. Right. Interpolate what they have and give it back to your team. One of the things that I really like to do, my team really chagrined it, right? They're like, oh, I don't know about this. I make them do customer service like no kid and go in the front. Like talk to the consumer. If you're an engineer, you're an artist. You should know what they want, right? Because you're part of the band, right? You need to do some customer service. Go sign some autographs. Right. Tell them that you're the artist on it. There'd be super jazz just to talk to you. I love that. Yeah.
Crystin Cox:Next one, when we did training our training, every single person who's hired at an Nexon does a two weeks of customer service\.
Mike Pagano:No, and even I think showing you got a little bit matter who you are nexon was one of the folks that we were looking at like next on their, they were OG live operations from the web days where there is no platform besides the web. Yes. So how do you, how do you engage and retain your consumers?
Speaker 3:[inaudible]
Speaker 1:Thanks for listening to the art of live ops podcast. If you liked what you heard, remember to rate, review and subscribe so others can find us and visit playfab.com for more information on solutions for all your live ops needs. Thanks for tuning in.