Steve Meretzky has been called the Steven Spielberg of adventure games, and the funniest man in the game business. Seeing his name on a game box is akin to seeing The Beatles on an album cover, and his resume reads like a best-of compilation. In this week’s episode, this game design all-star talks about what LiveOps looked like back in 1982 and the importance of using data throughout the game design process.
Steve Meretzky has been called the Steven Spielberg of adventure games, and the funniest man in the game business. Seeing his name on a game box is akin to seeing The Beatles on an album cover, and his resume reads like a best-of compilation. In this week’s episode, this game design all-star talks about what LiveOps looked like back in 1982 and the importance of using data throughout the game design process.
00:05 Hello. I'm James Gwertzman.
:Crystin Cox 00:06 I'm Crystin Cox. Welcome to The Art of LiveOps podcast.. Crystin Cox 00:15 I've been making games for 16 years as a designer and a game director, focused on LiveOps. James Gwertzman 00:20 I founded PlayFab five years ago because I saw a huge gap in the kinds of access to LiveOps technologies game studios needed to be successful. We've put this podcast together because a lot of the information on how to do LiveOps effectively just isn't out there. There's way more information about how to optimize your graphics pipeline or how to put together effective game design than there is on how to do effective LiveOps. Crystin Cox 00:39 And since it's pretty tough to go around yourself and just find everyone that does LiveOps and ask them questions, we did it for you. James Gwertzman 00:45 We've picked together a list of some of the top practitioners in the industry, folks who we think are really pushing the boundaries and doing some cutting edge work. Crystin Cox 00:51 And we interview a different one every week and ask them about their experiences running live games, doing LiveOps, and even having LiveOps disasters. James Gwertzman 00:59 Yeah. Train wrecks are the best. That's where you really learn how to do things effectively. Crystin Cox 01:07 Hey, James. James Gwertzman 01:07 Hey, Crystin. So today we have Steve Meretzky, and I mean, I might say grease for the world of free-to-play gaming. Crystin Cox 01:15 Yes. The thing that really holds all of online gaming together is Steve Meretzky. James Gwertzman 01:20 Right. And it'll be fun because he's famous for having a very dry wit and a very deadpan humor. Steve Meretzky 01:25 Why do you have cool headphones and I don't? That doesn't seem fair. Crystin Cox 01:31 Yeah, but I mean he really has been around in the game industry for a very long time. He's seen a lot of changes in the game industry. I think he started out in Text Adventures. Steve Meretzky 01:40 I have been a game designer since roughly 1982. My first game was launched in 1983. It was called Planetfall. And I have evolved across many different genres and platforms and business models, but have remained a game designer. Crystin Cox 02:00 And so I think- James Gwertzman 02:02 So he was at InfoCom for awhile? Crystin Cox 02:03 Absolutely. Yeah. He has a lot of really interesting insights into the way the industry has changed and also just a lot of perspective. James Gwertzman 02:08 Yeah. So this is going to be great. So let's go to it. Steve Meretzky 02:18 We didn't really do LiveOps, but we kind of did LiveOps, in that basically we would launch a game... So the way the whole InfoCom system worked is, you had a machine independent game file that was called a ZIP file, short for Zork Implementation, no, Zork Interpreted Program. That's what it stood for. Steve Meretzky 02:43 And then there was an interpreter that basically took that, took any such ZIP file, and made it run on an Apple, an Atari, Commodore 64 or a TRS-80, what have you. So you would write a game, and particularly over the course of the first few years of the 90s, there was a new PC coming out every other day. So, we would then write an interpreter for that new machine, and then suddenly the whole library of Infocom games would be running on that machine. Steve Meretzky 03:18 The best instance was when the Mac came out and we wrote an interpreter for the Mac and we had a whole library of games, like at that time, 15 games out on the Mac, at a time when there were maybe 25 games total available for the Mac. So in the first 30 days that the Mac was on the market, we sold more Infocom games for the Mac than Macs were sold. Because obviously some people-- Crystin Cox 03:46 That's a good attach rate. That's not bad. Steve Meretzky 03:51 So anyway, my point is, you would write a game in 1981 or 1982 or 1983, but then new versions of that game would come out, the Amiga would come out, and suddenly, this three year old game would be rereleased, not rereleased but released for the first time on the Amiga. So basically, the attitude was we should always be kind of correcting and updating the games. And obviously a lot of that would be bug fixing. So the game would get out in the wild and people would report bugs. You would fix that, but sometimes it was just sort of retrofitting best practices. Steve Meretzky 04:27 For example, one best practice that came along... middle of the 1980s, so after many InfoCom games had already been released, we came up with the idea of the Oops command. So if you typed a long sentence but had like one letter misspelled, if you typed"yellow" with one L or something, instead of having to retype the whole thing, you could just say"Oops yellow" and it would basically replace the word it didn't know with the word after"oops" and re-input that into that sentence. So improvements like that would then have to get retrofitted to the older games. So not really LiveOps as we think of them today. James Gwertzman 05:09 So the data driven gaming, I mean that's... You were basically doing data-driven games from the early PC era. Steve Meretzky 05:17 Yeah, there wasn't that much data. We had sales data, of course, we had, mostly anecdotal data that would come in from players, but nothing like what we think of today when we talk about data-driven. Crystin Cox 05:32 As you said yourself though, you've reinvented yourself across a lot of different genres in the industry, in a lot of different platforms. Steve Meretzky 05:41 I think of myself as the Madonna of gaming. Crystin Cox 05:42 Yes. As yes, I think most people do. They think Steve Meretzky, they think Madonna. But I think a lot of other devs right now are going through something similar. A lot of them for the first times in their career, where they've maybe been doing something like console AAA development or they've been doing premium PC development and they're now starting to emerge into a world where they need to start thinking about things like LiveOps. Do you have any pearls of wisdom for people who are going through that kind of reinvention? Steve Meretzky 06:12 Yeah, they should probably just get out of the industry because there's enough competition really, and free-to-play gaming as it is, and if more people learn how to do it, then the competition will just get that much worse. Crystin Cox 06:23 So you're very supportive.[crosstalk 00:06:23]. James Gwertzman 06:23 So we should Discount all the advice you give us from here on forward. Steve Meretzky 06:27 Oh I'm planning on giving entirely wrong advice for exactly that reason. Crystin Cox 06:30 I like that. That's good. So you're very supportive of what we're doing with the LiveOps Academy. Steve Meretzky 06:38 I mean, you know a lot of what you do with LiveOps is pretty common sense. A lot of it is cooperation. There are a lot of disciplines that sort of come together for LiveOps. Obviously you want the data from your BI unit or whatever. You want a project manager who's sort of overseeing the product, or the product line, to be kind of identifying the key metrics where the game is underperforming or could stand to be improved. And you want designers to work with that PM to kind of figure out new content or new system or whatever that will help improve that KPI but still be true to whatever the vision of the game is. And then of course you need all the other disciplines to execute on that plan. Crystin Cox 07:31 How do you think, in general, design has changed as we've moved into this new era? I mean it's changed quite a bit just with the introduction of the internet, but now we're seeing sort of constant connectivity. Steve Meretzky 07:46 I used to give a talk in the very early days of free-to-play, sort of 2008, 2009 kind of timeframe, explaining free-to-play to people who were just coming into it for the first time. And I showed a slide, kind of saying how in free-to-play design and business are just much more closely intertwined than they were before. And I showed a, I'm not even sure how you pronounce it, a caduceus, the medical symbol with the two snakes? And I'm saying but this isn't really accurate and in free-to-play, business and designer are really connected this way, and then I would just show a blender with two arrows pointing to the content of the blender saying design and business. Steve Meretzky 08:31 Because it really is that intimate of a connection. There's really no place where design stops and business starts. You need, as a designer, to be thinking about the business of free-to-play games right from the get-go. And I think that's the biggest change, in the old days, obviously when you embarked on a game you'd want to be thinking, is there a market for this game? And stuff like that. But for the most part you would write the game and hand it over to business people and then they would do the grubby work of actually turning that into money. James Gwertzman 09:05 The game industry has always been interesting because there has been, I think for a long time there's almost this sense that money... It's to me you should not... It's a bad word. And I think if you talked to game developers about this, they might make more money in this game than that, they were like, horror, that that's a terrible idea. And I think you're right, I think a lot of the initial pushback to... and yes, we kind of overdid free-to-play when it first came out in some ways by being a little bit too focused sometimes on these metrics. But I think there was the sense of we have to get our hands dirty now and this is somehow impure. And I'm curious if you kind of have a sense of where that attitude came from or have you sense of Steve Meretzky 09:41 You know, I agree that that attitude exists and I think it came from exactly this aspect of designers having to suddenly be thinking about business constantly throughout the entire process of creating a game and through launch and post-launch. Fortunately it's never been an issue for me since I have no morals whatsoever and I'm completely 100% a total money-grubber. Crystin Cox 10:08 It helps. James Gwertzman 10:09 Helps out, yeah. Crystin Cox 10:11 So when you look at where we're going, where we see live games headed today, are there things that are getting you excited? What do you want to see us do more of? Steve Meretzky 10:23 I don't really have any ideas on, in terms of sort of the evolution of live games or free-to-play games from a business point of view. I certainly have lots of ideas from a creative point of view about genres that have been popular in the past and have never really either been done at all, attempted, or done particularly well yet, in free-to-play. And that kind of excites me. Steve Meretzky 10:54 But you know, as far as the business model itself, I feel like we've been doing it in the Western world for 10 plus years, obviously in Asia even longer. I think most of the kinks have been ironed out. You know, as you said, most of the more egregious money-grubbing aspects have clearly generated enough resistance in the market that game companies kind of know where to draw the line and what's reasonably acceptable practices and what aren't. Steve Meretzky 11:32 And I don't foresee any kind of major new revelations on the business side. Like,"Oh my God, why didn't we think of this 10 years ago?" kind of stuff. Crystin Cox 11:43 That's kind of comforting to think that we might actually be emerging into some level of stability and actually starting to learn to do things well. Steve Meretzky 11:54 There certainly could be disruptions in terms of platforms that deliver free-to-play gaming in really new ways that make us have to rethink the business side of things, but nothing is occurring to me. Crystin Cox 12:07 Yeah. We spend a lot of time in these talks, we end up diving really deep into analytics. It seems that analytics is still one of the things, and data in general is still one of the things, that a lot of developers have a hard time wrapping their heads around or have a hard time really diving into. How do you... How has your sort of relationship been with data as a designer? Steve Meretzky 12:30 Well one problem with data is there's just so much of it you can drown in it. It's really hard to kind of separate the wheat from the chaff and know what's really giving you insights and what is just a lot of noise and occasionally what's actually misleading you. Steve Meretzky 12:48 So when I've... Sort of what I think of as my first time in LiveOps or free-to-play, is when I was at WorldWinner from 2000 to 2005, which was a website in the early days of the internet. We were basically pioneering the cash tournaments or also known as the skill tournaments model. Steve Meretzky 13:10 And we never used the term free-to-play, no one was at that point, but in every way it was free-to-play. And... Why am I telling this story? What was the question again? Crystin Cox 13:24 Tell me about data?[crosstalk 00:13:27]. Steve Meretzky 13:33 We were tracking revenue by day and it was kind of gradually going up. We launched in sort of late summer of 2000 and kind of gradually going up and up, and then we hit June, 2001 and suddenly our revenue dropped 15% ish. It was something noticeable, not catastrophic enough that management was sort of a little bit panicked. And there were of course lots of theories, something is broken, we either changed the experience somehow, or players are not able to get in, or all those theories that you could imagine kind of being batted around. Steve Meretzky 14:19 And my theory was this happened right around the third week in June when kids get out of school, families start going on vacation. Most of our players were middle aged women. Suddenly they had kids at home, they don't have as much time during the day, et cetera. And nevertheless we spent all summer trying X, Y and Z without any effect. And then suddenly around labor day, that 15% just went right back up again. Steve Meretzky 14:50 And then next year, on June 21st, at that point there were new managers and again they panicked, and I said,"Hey, a year ago this happened and we tried everything, and I think it's pretty clear it was just summer seasonality." And having the data from last year to back it up kind of helped us not have to go through the same circus again. Crystin Cox 15:21 I love... Every game has slightly different seasonality. I love... And talking to other people about their games, depending on who your demographic is, like you said that you had a lot of middle aged women who were like,"Oh, now that it's summer, I don't have as much time," whereas when I was working on MapleStory, summer was fantastic for us. We actually can tell where school got out, what day school got out in different parts of the country, because we would see geographically all of our logins start going up in the middle of the day. Steve Meretzky 15:55 My favorite kind of heartbeat story is on 9/11 we kind of weren't looking at the heartbeat at the time, looking back on it the next day or whatever, the typical kind of day cycle was an oscillating sort of sine wave peaking at kind of early evening Eastern time, with its trough maybe 5:00 AM Eastern time, and so on 9/11 it was just starting to come out of the trough and climb into the morning as it usually did. And then right when the first plane hit, it just dived down to zero and stayed at zero for like an hour or so. And then it went right back to normal. Crystin Cox 16:50 Whoa. That's crazy. That's fascinating. Steve Meretzky 16:54 Yeah, it was totally fascinating. Crystin Cox 16:55 One of my favorite things for that, and I wonder if you experienced it, we used to kind of celebrate Chinese New Year because on Chinese New Year, every year when Chinese New Year began, suddenly all of our bot activity would plummet and our bot activity would stay plummeted for about a week and a half. Even though... It didn't matter. Crystin Cox 17:23 I think every game I've ever worked on had that pattern. The bots would drop. Because everyone takes the time off. Literally everyone in the world, well not everyone in the world, a large portion of the world celebrates Chinese New Year. So a large portion of the world will be taking time off. And since most people who are running bots, that's a job that they have, they would take that time off too, and we would have this sort of short period of time in the game where there just really weren't any bots around. James Gwertzman 17:51 The gold farmers were all on vacation. Crystin Cox 17:51 Yeah, the gold farmers are on vacation. Yeah. James Gwertzman 17:55 That's awesome. Steve Meretzky 17:58 Yeah. James Gwertzman 18:00 So you talked about data, and drowning in data, and I think that that is in fact a common theme. The themes we've heard so far is"What do you do with this data?" and"What if the data's wrong?" because a lot of times wrong data means wrong decisions because the data's erroneous. But where have you actually found data helpful? Can you think of any examples where you've actually looked at data and said"Oh, here's a thing," and made a fixed and... you know. Steve Meretzky 18:21 Oh, I mean a great one is just kind of creating your FTUE, your First-Time User Experience. That's obviously super crucial with free-to-play games. Because once you've spent a few dollars to acquire a user, you want them to stick around. And you lose most of your players just during the first few minutes of game. So really perfecting that first 20 or 50 clicks or taps is just so crucial. Steve Meretzky 18:57 Basically having the data on exactly what moment in that flow, what percentage of players just stop and go away, and being able to look for the ones where you have... I mean on every one of those steps you're going to have a percent or two or three, but if you have one step where all of a sudden you're losing 10% of your players, or 15% of your players, you look carefully at that and what's going wrong there. Is there a long load time, is it a bad UI design. Just game after game, that's been a hugely useful use of data. Crystin Cox 19:37 Yeah, I've actually noticed that specifically First Time User Experiences are a place where I often recommend the designers actually be data driven, not even data, just data informed. It's so difficult to get perspective on your own tutorial, your own onboarding experience after you've been around the game for even a short amount of time. You really want to look at that data. Crystin Cox 20:03 Are there other places where you feel like that's true? Because I know some designers kind of chafe against the idea of letting data drive their decisions. Steve Meretzky 20:11 I think they're... Basically decisions that are easily measurable are the ones that should be driven by data, and the ones that aren't are the ones where you should rely on experience and intuition. And certainly anything that you can A/B test, whether it's the color of a button, the text on a button, the price of a first day package offer, which package in the store you put the little starburst on that says"most popular." There's just hundreds of things like that in every free-to-play game and they all easily lend themselves to A/B testing but there are obviously all sorts of creative decisions, particularly early on in, in the life of a product. But certainly, sometimes during live operations as well, they just don't lend themselves to easy measuring of that sort. James Gwertzman 21:14 Have you used A/B testing much yourself? Another thing that we've heard a lot about is A/B testing is very useful, but very hard and very easy to screw up and get wrong. Steve Meretzky 21:22 Well, I mean that's where it helps to have a really good analytics team, which at both of the companies where I did most of my free-to-play work, which was Playdom and then later at GSN, we had very good data analytics groups. But it's certainly a matter of designing the test well, knowing when you have collected enough data that you have a really high confidence level, analyzing the data to make sure that you're not essentially looking at noise, that you're really looking at results.. Steve Meretzky 22:00 One that I can think of that we did at GSN, we redesigned the store where you would go to basically buy the tokens that you use to play the social casino games. And I can not remember even now what the change was, whether we increased the number of coins you got and the price, or decreased the number of coins you got and decreased the price. We made some kind of major change like that across the board, and then A/B tested it for a really long time to get confidence. And it's sort of one of those cases like when you're doing drug trials and the trial was so effective that you kind of stop giving people the placebo[crosstalk 00:22:49].. Steve Meretzky 22:55 It was so effective that people were arguing"Let's just cut the A/B test off even though, sure, we haven't gotten to our 99% certainty or whatever that we normally like to get to." But no, they stuck with it for a month and a half, several months, I forget how long it was. I mean, we were leaving so much money on the table, when we finally switched everyone over to it, it just felt so frustrating that we weren't just calling the test off.. Crystin Cox 23:27 Did you guys worry at all about crosstalk?. Steve Meretzky 23:30 Crosstalk is always a problem when you're doing an A/B test, less so with a casual audience than with a mid-core or hardcore audience.. Crystin Cox 23:42 Especially for things that have to do with purchasing. It always made us so nervous, so worried. Like you said, you were in a situation where one group is clearly having a better experience than the other two and you're worried that the other groups are going to be like,"Why aren't we having this?". Steve Meretzky 24:00 Yeah. Anyway, I mean the customer service people were pretty good at saying,"Well we're just doing some testing and you know, it's temporary.". James Gwertzman 24:12 What about, so I want to go back to talking about community a little bit, the Marshmello concert in Fortnite. I keep thinking that this is... Because I don't know if I think this is... we're going to look back and say that was really a changing point in how games are in pop culture or relating, or maybe it's actually just a blip and I shouldn't read too much into it. What do you think about how games really are becoming these community platforms now, and we're seeing a level of, I don't know, pop culture impact by certain activities?. Steve Meretzky 24:42 Yeah, I mean I don't really think"becoming." I mean if, I think if you go back to certainly Candy Crush, certainly when it hit Facebook and definitely when it hit mobile, which was what, six or seven years ago now, I think? I mean that was just so widespread and I can even remember like TV commercials for other products, Angry Birds certainly was another one like that, Farmville even, I mean 29 million people per day at its peak. How many per month? A hundred million per month probably?. Crystin Cox 25:16 It's a lot. And I mean Pokémon Go, too, that was... If you're talking about something on sort of the level of the Marshmello concert, seeing videos of people just kind of wandering around in parks with phones in front of them. I mean it was certainly something, and I do think that we are seeing games take a... not a more important, but a different place in general pop culture. They've definitely become mainstream in the last 10 years in a way that they weren't before. The industry has certainly grown.. James Gwertzman 25:55 What about train wreck stories, so one thing we've been having a lot of fun with is sort of train wreck stories.. Crystin Cox 25:58 Yes. Everybody wants to hear your LiveOps disasters.. Steve Meretzky 26:06 Gosh, I should be able to think of all kinds of anecdotes, but nothing's coming to mind.. Crystin Cox 26:13 You're like,"I've never had a disaster. It's just I've been perfect.". Steve Meretzky 26:17 I'm sure we have, but it's just... I guess I've done a good job suppressing them.. James Gwertzman 26:25 One of the worst, I'll seed you with one. I did actually an entire night of LiveOps train wreck stories to Space Ape's offices in London one time where we just... I shared a couple, other people got up and shared a couple and then we opened the floor and people just got up and one at a time, it's like a AA meeting."Hi, I'm a game designer and here's my LiveOps train wreck story." And one of my favorite ones was, it was some poor person who essentially, they inadvertently created a level inflation or economy so dramatic that they honestly couldn't recover from, and they were not able to roll back their economy, and they said they literally killed the game. From that... a couple of days later, they never recovered from it. Wow. And I, I had trouble.. I guess they had lost so many players and such a short amount of time that they just.... Steve Meretzky 27:11 Well, the first game that I worked at, at Playdom, was Mobsters, which was already live when I arrived at Playdom even though I was pretty early on at Playdom. And Mobsters was a clone of an earlier Facebook game called.... James Gwertzman 27:29 Mafia Wars?. Steve Meretzky 27:33 No, Mafia Wars was Zynga's own... Mob Wars was the original game. And then Playdom cloned it, called it Mobsters, Zynga cloned it, called it Mafia Wars. But all three games basically had the same problem, which was you would buy territory, and then the territory would generate soft currency, which would allow you to buy more territory, which would generate more soft currency. And within the first 10 days or so of playing the game, you had more soft currency that you needed to buy everything in the game, and the economy was basically a joke. The solution was basically not to rely on sales of soft currency as the way the game monetized. Of course it never... After the first few days, no one ever monetized by buying soft currency, but we figured out a bunch of other ways to monetize players. So you can kind of recover even from destroying your game's economy.. Crystin Cox 28:34 Yeah. I'm a little surprised that they ended up folding it because there's a number of, there are things you could do.. James Gwertzman 28:39 Maybe it was dramatic intent. I would love to find this poor person and say--. Crystin Cox 28:40"Are you okay? Is your game alright?". James Gwertzman 28:41 Yeah.. Crystin Cox 28:50 We once accidentally turned off all combat in our World v world mode. So it was a mode that was just about PVP, and it's the only thing you can do in those maps, and we accidentally just completely disabled it. And what's great about that story is that the community actually loved it so much that they marked it as a day, and it became known as Guild Wars 2 Friendship Day. And then every year after that they had a ceasefire inside of World v World on that day to commemorate the time when they could not fight each other. It only lasts a couple of hours.. James Gwertzman 29:31 It's like the famous World War 2, World War 1 Christmas story when the Germans and the Brits stopped fighting for one, I think it was Christmas Eve or whatever, and then went right back to shooting people the next day.. Crystin Cox 29:42 It's very beautiful.. James Gwertzman 29:43 That's great.. Crystin Cox 29:44 Very very community.. James Gwertzman 29:47 What else have we not covered? We've talked about... Let's talk about tools. I mean it's obviously PlayFab's a tools company. I personally love tools and I think lots of companies have built various versions of LiveOps tools over the years for themselves. What are some of your kind of go-to tools or what do you like to see in your LiveOps?. Steve Meretzky 30:05 Pretty much everything has always been proprietary to whatever company I worked at. Mostly it was geared toward taking the data and putting it in the hands of non data analysts in a way that made it accessible. So it was some kind of a dashboard with all sorts of pre-designed reports that you could just run, pick a report and give a date range or that sort of a thing. And then sometimes some very simple tools that would let you create your own report without having to bother the data analysts.. Steve Meretzky 30:43 And certainly I relied on those very heavily to get a sense of what was working and what needed attention. There were times, certainly Playdom was a very PM intensive environment, but like at WorldWinner for example, I was essentially my own PM. So I was kind of the guy telling me what to do when I wore my designer hat. I would have frequent arguments with myself and say,"Steve, you money-grubbing idiot," and then I would say like,"Steve, you touchy feely, visionary asshole, just do it. I'm telling you." Fortunately I did that in my office with the door closed. So I was never actually taken away.. Crystin Cox 31:35 Do you have any... I mean I know we had little proprietary tools at a couple of the places I worked that I spend a lot of time sort of fantasizing about, just because they were built really specifically for like one game, you can never really replicate them in other places. Like we had tools that let us actually take an item in a game and pull it back out of the game, but it would actually do it through every step had ever existed inside of the game's economy. So if someone had it in their inventory it wouldn't just take it out of their inventory. It would like roll back--. James Gwertzman 32:11 Reverse transactions?. Crystin Cox 32:12 Yeah, it would basically reverse all the transactions so that you would leave all the people in between whole.. James Gwertzman 32:16 So it's erasing members.. Crystin Cox 32:19 It was really cool. It was actually something that was really neat, it let us do a lot of crazy things with, with fraud. Do you have any crazy tools? You guys ever had.... Steve Meretzky 32:27 Well, one going back to Mobsters at Playdom. We, as I said, the soft currency economy was ruined, so most of the money that the game made was by selling premium items which could only be bought for money instead of for soft currency. And so a big part of my job was designing those items and sort of figuring out how much it would cost, what they would be called, what they would look like, how much power they would have, and then how long would they be on sale for. And also we did sort of cheesy things like sometimes we would pretend there was a limited number of them and that they'd run out even though there wasn't.. Crystin Cox 33:10 Virtual scarcity.. Steve Meretzky 33:14 And so basically we just had a huge spreadsheet, an Excel spreadsheet that all of that information would be in, and I would design that months ahead of time and the game would just read the data out of that spreadsheet and do the right thing.. James Gwertzman 33:28 Yeah. We were talking about spreadsheets, I mean spreadsheet as design tool I think is an underappreciated resource.. Steve Meretzky 33:36 Other than the fact that the sentence"Spreadsheets is a design resource," or"an underappreciated resource" has been said over and over again.. Crystin Cox 33:42 Yeah, but it's funny, at Microsoft, I think, they still kind of don't get that... I just told them a couple... Excel is the most popular game design total in the world.. James Gwertzman 33:54 Yeah, we were planning our booth at GDC and we had a design section and we have nothing in Microsoft or design. I'm like,"Uh, Excel?". Crystin Cox 34:00 Yeah, Excel is the most popular design tool in the whole world.. Steve Meretzky 34:02 Yeah. I remember it was... Dr. Cat was running kind of an all day video feed on the, I think 20th anniversary or something, in this game Furcadia. And I went to his house and sort of co-hosted with him and he had just dozens of guests all day long, calling in for 15 minutes at a time or whatever. And one was Ed Fries, and he's like,"I don't really know why I belong here, I've never, I'm not a game creator," and I said,"But you created Excel! The greatest game creation tool.". Crystin Cox 34:38 Our favorite tool ever! I mean some people make games in Excel. It could a game engine as well. Can do anything.. James Gwertzman 34:48 I think we should build a LiveOps for Excel.. Crystin Cox 34:50 Yes we should!. James Gwertzman 34:51 We should have a LiveOps game in Excel.. Crystin Cox 34:52 I agree.. Steve Meretzky 34:54 You know that scene in A Beautiful Mind, towards the beginning when he goes to some defense contractor or whatever and they think that there's like a secret code embedded in these messages for Cuba or something like that, and he's just looking at this giant wall of numbers and he's just staring at it for an hour and no one is talking to him, and numbers are sort of starting to light up and then patterns are filling in and then he turns around and says,"Okay, I've cracked the code." I often feel that way when I'm staring at a game design spreadsheet.. Crystin Cox 35:30 Yeah. That's, basically how game design happens, is you just stare at a spreadsheet full of numbers until something emerges, and then the game just sort of... That's the secret. We don't want anyone to know.. James Gwertzman 35:43 There you go. Anyone listening to this, that's the secret.. Crystin Cox 35:45 That's the secret to game design.. James Gwertzman 35:47 And then you compile the spreadsheet, you pump it out as a GSN file and you upload it into your--. Crystin Cox 35:51 To PlayFab.. James Gwertzman 35:55 Any last pearls of wisdom? Any, any last musings on the discipline of LiveOps?. Steve Meretzky 36:02 Well, I guess just, it's sort of in terms of fortune cookie rules, Richard Rouse at the Rules of Play, Rules of the Game? What is that session called? Anyway. He was talking about these sort of pithy little rules of game design as fortune cookie rules. And I'd say for LiveOps, really you're trying to keep the game fresh, and you're trying to improve KPIs and pretty much every LiveOp you do probably should be doing one or the other. And you don't want to do LiveOps just for the sake of doing LiveOps. You want to do it either because it's making the game feel alive and new and changing, or because you're improving one of those KPIs like D97 retention or monetization from among males 37 to 39.. Crystin Cox 36:57 The core demographic.. James Gwertzman 36:59 As it were. Yeah. Cool.. Crystin Cox 37:02 Thank you so much. This is great.. Steve Meretzky 37:03 This was fun.. James Gwertzman 37:04 Thank you.. Crystin Cox 37:04 Thanks for coming over.. Crystin Cox 37:12 Thanks for listening to The Art of LiveOps podcast. James Gwertzman 37:14 If you like what you heard, remember to rate, review and subscribe so others can find us. Crystin Cox 37:19 And visit playfab.com for more information on solutions for all your LiveOps needs. James Gwertzman 37:24 Thanks for tuning in. Rev’s Quality Team reviews all transcripts rated 3 stars or less.