Jill Braff, GM of Integrations and Mobile Games at Microsoft, has a 25 year career in gaming and media. She's worked on early console games with Nintendo, Sega and Konami; helped build Glu mobile through the launch of the iPhone; and spent a decade in media at Home Shopping Network and the Ellen DeGeneres show. She currently works with the Zenimax business for Xbox Games Studios.
Jill discusses the similarities between the live call-in aspects of Home Shopping Network with LiveOps in gaming today and how the advent of the iPhone and Facebook changed the landscape of mobile game development. Jill shows how the implementation of live data has given developers insights into player's habits and the amount of time they spend playing mobile games. And she illuminates us on an idea that "casual" games don't really exist.
This episode is brought to you by Azure PlayFab. Visit https://www.playfab.com for all your LiveOps needs.
Jill Braff, GM of Integrations and Mobile Games at Microsoft, has a 25 year career in gaming and media. She's worked on early console games with Nintendo, Sega and Konami; helped build Glu mobile through the launch of the iPhone; and spent a decade in media at Home Shopping Network and the Ellen DeGeneres show. She currently works with the Zenimax business for Xbox Games Studios.
Jill discusses the similarities between the live call-in aspects of Home Shopping Network with LiveOps in gaming today and how the advent of the iPhone and Facebook changed the landscape of mobile game development. Jill shows how the implementation of live data has given developers insights into player's habits and the amount of time they spend playing mobile games. And she illuminates us on an idea that "casual" games don't really exist.
This episode is brought to you by Azure PlayFab. Visit https://www.playfab.com for all your LiveOps needs.
00;00;05;00 - 00;00;06;14
James Gwertzman
Hello. I'm James Gwertzman.
00;00;06;14 - 00;00;09;23
Crystin Cox
I'm Crystin Cox. Welcome to The Art of LiveOps podcast.
00;00;14;03 - 00;00;19;02
Crystin Cox
Hey everybody, it's me, Crystin. Just me this time. But I'm joined by an awesome guest, Jill Braff.
00;00;19;03 - 00;00;47;16
Jill Braff
I've spent the last 25 plus years in the gaming and media space. Started in console and Nintendo and Sega during the console wars and then moved into third party at Konami and something called Bam Entertainment, which came in and left. And then I helped build Glu Mobile from just a tiny company pre iPhone through the launch of iPhone and taking it public, which was fascinating and we talk a lot about LiveOps and how the world changed right around then.
00;00;47;16 - 00;01;11;20
Jill Braff
And then just before it came to Xbox, I spent about a decade in media, first at Home Shopping Network helping their digital transformation, which is all about liveops and then at Ellen DeGeneres helping build Ellen's digital business. I joined Microsoft about a year and a half ago. I've been working on the Zenimax business, starting to branch out now into casual and perhaps some mobile exploration as well.
00;01;11;26 - 00;01;41;10
Crystin Cox
Jill works on the Xbox Game Studios leadership team. She handles integrations and mobile development exploration. She has recently worked on leading the Zenimax acquisition for Xbox Game Studios and the integration there, but she also has an amazing background. She's been in the industry for 25 plus years. She's worked in traditional triple-A console development, and then she's also worked in a bunch of different industries and across media and entertainment.
00;01;41;10 - 00;02;05;12
Crystin Cox
She is going to be talking with us about her experience going on this journey to LiveOps and her own personal experience with LiveOps. Really interesting stuff. Jill comes from a background sort of product marketing. So that's sort of where her point of view comes from but she's, as I said, had so much experience across all of these industries, all of which have been deeply touched by liveops. I'm super excited about getting to chat with her.
00;02;05;16 - 00;02;34;08
Crystin Cox
So let's dive right in. [music] It's super awesome. I'm so happy to get a chance to chat with you today because over the course of doing this show, we've talked to a lot of people who have talked about how liveops has changed in the games industry. And they'll often sort of get a hint around to...and all this other stuff is happening out in the world of media entertainment too.
00;02;34;17 - 00;03;00;23
Crystin Cox
But you've really come across so many different industries and brought that expertize as a through line through that. So I'd love to hear you talk a little bit more and I think we'll go back and talk a bunch about Glu at some point, but talk a little bit more about the more recent time that you spent in media and how your experience and games inform that and sort of what you're bringing back to the gaming space now.
00;03;01;00 - 00;03;24;22
Jill Braff
Yeah, for sure. It's a great question and a good place to start because when I sort of moved sideways from gaming, it was never to leave games. Who could leave gaming? But what I realized was that once mobile was introduced the sort of, you know, engagement focus that we think about in games was starting to become apparent in other businesses.
00;03;24;22 - 00;03;58;14
Jill Braff
So Home Shopping Network, for example, had been around for 35 years and has, it was built on a fan community and actually looked a lot like a game in some ways. Me, if you think about it, every day there's a new daily offer and it launches at midnight. And what I realized was that the fans of HSN or QVC they sort of think of the hosts and the guests as almost characters and they, they know almost immediately whether it's a good offer or a bit of a dog.
00;03;58;14 - 00;04;25;10
Jill Braff
And so a lot of the business was sort of launching at midnight, seeing what happened in what was called the A hour between midnight and 1 a.m., and then frankly adjusting the rest of the day in real time based on that feedback. And the whole business was predicated on dollars per minute. Think about that for a minute. It's literally a run rate by the minute business.
00;04;25;10 - 00;04;52;05
Jill Braff
And I was really intrigued about that because I felt as if I understood that sort of relationship and a totally fundamentally different way of monetizing an audience that it would and end up bearing good fruit back in gaming And it was really interesting to sort of understand the rhythm of the business, which is based on merchandising, not anything to do with you.
00;04;52;06 - 00;05;04;14
Jill Braff
Let's say a mission or a level. And yet, frankly, a lot of the rhythms of what one did to manage that business felt a lot like what we see today. In games as a service.
00;05;04;28 - 00;05;38;04
Crystin Cox
Yeah, I think it's so interesting because when I've talked to a number of people over the last couple of years about LiveOps, there's been a lot of interest outside of games. People will look into our industry and say, Wow, you're doing something right. There's all this engagement. We want engagement, right? Like we want all this engagement and when I hear you describing that experience with the home shopping, it's like in order to have that, you kind of have to have this very deep relationship with the community and you have to open yourself up to that relationship with the community.
00;05;38;11 - 00;05;58;09
Crystin Cox
I can imagine that that might have been a little easier for them because it really is a direct relationship, right? Like, what are we doing? We're like showing you something and then you're going to buy it. But as you maybe moved over to Ellen, like, was it was it more difficult for them to understand, like, how to have such a tight relationship with their community?
00;05;58;20 - 00;06;34;24
Jill Braff
Yes and no. And I would I would say is, you know, even I would say that at each business, every business has its challenges. And so at HSN, maybe it wasn't, as you say, it wasn't a challenge to sort of embrace the community because you are right. It was really predicated on, hey, this is a relationship. I mean, what I was struck by is way back when it started, early cable access, the whole reason it worked is lonely people would call in the middle of the night and have someone to talk to and you could live talk to the show.
00;06;35;03 - 00;06;58;01
Jill Braff
And that sort of loop as we think about it in gaming terms was there. And that was you're right, that's sort of what we built the digital business on top of was that truth? But they really struggled more with the technology and they were so TV focused that this idea of somebody shopping from a mobile device a decade ago was a really new concept.
00;06;58;01 - 00;07;28;00
Jill Braff
And so the transformation was more about thinking about it as a 360 around the consumer and getting them to think that way. Where is it? Ellen again, similarly, they were very grounded in a television culture and yet had a huge audience on YouTube. Or Twitter or Instagram. Instagram was earlier than but the question was, well, what's the worth of that?
00;07;28;06 - 00;07;28;15
Crystin Cox
Right.
00;07;28;15 - 00;08;04;29
Jill Braff
Should we be monetizing that? Or that's nice, those millions of people, but aren't they just going to come watch a television show? And so part of my role was, hey, they're never going to watch the television show. They're going to watch it in clips on their phone. And so a lot of what has changed, I think, is the mentality of the rhythms of business, embracing all of the community, wherever the community lives, valuing that participation and then thinking about how to build on ramps so that one plus one literally equal three.
00;08;05;02 - 00;08;34;20
Crystin Cox
Right. It is fascinating to think about so many of these disparate industries seem so different. But ultimately, we're all looking to grow communities, right? We're all just looking to grow communities. And figure out like how can we engage them and sort of meet them where they are. But I would think, you know, going back to your time at Glu, when mobile gaming was first starting, we were in a very different place as an industry.
00;08;35;17 - 00;08;54;11
Crystin Cox
I'd love to hear you talk a little bit about what it was like as this big disruption sort of came into the gaming space in the West. And eventually I think in our minds now we think mobile comes and then fast follow here comes free to play.
00;08;54;15 - 00;09;21;23
Jill Braff
Yeah, right. And, and just to remind like just to bring it up, I sort of grew up like everybody else in games. You know, I shipped box products and then moved on to the next one. I worked on all different platforms and I actually think the web was the first inkling we got. So being between Glu, you know, leading gaming and Glu, I worked at Ancestry as their first head of marketing briefly, very briefly.
00;09;22;04 - 00;09;55;22
Jill Braff
But it was really interesting because this was the first time that you had sort of this direct marketing spend and this idea of like, I'm going to spend money and then there's an expectation that I should be able to measure it. And that really started to shift quite a lot because people want to follow their dollars. And you know, this whole concept now that we think about of top of funnel and retention and where people are in their relationship and their consumer journey, these are all very new concepts.
00;09;56;11 - 00;10;17;21
Jill Braff
You know, just now I guess it's been almost 20 years since we've had like a real web 1.0 business, but in the beginning it was pages and search, right? And so you were just trying to follow the people. So at Glu we were in a shipping mentality, a sort of box mentality because we didn't think about games as a service quite yet.
00;10;18;05 - 00;10;46;03
Jill Braff
But what started to change your right was Facebook launched and got bigger and because of Facebook's whole business is predicated on ads. People were realizing that they could spend money on Facebook and acquire digital users on other platforms. And I think that was really the big shift that started to happen. So it affected things at Glu, but really at the same time what's happening with social gaming was starting.
00;10;46;13 - 00;11;14;00
Jill Braff
You know, folks like Zynga were starting to play on Facebook and then that met up right at the same time with the with the iPhone launch and so iPhone really changed things because it enabled a platform for mobile. Before that, everybody was on a different device. If you recall, you were going to your carrier's store. So this was a pretty huge sea change.
00;11;14;07 - 00;11;45;13
Jill Braff
And I think once we had iPhone plus Facebook marketing that's really when people started to get into much heavier quant around where are my users coming from, how much my spending to acquire, how long are they staying? So marketing sort of moved, I think more quickly. And then we started to realize, hey, well, if I'm acquiring all these users and they're not staying in the game, we have a fundamental math problem as well as a fundamental design problem.
00;11;45;13 - 00;12;11;25
Jill Braff
And so, you know, I would you know, now we think of as this combination of product marketing, design, developer and good analysis, you know, that tight sort of team work that needed to happen that, you know, we definitely saw that beginning to emerge during the time at Glu. And part of it was just because you could, you could actually measure.
00;12;11;25 - 00;12;16;15
Jill Braff
So it was like giving you a roadmap to what is working and what isn't.
00;12;16;24 - 00;12;43;29
Crystin Cox
Yes. It's fascinating. During that time, I was in working in Asia on free to play MMOs and it was such a different design culture because there had not been consoles, like that had not been a sort of environment reality there. And so there was a just a very different design culture that had come up that sort of looked at things quite differently.
00;12;44;00 - 00;13;11;16
Crystin Cox
It was fascinating watching the wave hit the West when it didn't have that situation, like the design culture was what it was. But then here comes both social and mobile which felt a little freed from that traditional Western design culture. And as you said, maybe it was more interested in embracing the idea that, like, we can measure this, let's go build a bunch of processes around that.
00;13;12;00 - 00;13;30;23
Crystin Cox
When you started interacting with the traditional boxed product, you know, traditional Western game development, you know, post that switchover, like how did you find their sort of adoption of those practices and their relationship to data?
00;13;31;09 - 00;13;54;17
Jill Braff
Yeah, I think, you know, what's interesting is everybody loves data. But what I found in the early days was there were a couple of different problems. One was sometimes people overread on the data or they misinterpreted the data or the data was bad. So, you know, I used to say, like, the only thing worse than no data is bad data, right?
00;13;54;18 - 00;13;54;29
Crystin Cox
Yeah.
00;13;55;14 - 00;14;20;08
Jill Braff
And so I've always been a fan, even to this day of sort of marrying the quant with qual and understand making sure that you understand you know, what is actually happening in the minds of the people. And what I have found is that sometimes there is some nuance there or some reason that it's different than what you thought it was.
00;14;20;08 - 00;14;44;16
Jill Braff
And so any time I just think anytime that you're creating products and you're not talking to your users, there are assumptions being made and that's where you can get into some difficulty. So, you know, the change from box product to more or ongoing live services, I think everybody embraces the Let's have the insights. And then what's difficult for creative people is they're creative.
00;14;44;16 - 00;15;07;22
Jill Braff
So sometimes you know, they want to do things because they are being innovative and the, you know, the user, the data may not always reflect it right away. And so it's sort of discerning, I think, what is actually being told to you that remains truly like the art as you call this podcast, right? The art behind the science.
00;15;08;18 - 00;15;14;05
Jill Braff
Hopefully that'll never go away trying to. There is some art to it for sure.
00;15;14;11 - 00;15;25;08
Crystin Cox
I think we're not in too much danger right now anyway. Right?
00;15;25;08 - 00;15;57;15
Crystin Cox
[music into ad break] The Art of LiveOps is presented by Azure PlayFab. PlayFab is just one of the many Microsoft tools that game developers use to build games on any platform. From the industry leading development platform, Visual Studio, to the award winning physics of Havoc, Microsoft makes the tools that power game creators. To learn more about all the ways Microsoft can help you on your development journey, head over to MicrosoftGameDev.com to find documentation, pricing and other developer resources. That's MicrosoftGameDev.com [transition music]
00;15;57;15 - 00;16;17;11
Crystin Cox
You're listening to The Art of Live Apps. Welcome back today. We are talking with Jill Braff who is talking to us about her experience across a multitude of industries, a lot of deep experience in gaming, but also other parts of digital business
00;16;17;11 - 00;16;30;10
Crystin Cox
and entertainment and media. Really fascinating insights just about the way Liveops has touched all these disparate industries over the last 20, 25 years. So let's dive back in and keep this conversation going.
00;16;33;18 - 00;16;54;27
Crystin Cox
I'd say data is amazing. And yeah, like and even like there's I think, an idea that designers are like resistant to data. But every designer I've ever known as a huge data nerd, like, of course, you want to see how the numbers are changing. Of course you want to see what people are doing with your content. But it is challenging because it is only it only looks backwards, right?
00;16;54;27 - 00;17;29;21
Crystin Cox
Data can only... the actual raw data itself is always backward looking. And the trick of using it to forecast forward is very challenging. You know, if we had that totally figured out, I mean, that would be amazing. But we often really don't. How you know. How has your relationship with data changed? Do you feel like, as you said early on, there was a lot of worry about bad data, but, you know, now are you like there's some specific kinds of data that you really love to work with or things that you think are the most useful that you see consistently?
00;17;30;09 - 00;17;58;20
Jill Braff
Yeah. You know, I think, you know, the thing I've come to understand is that every product or service is a little unique unto its own. So I don't have one blanket go to approach. I think my main, you know, the one or one of it to me is ensuring that you think about it at the ground floor, you know, whether it's retention you're thinking about or for some user experience or monetization.
00;17;58;20 - 00;18;28;09
Jill Braff
It just to add it later never seems to work. And so there's so much enthusiasm for the build stage versus, you know, the editing, anything is not as much fun as building it. So, you know, taking the time to think through what are the events we're going to want to measure? The hypothesis and getting the telemetry right as you're building, I think is the probably the most painful part, I think, or like the tug of war.
00;18;28;21 - 00;18;46;29
Jill Braff
But I've just seen too many things come to pass and then afterwards say, Oh, gee, I wish we knew the thing and we just don't have the information. And that just seems to me a bit of a shame. And I come, you know, I come up through product marketing ranks. That's sort of, you know, how I got here.
00;18;47;07 - 00;19;11;10
Jill Braff
And so that's a I'm still a nerd for that part of the process. Like, I love the feedback that you get from the data plus some insights because I just think it's telling you what's working. There's so much enthusiasm you can see for when things pan out. And yet it can also really point to some bright spots that you might just be missing because you're too close to it.
00;19;11;15 - 00;19;32;29
Jill Braff
I mean, I have a funny story. It's not particularly LiveOps. It's more on the qual, but, you know, back at Glu I remember we were a tiny company and we were making a really big bet to do a cross platform launch at that point on mobile things would just go live whenever they were ready and I thought that was just dumb.
00;19;32;29 - 00;19;54;11
Jill Braff
I was like, Wait, why are we giving up all of our good marketing that we know about? Let's make this an event. And so we were of course racing to get it done. And right at the end we threw together just a quick focus group. What are we missing? And the whole point of the game was that our big innovation was you could get out of the car and walk around the world,
00;19;54;11 - 00;19;54;21
Crystin Cox
Right.
00;19;55;02 - 00;20;14;20
Jill Braff
But it was a tiny little mobile game. It was probably like 256k the whole file. And so, of course we started you in the car, and the biggest thing that came out of Focus group was no one knew you could get out of the car. And so we literally changed it. We started it where you're standing next to the car, and the very first thing you do is get it
00;20;14;20 - 00;20;14;25
Crystin Cox
right.
00;20;14;26 - 00;20;35;00
Jill Braff
And by making that tiny change it unlocked just a whole bunch of goodness because it was seen as a revolutionary thing on mobile. And it's so trite and silly now to think about that it's a great example of just sometimes, you know, you can have it right in front of you and not see it.
00;20;35;10 - 00;20;57;00
Crystin Cox
I think that is fundamentally like one of the stories that we hear over and over again in Liveops, right, is that so much of it is it's not necessarily technology, and especially at this point where that technology is somewhat ubiquitous. Like, you know, you don't get to stand up on a stage and say we're online. Like, yeah, of course you are like everyone, right?
00;20;58;00 - 00;21;30;23
Crystin Cox
And instead it is so much more about that feedback with your community. And I think that story also like highlights for me, something that was very much happening at the time and continues to happen in the industry is this big expansion of who is playing games. And this huge demographic expansion has led to a world where we can't really make games like we did 25 years ago, where we say like, Well, we're all going to sit in a room with a small team, we're tastemakers, we know what gamers like because we're gamers and what is gamers.
00;21;30;23 - 00;21;41;28
Crystin Cox
Everyone is a gamer at this point. So I think you have to have some of that interaction with your community to even know who is your community and what do they want.
00;21;42;07 - 00;22;06;04
Jill Braff
Well, it's so true. I mean, think about what mobile and free to play taught us about first time user experience. Right? And, you know, it used to be that you could just build for your fan base and know that they were sort of going to get it with a nod and a wink sort of thing. And then that was going to fundamentally support your game into its next evolution beyond that.
00;22;06;04 - 00;22;29;03
Jill Braff
and now, I mean, even Game Pass has changed the Xbox world so much because you have people I think they call it like a hummingbird effect, right? You have people who can go into anything and try lots of games, which is wonderful for the player and exactly how it should be, but really forces design teams to think about what is that feel like in the first few minutes.
00;22;29;03 - 00;22;53;08
Jill Braff
And certainly we experience that in mobile for all the mobile products I've worked on have been that hey, what will somebody think in the first minute or two? And yet also, you know, there's so many preconceptions of just preconceptions of what it is to be even a mobile game. You know, people think that, oh, mobile game is for 1 to 5 minutes.
00;22;53;19 - 00;23;29;01
Jill Braff
Even when I was at Glu well over a decade ago, that was just not the case. People were sitting on their couch, but they were playing for very long periods. It was just that was the device or experience they were looking to have. And so again, without LiveOps, without that data, letting us know, hey, people are playing this game multiple times a day, but more like 30 minutes a session and that changed design fundamentally because you couldn't just say, Oh, this is a three minute cycle and then someone will be done if that wasn't what they were looking for.
00;23;29;09 - 00;23;54;09
Crystin Cox
Yeah, and I think that we're going through a similar process now when I will still hear lots of people say, Well, mobile is casual, mobile is for casual players. And then I'm like, there are people who have been playing Clash of Clans for five years with the same clan. They've had deep community engagement with that game, and deep investment. That's just one example.
00;23;54;09 - 00;24;11;23
Crystin Cox
It's obviously a very successful example, but it, you know, those ideas, those desires to kind of create really simple little boxes that we can sort of put platforms into. They just keep getting shattered because it's just not the way that players work.
00;24;11;25 - 00;24;53;04
Jill Braff
Right? And I mean, the bigger the industry has gotten, right now, we talk about two and a half billion players, and there's no way that the habits would be the same. But truthfully, I mean, I get it. People want to sort of try to wrap their mind around it. But I gave a keynote at the L.A. Games conference over a decade ago just talking about how there was nothing casual about casual games and casual was like a four letter word in the gaming industry, and all you need to do is spend time with people who play those games and they're just completely big, big fans. And like you said, huge investment and time and money and just energy and emotional connection as well.
00;24;53;04 - 00;25;18;26
Crystin Cox
And sometimes I think, you know, this is a journey that we're as an industry very much on right now. There, sometimes it can be hard not to hear a coded idea inside of that, right? That like well, something like, for instance, well, games for women must be casual because women are casual gamers and so even though we hear these messages around, there's 2.5 billion gamers.
00;25;19;10 - 00;25;49;20
Crystin Cox
Everyone plays video games in the world at this point. Obviously, it's incredibly diverse. We still do struggle with that. Diversity is a challenge for us in the industry. And it's not just a struggle around diversity of creators voices, but a struggle in things perception like that, where we say things like, well, this is casual. And you know what that really maybe means sometimes all the time is it's for it's for a feminine audience or it's for an audience that doesn't have this particular kind of interest.
00;25;49;25 - 00;26;25;22
Jill Braff
The way I have started to wrap my brain around this more recently was if you walked into a restaurant and you said, How many people played Halo? You know, some percentage of people would raise their hand if you walked into that restaurant and said, how many people have played solitaire? Most everyone would raise their hand. So I think of it, I'm sort of losing the casual moniker and thinking of it more in terms of just more mass market and that certain things are just more readily understood and other sorts of experiences are more specialized.
00;26;25;27 - 00;26;42;03
Jill Braff
And I think that helps me get away from some of the old labels and think about things just more based on size of audience and certainly not depth of interest, but just overall, what is the...who are you reaching?
00;26;42;03 - 00;27;04;01
Crystin Cox
Yeah. And I mean, not to get all like, you know, rah rah Xbox, but it is something that I think is very interesting about the Xbox approach right now, which is vey much this attitude of We shouldn't even really be thinking about it like, you know, hardcore mid-core. We're really thinking about there are lots of players out there.
00;27;04;01 - 00;27;25;10
Crystin Cox
Where are they? What are they doing? You know, embracing the idea that, you know, what people who raised their hand for Halo may also raise their hand, you know, for Words With Friends. They may raise their hand for, you know, narrative and mobile narrative games. They're not like this totally separate population. Just got to think of them as people.
00;27;25;14 - 00;27;47;29
Jill Braff
I'm so glad you brought this up. I think this is like the hidden gem of truth in gaming in that, you know, I think about myself, but I was telling someone the other day, right now the mobile game I'm addicted to is Duolingo. I've been like brushing up my French, but they totally gamified it and got me addicted, and my console play is more core than it's ever been.
00;27;48;00 - 00;28;17;15
Jill Braff
And so I do think I think the on ramps with things like Game Pass, not to be an Xbox commercial, but it has really changed things. And I also do I don't know, I feel like the next ten years of gaming are going to be the broadest we've seen some of because generationally we've shifted and some of it because of convergence of screens and some of it just because there are now more different types of games on all screens.
00;28;17;15 - 00;28;51;19
Jill Braff
Mobile 10 years ago wasn't as big for core and mid core games as it is today. So you know, the rivers are coming together, but they're also just you know, they're changing in a very positive way, I think, for the players. So it's good. And I do think LiveOps, just to bring it back, plays a role in that too, because the feedback we're getting is also starting to change some of the age old thinking because we're getting real data that's telling us different information than maybe we knew in the past.
00;28;51;19 - 00;29;21;25
Crystin Cox
Yeah. It's such an important I think part of the Liveops process is you want both. You said it, like you want to qualitative and quantitative and you also from a creative perspective, you want to bring vision and direction and then also meet your players halfway and iterate with them. And I think something for me that is fascinating, I'd love to hear your thoughts on it is I've seen this repeated over and over again for game teams where before launching a game, we'll do psychographics and we'll get together and be like, Who's our player?
00;29;21;25 - 00;29;48;17
Crystin Cox
And we'll come up with these ideas, but who the player is. And usually they're very archetypal before the game launches. They are these really big ideas about, you know, this is our PvP player and this is our PvE player. And I've never worked on a game where those archetypes, those psychographics matched actual data. They came back for like actual experience.
00;29;48;22 - 00;30;00;14
Crystin Cox
The behavioral data that comes back, they're super useful. This is not to say don't do psychographics, they're super useful for the design team, but it is fascinating that but it doesn't quite match up with the reality when we get into the live game.
00;30;01;20 - 00;30;34;23
Jill Braff
Yeah, yeah. I mean, crystal balls are hard to come by, but I think that, you know, you bring up for me, there is actually an experience I had at HSN. So here you have a multibillion dollar direct to consumer business with 30 years of data. Pretty rich under the hood, really. And one of the most powerful things that happened while I was there was we did a big segmentation study project, whatever you want to call it, and we segmented our personas.
00;30;34;23 - 00;30;55;17
Jill Braff
So it was psychographic, but it was also buying data and that actually really unlocked probably more successfully than anywhere else I ever saw because it was so rich and, and it did it. And it also it was a little orthogonal in thinking sort of what you were saying, like I could play this game, but also that game, you wouldn't think of me as the same person.
00;30;56;02 - 00;31;19;19
Jill Braff
And it really did create more of a roadmap for how we should direct new users. Hey, you're here for the first time. Here are some products you might like. We were able to really see what the consumer journey looked like. And similarly, after HSN, I was involved with another direct to consumer business called Paula's Choice Skin Care Business.
00;31;19;19 - 00;31;53;10
Jill Braff
So I like these e-commerce businesses because the performance marketing is the backbone of the business. And I think it keeps me sharp in terms of what's happening with gaming. And I think those sorts of cohort data analysis and sort of combining psychographics with the rich sort of quant under the hood, that starts to become really insightful versus like, hey, let's just get in the conference room and sort of come up with what we think it is.
00;31;53;17 - 00;32;06;17
Jill Braff
I could see why that is informative, but maybe a little bit more difficult. But this came from the actual data and putting the nuggets together and it ended up being extremely powerful.
00;32;06;17 - 00;32;27;26
Crystin Cox
I can imagine that I think that is like that's the magic, right? When you go like, let's bring them both together, like it's not throw out all of your creative insights or you know, your instincts or your taste and it but it's also not ignore the actual experiences that our players are having, which are super important to them too.
00;32;27;29 - 00;32;53;27
Crystin Cox
I know one game I worked on, we did a big segmentation analysis after the fact and you know, at first it was like, oh my goodness. Because I think we identified like 17 unique, you know, user segments and player types and engagement and we're like, how do we even wrap our heads around that? And that was where we like No, this is where we bring it together with that more traditional marketing, you know, understanding of like who your customer is to make that useable.
00;32;54;03 - 00;33;16;02
Jill Braff
Right? And also what we did was we just, you know, we sort of discarded. A number of them were much smaller. So, you know, and the beauty of shopping and gaming is if you remove the verb and you think about sort of the mental journey, they're pretty similar. I'm on a hunt. I'm parsing through. I've acquired. I want to tell people about what I got.
00;33;16;02 - 00;33;16;14
Crystin Cox
Yeah.
00;33;16;14 - 00;33;35;16
Jill Braff
There's a lot of sort of richness that's similar. And so the point was we were able to sort of, similar to games, look at sort of our best customers and then the people, you know, nearest them, the three or four rungs down the ladder. And, and that business similar to gaming is sort of built on your best customers.
00;33;35;16 - 00;34;07;21
Jill Braff
And so we really thought about how do we create more of them? How do we get more people up into the higher spending ranks, but higher loyalty, more engagement, etc. And by looking at it that way, we unlocked some insights like, hey, those are people who are on multiple screens. They come more often. I mean, not necessarily rocket science, but it did give us some things to glom onto that were more nuanced and that was really helpful as well.
00;34;07;21 - 00;34;30;03
Crystin Cox
Yeah. So I'd love to hear some of your thoughts about the future. You know, like where do you, where do you see us going? You know, I mean, everyone's got predictions, but I'm curious to know like so what do you see are our big challenges in the next couple of years as an industry? Especially as it pertains to LiveOps and where there's this big opportunities you see coming down the pike?
00;34;30;10 - 00;35;01;07
Jill Braff
Yeah, I, you know, again, my crystal balls is rusty, but you know, I think it's a fascinating time for games. Seriously, I've probably never been more bullish about gaming. And I think that, you know, the next chapter is we've truly finally arrived at this concept of converged screens that I could go and play might be a slightly different experience, but I can play my favorite games or types of games on whatever screen I want in the moment.
00;35;01;07 - 00;35;27;10
Jill Braff
And that's a lot of power for the player. And so now I think, you know, there'll be more feedback from players than ever before. And the burden on the design and development teams is, you know, where to take the storytelling. How much should all screens, for my experience, be equal? Is one more of a companion thing? It won't be a one size fits all at all, but I think it leaves a lot of room.
00;35;27;10 - 00;35;49;25
Jill Braff
You know, on the one hand, we have new worlds being created. Like I'm thinking about Starfield later this year. On the other hand, we have worlds like Roblox where people are now creating new worlds and new types of play on Roblox. I heard someone the other day talk about that platform in the way we used to think about iOS, they thought of
00;35;49;25 - 00;36;16;27
Jill Braff
it as like lightweight, inexpensive, and able to get a signal on whether they have an audience pretty quickly. And I thought that that's a shift that I hadn't really thought about before. And then, of course, we have all the noise of NFTs and metaverse and everything else that is in the news. And my sense of that is for those of us who've been in gaming, this is more of an evolution, not a revolution.
00;36;16;27 - 00;36;37;23
Jill Braff
I think especially near term. You know, we'll see how things truly play out. There's an awful lot of legal and finance work to be done around entitlements, but I think what's you know, when I think about it on a macro sense, I had somebody say to me, you know, Web one was about pages and search, and web two was about social and moving to mobile.
00;36;38;13 - 00;37;00;14
Jill Braff
This concept of web three is around tokens. And this idea that you could create experience. I could come in, I could pay you directly, I could then trade with someone else. I think the freedom for the player is what people are excited about, and that you don't have just ad based businesses or just businesses predicated on middlemen. We'll see.
00;37;00;14 - 00;37;21;09
Jill Braff
There's a lot of room, but for those of us who have been in gaming for a long time, these virtual worlds and my currency in the world, this is not a new concept. And so that just makes me excited because however that pans out, you know, the idea that I could do work in one game and it could pay off and be in my wallet somewhere else, that's pretty cool.
00;37;21;10 - 00;37;21;26
Crystin Cox
Yeah,
00;37;21;26 - 00;37;26;03
Jill Braff
but I, I don't, you know, it's unclear how exactly that will go.
00;37;26;03 - 00;37;59;06
Crystin Cox
Yeah, absolutely. I mean, there's, it's an exciting time because there's a lot of talk about it, but it's also sometimes a hard to parse time because there's so many voices that are saying so many things about this. And it can be interesting because I agree, like, I think back to the early 2000s in China and Korea and some of those very, very long standing now you know, 20 year old player to player marketplaces and go like we're not flying blind like it's not totally unexplored territory there are some things that we can look to.
00;37;59;09 - 00;38;24;29
Jill Braff
That's right yeah. Second Life and IMVU. I mean we've been seeing this before. I even remember people were paying lots of money for accessories. Right. You know it's so funny wear my fake shoes for a fake performance in a fake race that was worth something to people. So, hey, I am all for it. I also think the cloud is going to bring about a lot of freedom or gaming, just ease of getting into something.
00;38;24;29 - 00;38;50;23
Jill Braff
You know, the idea that I would have to set up my console and download something over night will increasingly be it'll be easier to get into experiences which I think will bring a lot more trial. Obviously, that depends on where you are in the world. But sitting in the Western United States with 5G around the corner for real, disrupting airline flights and all that sort of stuff, it's going to be really wild.
00;38;50;23 - 00;39;01;12
Jill Braff
To see because, you know, even today there are some games and I'm like, I don't want to wait for that download and if that goes away, it'll be easier to taste test.
00;39;01;13 - 00;39;18;20
Crystin Cox
Absolutely. Yeah. I mean, I, I think about the things that ten years ago would have seemed like absolute magic and now I don't even think about them. It's like, well, of course my whatever progress I made in this game will just be available without me stopping and creating is of course. Why wouldn't it be it? I think that would be incredible to me, you know?
00;39;18;21 - 00;39;30;26
Jill Braff
That's right. I mean, you know, even at HSN, one of the craziest innovations we did about a decade ago was that you could start shopping on your PC, jump in your car and your same cart would be there.
00;39;30;26 - 00;39;31;06
Crystin Cox
Yeah.
00;39;31;06 - 00;39;32;28
Jill Braff
And this was like a revolution.
00;39;32;28 - 00;39;33;11
Crystin Cox
Yeah.
00;39;33;11 - 00;39;39;26
Jill Braff
And of course, all it was was a warm cookie, but it really made a big difference.
00;39;39;26 - 00;39;58;17
Jill Braff
And improved the experience for people. And that's the thing. You know, obviously this whole piece of LiveOps is about making the community feel stronger, be happier, spend more time with your product and service and just give more feedback than can come in through an 800 call center which.
00;39;59;14 - 00;40;02;00
Crystin Cox
Which nobody does anymore.
00;40;02;00 - 00;40;02;20
Jill Braff
Exactly.
00;40;02;20 - 00;40;04;12
Crystin Cox
Gosh, I almost miss them, though.
00;40;06;26 - 00;40;29;01
Crystin Cox
So we're running out of time here. But there is one question we like to ask everybody who comes on the show. Can you share a LiveOps disaster with us? It's so good, I think, for that perspective. And people just love to know that even people with amazing experience who've done so many wonderful things, we've all got those stories.
00;40;29;19 - 00;40;53;09
Jill Braff
My disaster was sort of painful, but I'll share it because that's what, you know, we all do. So, back to my HSN experience. We so live 24/7 and 364 days a year. So never down. And yet we were relaunching our website, which, you know, no one expected to go perfectly.
00;40;53;09 - 00;40;54;02
Crystin Cox
Right.
00;40;54;02 - 00;41;22;22
Jill Braff
But the problem was that as we launched, actually everything was okay at first, but then there was trouble and we didn't know where and I just remember being up in the middle of the night because that was the nature of the business and we had a really good, you know, ups escalation process like you would expect, but it was still pretty rudimentary because
00;41;22;22 - 00;41;44;11
Jill Braff
we were in a really web service and so suffice it to say that my development team told me it was going to take like four months to fix it. And we had this meeting with the CEO in the board room where she was not happy. And I recall she put me on the spot of like what was the plan?
00;41;44;11 - 00;42;09;02
Jill Braff
And I delivered this terrible news that it was going to take us a lot longer than anybody wanted to fix it. And of course, she right there was like, well, that's completely unacceptable. And then my development team was like, oh, and she wanted it in like four weeks, not four months. And so I was still toeing the line for the developers and then they basically caved on me in the room and was like, Yeah, we'll do it in four weeks,
00;42;09;02 - 00;42;09;23
Crystin Cox
Oh no!
00;42;09;23 - 00;42;41;15
Jill Braff
And so basically the whole thing just taught me a lot about what's your plan? What's your backup plan? When things go wrong, how are you going to get a solid new plan together? I was frankly then thrilled they were going to fix it in four weeks. But the point was just that the whole thing was sort of really painful and the other part about that story, too, is I think when you're in I'll call it the fog of war, right?
00;42;41;15 - 00;43;08;27
Jill Braff
You're in a launch and you're trying to everything is just so noisy and you're trying to figure out what is real. I think the actual hardest part of the LiveOps process is right then because some of the data is just because it's so new and some of it is sort of telling you where your problems are and separating those two things and trying to really understand what is because it's new and what is really a problem.
00;43;09;02 - 00;43;18;27
Jill Braff
And getting everybody on that same page. I mean, I don't know a great solution for it, but I think that is like when I think about the hardest part of the LiveOps piece, that is it.
00;43;18;28 - 00;43;36;18
Crystin Cox
Yeah. I was talking with Alex Hunnisett on an earlier episode, who runs the LiveOps piece for Sea of Thieves, and he was saying that really what it is, is like I was asking like, what have you learned? How are you going to roll out in the next game? And he was like, I think like we're just better at knowing what's important.
00;43;36;25 - 00;43;54;26
Crystin Cox
Like, that's really it. Like, that's, that's what you really gain. As you said, the hard part is when everything is on fire, or even if things are not on fire, you just spun something new up. How do I know if this signal means anything and it takes a long time to develop? I think processes and instinct around it.
00;43;55;08 - 00;44;15;17
Jill Braff
I agree. And that's where the qualitative has also been helpful. I know it's small because you never talking to a multitude, but I just found like when there is a pattern and you can't quite put your words to it, just talking to the real consumers at least puts you on the right path. It doesn't, if not just the light bulb going off.
00;44;15;23 - 00;44;39;06
Jill Braff
But yeah, I've almost never had a problem. I also think you can find some gems through the data. I know we're at time, but the one last thing, story I'll tell, is when I was at Ellen, we had Heads Up, which I think everybody has played probably on a mobile device, and it was great, but we also launched a free to play game called Psych, which was not as widely known.
00;44;39;20 - 00;45;00;27
Jill Braff
But interestingly, because of our LiveOps data, we were able to tell that more people oftentimes are playing Psych than we're playing Heads Up in terms of concurrent players. And so a game that maybe because of the charts or because we weren't making a ton of money and like you might have just abandoned it. It was a really fun game.
00;45;00;27 - 00;45;08;06
Jill Braff
So we went and spent more time on it because we knew that there was some thing there that players really liked.
00;45;08;06 - 00;45;25;25
Crystin Cox
Yeah, that potential like that can be so rough because it really is there's so much value in the individual experience that can't really be translated to aggregate data. There's tons of value in aggregate data that can't always be translated individual experience. I mean, you really have to have both.
00;45;25;28 - 00;45;28;08
Jill Braff
I fully agree with that.
00;45;28;12 - 00;45;31;13
Crystin Cox
Well, this has been fantastic. Thank you so much for coming on.
00;45;31;19 - 00;45;34;08
Jill Braff
Thank you so much for having me. It was great. I enjoyed it.
00;45;37;10 - 00;45;39;17
Crystin Cox
Thanks for listening to the Art of Life Arts podcast.
00;45;39;23 - 00;45;44;22
James Gwertzman
If you liked what you heard, remember to rate, review and subscribe so others can find us.
00;45;45;00 - 00;45;49;18
Crystin Cox
And visit PlayFab.com for more information on solutions for all your liveops needs.
00;45;49;27 - 00;45;50;23
James Gwertzman
Thanks for tuning in.