WEBVTT 00:00:02.966 --> 00:00:18.353 Welcome to this week's edition of the Dan Rayburn Podcast, the show that curates the streaming media industry news that matters most, unvarnished, unscripted and providing you with the factual data you need to know, without any of the hype, the pulse of the streaming media industry. 00:00:22.140 --> 00:00:25.330 Welcome to the Dan Rayburn Podcast here with co-host Mark Donegan. 00:00:25.330 --> 00:00:34.560 End of the year, mark last podcast and it's a special one because we have a lot to cover from the Netflix Christmas NFL games. 00:00:34.560 --> 00:00:38.792 But you know, end of the year, we're recording this on Saturday, december 28th. 00:00:38.792 --> 00:00:42.429 I'm going to apologize in advance for my voice because I've been talking all day. 00:00:42.429 --> 00:00:48.024 You've been cheering too loud On the weekend Cheering is definitely not what I've been talking all day You've been cheering too loud. 00:00:48.064 --> 00:00:50.286 On the weekend Cheering is definitely not what I've been doing. 00:00:50.286 --> 00:00:53.609 I care less about football Far too busy to care about that but just a lot of calls still at the end of the year. 00:00:53.609 --> 00:01:07.743 A lot of people reaching out asking about expected things in the new year, people already talking about budgets for the new year, people looking for jobs, a lot of questions around around the nfl games. 00:01:07.763 --> 00:01:12.462 There's now some stats out which we'll go through, but full stats for global viewership don't come out on this until december 31st. 00:01:12.462 --> 00:01:19.266 That'll be after we drop this podcast, so we'll have to uh cover the global stats on on the next one in january. 00:01:19.266 --> 00:01:25.462 So, mark, let's just jump right in here to here's how we're going to break down the podcast today. 00:01:25.462 --> 00:01:28.426 We're going to go through some of the things I measured during the Netflix stream. 00:01:28.426 --> 00:01:45.441 A little bit on, specifically on latency, because for some reason, as we know, many in the industry care far too much about latency got wrong on the technical side. 00:01:45.441 --> 00:01:47.566 We're going to go through metrics that we have as of now from Netflix and Nielsen. 00:01:47.566 --> 00:01:49.712 We do have a little bit there, not super broken down as of yet. 00:01:49.712 --> 00:01:50.415 More is coming. 00:01:50.415 --> 00:02:05.349 We're also going to talk about the importance of what this means for the entire industry, because, while some think it's not a big deal, it is the progress of one, is the progress of all and Netflix did a great job on Christmas which helps all of us. 00:02:06.521 --> 00:02:17.655 We'll also talk about what Netflix has coming up and then we'll round out the podcast with a couple numbers, just for end of year in terms of viewership and some things to thank listeners for. 00:02:17.655 --> 00:02:19.366 So let's just start here. 00:02:19.366 --> 00:02:28.165 In terms of measurement, I did set up at home a setup kind of like how I do for Super Bowl a little bit less, not as many devices and whatnot. 00:02:28.165 --> 00:02:44.354 When you're testing from just one location, it's not really about how many devices you have set up, but I wanted to see what the experience was based on Roku, desktop, ios, fire TV, apple TV and then smart TVs from Samsung, lg and Vizio. 00:02:44.354 --> 00:02:51.007 I was using two separate Netflix accounts Some people were asking about that and then all devices were connected to TVs. 00:02:51.007 --> 00:03:05.828 For me, wired MacBook and iPads are on Wi-Fi and, thanks to my neighbor, I actually ran an Ethernet cable from their second floor window to my house so I could have two ISPs coming into my home. 00:03:05.828 --> 00:03:11.651 I used to have two ISPs here, but I finally dropped Verizon because it just was having too many problems. 00:03:11.651 --> 00:03:20.241 So I was running Optimum and files from the neighbor Optimum 500 megs and Verizon was a one gig connection. 00:03:20.241 --> 00:03:34.633 So a couple of just technical details here I guess we could start with was Netflix was doing two second chunks and originally for the Tyson Paul boxing fight they were doing one second chunks. 00:03:34.633 --> 00:03:40.673 So that was something being discussed online a lot in terms of what they did for the boxing and what they changed. 00:03:40.740 --> 00:03:43.149 Now, quick apology to Netflix. 00:03:43.149 --> 00:03:46.320 I originally said in my post it was three second chunks. 00:03:46.320 --> 00:03:46.883 It was not. 00:03:46.883 --> 00:03:50.932 That was a poor job of me looking at the manifest file accurately enough. 00:03:50.932 --> 00:03:56.792 I did correct that fairly quickly but it was two second chunks Subtitles. 00:03:56.792 --> 00:03:58.901 There was some information in there. 00:03:58.901 --> 00:04:03.209 The stream was using CMAP and dash HEVC H.264. 00:04:03.209 --> 00:04:05.171 There was no av1 support. 00:04:05.171 --> 00:04:06.112 That I saw anywhere. 00:04:06.112 --> 00:04:09.044 I don't think that surprises anyone. 00:04:10.507 --> 00:04:21.903 Now, interesting in terms of the stream manifest here mark that normally you can see how they're doing the encoding bitrate ladder, but in this case it didn't show the complete list of available profiles. 00:04:21.903 --> 00:04:25.752 So what was happening was they were pre-initializing the stream. 00:04:25.752 --> 00:04:31.326 It was checking your device, the browser, network speed et cetera, and it was picking the best option. 00:04:31.326 --> 00:04:35.110 So it was giving a single quality profile in the manifest. 00:04:35.110 --> 00:04:40.331 So I don't know what the max bit rate was that Netflix encoded at. 00:04:40.331 --> 00:04:43.367 The max I ever saw was just over six megs. 00:04:48.000 --> 00:04:49.262 So some, I noticed, were talking online about how they did that. 00:04:49.262 --> 00:04:57.738 Hey look, I'm no expert when it comes to encoding uh bitrate ladders and whatnot, but you know it does make sense that it's a good approach, the way netflix did it. 00:04:57.738 --> 00:05:03.988 When you're in charge of the whole workflow, yeah, as netflix is so that made a lot of sense. 00:05:03.988 --> 00:05:10.807 They were switching domains and C names in the stream based on several conditions, especially ISP IP four, six. 00:05:10.807 --> 00:05:16.031 Their player was only requesting the manifest during the initialization. 00:05:16.031 --> 00:05:21.199 Once it was initialized, video and audio subtitles are encoded separately. 00:05:21.199 --> 00:05:32.959 No surprise, because they had multi-language support In the URL. 00:05:32.959 --> 00:05:33.581 Query strings also showed that. 00:05:33.600 --> 00:05:37.084 Netflix generated the URLs to expire after 12 hours, so I don't know if that's normal or if that's something unique. 00:05:37.084 --> 00:05:39.007 I haven't looked at others in terms of that. 00:05:39.007 --> 00:05:52.363 Cryptid hash server-side authentication was being used, so typical there, so I didn't see anything super unique that they were doing Not surprised they went from one second to two second chunks. 00:05:52.384 --> 00:06:16.146 I did talk to quite a few ISPs I don't know the exact number, probably just under 10 in the US and globally during the stream, who were sending me a lot of information, and somebody asked me why I didn't put any of that out, like the boxing post boxing event, and the reason is nine ISPs don't necessarily make up a good representation of the entire experience globally. 00:06:16.146 --> 00:06:22.511 Now, some of these ISPs in the US are some of the largest, so it's definitely a large portion of the population. 00:06:22.511 --> 00:06:25.778 So it's definitely a large portion of the population. 00:06:32.379 --> 00:06:35.285 But I did see ISPs online especially some small ones, saying we had 50% of the traffic that we normally had for boxing Interesting. 00:06:35.305 --> 00:06:37.708 But then also I saw one that said, well, we were 150% over. 00:06:37.708 --> 00:06:40.932 But then you look them up and it's like, okay, but you have 200,000 subscribers. 00:06:41.454 --> 00:06:41.694 Yeah. 00:06:46.579 --> 00:06:46.980 So I'm not surprised. 00:06:46.980 --> 00:06:47.843 Uh, there is a minimum threshold of traffic. 00:06:47.843 --> 00:06:50.168 You have to be basically pushing to have open connect within your network. 00:06:50.168 --> 00:07:00.672 So I didn't want to put out any isp information or even just generally say too much, because I don't want people looking at it and going well, dan said all isps were doing 50 less traffic. 00:07:01.454 --> 00:07:13.459 Yeah, um, I did say, see a few people quote me incorrectly from the post-boxing event and I specifically said I only talked to seven and I don't know what those seven represent in terms of viewership. 00:07:13.459 --> 00:07:22.795 So what we do know, obviously from the numbers that were put out, just for US viewership, this is much smaller than viewership-wise, than the boxing event was. 00:07:22.795 --> 00:07:24.403 I think we all expected that. 00:07:24.403 --> 00:07:31.283 Yeah, not very surprising, but we'll have some additional numbers coming out. 00:07:31.283 --> 00:07:36.052 So now let's talk about things that we saw that were technical challenges here for some users. 00:07:36.052 --> 00:07:45.267 So we did have and I had this issue as well users reporting that when they joined the live stream they had to scrub to the live stream once joining. 00:07:45.267 --> 00:07:57.146 So when you click watch live, it would take you into the game, but you actually had to slide to the right to live to catch up. 00:07:57.146 --> 00:07:59.392 You have to fast forward it basically. 00:07:59.720 --> 00:08:02.730 I think it was 60 second delay right. 00:08:03.581 --> 00:08:05.509 So it was different depending on the device you're on. 00:08:05.509 --> 00:08:10.271 I didn't have this problem on fire tv, um, but I did have it, for instance, on the iphone. 00:08:10.271 --> 00:08:13.843 So, uh, that that also just. 00:08:13.843 --> 00:08:19.687 There's a couple people I've noticed online who said, well, netflix did that on purpose to relieve, you know, pressure and open connect. 00:08:19.687 --> 00:08:21.211 They absolutely did not. 00:08:21.211 --> 00:08:27.932 They did not purposely make it so that when you jump into a live stream it wasn't actually live. 00:08:27.932 --> 00:08:29.206 There's no reason to do that. 00:08:29.206 --> 00:08:37.525 That wouldn't be beneficial for users and they weren't having a problem with capacity across the network and Open Connect when it came to actually delivering this with QoE. 00:08:37.525 --> 00:08:43.285 So that was not on purpose, it was not for everybody, it was not across every device. 00:08:43.285 --> 00:08:54.543 I also saw some complaints from users who said they weren't able to cast from their phone to the tv but it's not supported right, it's not supported across all the plans. 00:08:54.745 --> 00:09:02.446 Yeah, exactly uh, it's supported in the higher tier yeah but I don't know if it was specifically was supported for nfl. 00:09:02.446 --> 00:09:04.288 Yeah, that I don don't know. 00:09:04.288 --> 00:09:07.414 But you can't do it on the ad tier. 00:09:07.414 --> 00:09:14.486 So users can connect using Chromecast built in, which was formerly called Google. 00:09:14.486 --> 00:09:29.726 Cast Allows you to cast from your mobile device, but Chromecast built in is only available on Philips Poland, sharp, skyworth, sonic, sony, toshiba and Vizio TVs, so not surprising there. 00:09:29.726 --> 00:09:31.265 Well, that's not Samsung. 00:09:32.601 --> 00:09:36.337 It's not LG, it's not TCL, it's not LG, it's not iSense. 00:09:36.841 --> 00:09:40.250 I did see some complaints from users about hey, this isn't working. 00:09:40.250 --> 00:09:49.312 And, as usual with any live event, no matter how great a job you do, you're going to have users trying to do things that, simply, the player, the device or the stream does not support. 00:09:49.312 --> 00:09:51.905 That's just the reality of it. 00:09:51.905 --> 00:09:56.779 I did see two users complaining about not working on the iPad. 00:09:56.779 --> 00:10:06.884 Now I was sending messages back on Twitter to people who were complaining about things or sharing screenshots and asking hey, what are you running? 00:10:06.884 --> 00:10:07.706 What device? 00:10:07.706 --> 00:10:09.490 What iOS, what ISP are you on? 00:10:09.490 --> 00:10:18.990 And, Mark, I was surprised how many were actually sending me emails or whatnot, which was, I guess, kind of not what I was expecting from the average person watching. 00:10:18.990 --> 00:10:21.346 Yeah, but a couple of things I noticed. 00:10:21.346 --> 00:10:38.370 For two people was Netflix does not support iPads running iOS 16 or earlier, so two people running much older iPads Okay, well, that's not going to be supportive from a player standpoint and you can't update those iPads anymore, so you're always going to have some user issues. 00:10:38.370 --> 00:10:39.960 Never surprising there. 00:10:39.960 --> 00:10:40.980 Never surprising there. 00:10:40.980 --> 00:10:59.429 We did see some issues with screenshots where people were recording and showing off that there was some massive pixelation or an error on the screen, that you need to restart your device or the player, but I did not see a large quantity of those. 00:11:00.210 --> 00:11:08.514 We don't know exactly what percentage of users had a great experience or encountered some sort of QOE issue. 00:11:08.514 --> 00:11:24.975 But the other thing I was thinking here, mark, is and this would be an interesting topic for the industry to debate, because that's really what it is the debate is what defines a poor QOE experience? 00:11:24.975 --> 00:11:31.190 So at one time, across every device platform I was on, I got kicked out of the stream once and I had to restart it. 00:11:31.190 --> 00:11:35.980 So if my let's see, I started at 12, I think it ended around eight. 00:11:35.980 --> 00:11:50.366 So if my eight hours of viewing, across those eight hours I had a 99.9999% great viewing experience, does that one kick out of the stream? 00:11:50.366 --> 00:11:52.513 Is that considered a bad viewing experience? 00:11:52.513 --> 00:11:54.764 Would I be labeled in a box? 00:11:54.764 --> 00:12:04.389 If someone was making a chart of bad experience, I would say no, yeah, yeah, but I don't know how each person individually and company measures that. 00:12:05.131 --> 00:12:08.840 Yeah, but I don't know how each person individually and company measures that. 00:12:08.840 --> 00:12:18.451 Yeah, so you know, the reference that I have here is broadcast and you know MVPDs, so basically the pay TV guys, they all talk about five nines. 00:12:18.451 --> 00:12:21.801 Five nines uptime, and you know. 00:12:21.801 --> 00:12:31.907 So that means that basically the system's fully operational 99.999%, that's about six minutes per year, you know, is what he equates to. 00:12:31.907 --> 00:12:41.147 So like in that world, if you're meeting five nines, it means that you could have five minutes of downtime and it meets five nines. 00:12:41.147 --> 00:12:42.869 Now, that's in a year. 00:12:42.869 --> 00:12:45.171 You know. 00:12:45.171 --> 00:13:01.517 So I think it's fair to say that OTT, just even if you equate, you know a restart, for example, you know, let's say you have a restart, only one in every 10 events, you know, let's just just for sake of just argument. 00:13:01.517 --> 00:13:08.326 Well, depending on the frequency of events are getting watched, you're beyond five nines, you know. 00:13:08.326 --> 00:13:17.879 Now, I don't know, in the user's view, you know people are pretty pissed off, even if that five nines from Comcast, and I'm just picking a name. 00:13:18.480 --> 00:13:18.701 Sure. 00:13:19.263 --> 00:13:23.171 You know happens right in the middle of the Superbowl, it doesn't matter that. 00:13:23.171 --> 00:13:28.125 You know right in the middle of the Super Bowl, it doesn't matter that, you know it's. 00:13:28.125 --> 00:13:29.347 In other words, it's what's the event, you know. 00:13:29.347 --> 00:13:31.292 So I think this also is where you're raising a good point. 00:13:31.292 --> 00:13:49.168 You know I saw comments people are quick to talk about, you know, kind of throw around the term user experience and yes, we all care greatly about that, and we should, and certainly Netflix does, and you know so, everyone does across the ecosystem. 00:13:49.168 --> 00:13:51.474 But what does it mean? 00:13:51.474 --> 00:13:52.196 You know? 00:13:52.437 --> 00:14:02.405 just because, yeah, and how do you measure it and how do you quantify that and what does it mean for you, for me versus for you versus for your neighbor, you know? 00:14:02.405 --> 00:14:19.994 So I think the hand waving of oh, this was a good user experience or this was a bad and I'm just now speaking generically not even you know, not even the, uh, football game, or football games on christmas day, that's kind of where our industry gets. 00:14:19.994 --> 00:14:21.659 A little goes sideways. 00:14:22.100 --> 00:14:24.989 Well, there's no methodology and no one's agreeing on anything. 00:14:25.048 --> 00:14:27.053 So for instance internally at Netflix. 00:14:27.094 --> 00:14:29.279 They obviously have some sort of number where they're saying-. 00:14:29.299 --> 00:14:36.428 They have some sort of metric Absolutely X percentage of our users they're tracking their team and they're tracking their systems. 00:14:36.428 --> 00:14:37.091 Yeah. 00:14:37.171 --> 00:14:41.783 Yes, we don't know what it is, but X percentage of their users had what they would consider to be a good experience. 00:14:41.783 --> 00:14:46.548 I don't know what that threshold is, but it's an interesting conversation the industry continues to have every year. 00:14:46.548 --> 00:14:48.789 I gave Netflix an A in my rating. 00:14:48.789 --> 00:14:56.136 If I'm thinking rating it A to F, which I think is better than saying numbers, what's the difference between an 82 and an 87? 00:14:56.136 --> 00:15:02.240 I don't really know One to five. 00:15:02.240 --> 00:15:09.006 I don't really know the difference between a three and a four, so I thought the A rating was just okay From everything I saw, from everything I saw online. 00:15:09.147 --> 00:15:10.871 Obviously I'm limited in the data I could see. 00:15:10.871 --> 00:15:19.567 Now also, just a quick shout out and thank you to people listening because, mark, I had dozens of people sending me screenshots, trace routes. 00:15:19.567 --> 00:15:35.802 You know, at one point, two ISPs were having a little bit of trouble in one particular city in terms of routing some traffic and I actually sent those IP addresses to folks who you know, who should know, and it was just. 00:15:35.802 --> 00:15:54.443 It was awesome just how many people were pitching in and sending information people from inside, mostly isps, but also people who were tied to networking, who watching from home and you know, could share charles logs and transfer out information and so actually I was, I was getting a little too much and actually I I didn't. 00:15:55.847 --> 00:16:01.224 I I didn't get to really study some of it, like I should have a little bit deeper because so much was coming in. 00:16:01.224 --> 00:16:03.830 So maybe a lesson for next time. 00:16:03.830 --> 00:16:05.813 I I try not focus on everything. 00:16:05.813 --> 00:16:15.653 I also didn't get to look at Roblox in terms of how well it did in the universe there because I had it on the iPad, but then I never ended up getting to the iPad, to that app. 00:16:15.653 --> 00:16:19.971 So from what I saw online, nobody was complaining there. 00:16:19.971 --> 00:16:27.014 And then, mark, just the final thing here on this piece is no one's talked about this and they never do after a live event. 00:16:28.221 --> 00:16:30.629 No dynamic ad insertion was used, no DAI. 00:16:30.629 --> 00:16:38.950 So for all the talk of ad personalization down to a local level based on the individual DAI wasn't used for the Super Bowl. 00:16:38.950 --> 00:16:42.668 It's not used for Amazon Thursday Night Football. 00:16:42.668 --> 00:16:52.369 It's not used for the NFL wildcard game on Peacock early in the year, wasn't used on Christmas, wasn't using the game on day after Thanksgiving. 00:16:53.759 --> 00:17:07.593 Dynamic ad insertion at scale is hard and anyone who tells you it isn't go ask any of these companies why they're not doing it, because what we've heard with ad insertion is you need a large audience and the moment you have a large audience you can make a lot of money because you can do targeted advertising. 00:17:07.593 --> 00:17:17.901 So now the largest audiences we've had in our industry for streaming still no DAI, and that's just how hard it is to do at scale. 00:17:17.901 --> 00:17:31.246 So to me, that's the next thing we're looking for in the industry is who can do this at scale for large scale live events and at some point right, someone's going to figure that out and make that work and invest the time and money. 00:17:31.246 --> 00:17:34.720 But we're still talking about targeted ads inside streaming. 00:17:34.720 --> 00:17:35.340 For what now? 00:17:35.340 --> 00:17:38.307 15 years, probably, at least. 00:17:39.269 --> 00:17:42.253 Yes, and still nothing for large scale live events. 00:17:42.920 --> 00:17:44.663 Yeah, so at some point it's coming. 00:17:44.663 --> 00:17:46.606 Let's do latency really quick. 00:17:47.288 --> 00:17:49.432 I broke out latency in a separate post. 00:17:49.633 --> 00:17:55.042 Yeah, you know people are going to want to have a lot of comments here. 00:17:55.042 --> 00:17:59.973 I was kind of happy, Mark, to see that not too many people commented on the latency thing, which I think was good. 00:17:59.973 --> 00:18:03.328 So the way I compared latencies, I didn't have OTA where I live because it wasn't the local teams. 00:18:03.328 --> 00:18:04.999 So the way I compared latencies, I didn't have OTA where I live because it wasn't the local teams. 00:18:04.999 --> 00:18:20.002 So I had people I know actually calling me on the phone and then I was timing when they would get video via OTA, in terms of like, okay, he just kicked the ball. 00:18:20.002 --> 00:18:21.954 In the moment they said kick, okay. 00:18:21.954 --> 00:18:24.541 I was then doing stopwatch of when I got the stream. 00:18:24.541 --> 00:18:30.942 So it wasn't an exact science, but it was close enough to give me an idea. 00:18:30.942 --> 00:18:35.039 So streaming latency varied, of course, based on HLS and Dash. 00:18:35.039 --> 00:18:41.702 On the MacBook it was 16 seconds, Fire TV Stick 14, Roku 10, Apple TV 10, iPad 3. 00:18:41.702 --> 00:18:43.355 That's what I got. 00:18:43.355 --> 00:18:48.803 I did not get to test latency within the Netflix app on the connected TVs. 00:18:48.803 --> 00:18:54.575 Again, I was just trying to do a bit too much in an eight-hour window, but that's what I got. 00:18:54.575 --> 00:19:03.223 On latency, I did see some people in other regions of the world Europe and APAC who were talking latency as high as 25 seconds. 00:19:03.223 --> 00:19:12.703 Here's 1, 2, 3, 4, 5, 6, 7, 8 people who responded in LATAM that they saw latency up to 20 seconds. 00:19:12.703 --> 00:19:20.083 That was about the highest I saw, but I didn't have specs on their hardware so that I I wasn't really sure. 00:19:21.611 --> 00:19:24.980 Uh, I don't think latency is as significant a problem as many industries suggest. 00:19:24.980 --> 00:19:26.674 We've obviously talked about this a lot. 00:19:26.674 --> 00:19:32.678 Um, if Netflix had the latency down, uh, further on certain platforms, would more people have watched? 00:19:32.678 --> 00:19:34.164 No, Would they have made more money? 00:19:34.164 --> 00:19:37.272 No, Um, would it contributed to the complexity of the setup? 00:19:37.272 --> 00:19:46.465 Yes, and there's always a trade-off with reducing latency when you get to a certain degree in terms of how low you want to get it. 00:19:46.465 --> 00:19:54.804 All the OTT, major OTT platforms have talked about that publicly the negative impact it can have on some users. 00:19:54.804 --> 00:19:58.074 So not surprising what Netflix did there. 00:19:58.074 --> 00:20:05.484 I don't know what their goal is over time to get it to under a particular number, Don't know. 00:20:05.484 --> 00:20:10.921 Now let's go to what the media is getting wrong in some of this, because it's important. 00:20:10.921 --> 00:20:19.440 So this quote was a multitude of Netflix subscribers voiced frustrations over severe stream interruptions. 00:20:19.440 --> 00:20:22.584 I don't know what a multitude means. 00:20:22.584 --> 00:20:23.724 That's not an actual number. 00:20:25.425 --> 00:20:30.627 And you were monitoring, you know X and other social platforms. 00:20:30.647 --> 00:20:32.229 I was Reddit Facebook. 00:20:32.410 --> 00:20:37.705 Yeah, so would you characterize that there was a multitude of disgruntled and frustrated? 00:20:38.106 --> 00:20:38.990 Netflix subscribers. 00:20:38.990 --> 00:20:41.125 Well, I'll be honest, I don't know what the definition of a multitude of disgruntled and frustrated netflix subscribers. 00:20:41.125 --> 00:20:41.164 I'll be honest, I don't. 00:20:41.164 --> 00:20:42.761 I don't know what the definition of a multitude is. 00:20:42.922 --> 00:20:47.612 I know that's here and it simply says a large number of people. 00:20:47.652 --> 00:20:53.388 Well, large is not a number either exactly and then it says a mass of people. 00:20:53.409 --> 00:20:56.732 all mass isn't a number either, so I would not have picked multitude. 00:20:56.732 --> 00:21:01.317 I think I used the word some right, because I don't know how many. 00:21:01.317 --> 00:21:10.106 I don't know what percentage of users are actually commenting that are not having problems versus ones that are, so there's no way to know. 00:21:10.106 --> 00:21:28.771 Were things like Netflix premiere NFL broadcast has more unforced errors than Steelers Chiefs combined, and I saw headlines of you know Netflix kicks off NFL on Christmas with you know major pain points? 00:21:28.771 --> 00:21:34.290 And then the things that people pointed to actually were not tied to the game or the stream. 00:21:34.290 --> 00:21:34.913 Technically. 00:21:34.913 --> 00:21:37.660 It was when the pregame started. 00:21:38.801 --> 00:22:02.493 you could not hear the studio host k adams voice in the intro for the first 10 or 15 seconds in the pregame yeah now the other thing is they were saying that the graphics indicated steelers had two timeouts when they really had three, and I'm thinking like, if this is the best thing, you have to complain about the netflix stream, then they did very well. 00:22:02.493 --> 00:22:06.502 Yeah, because how many times are you watching fox or cbs? 00:22:06.502 --> 00:22:08.554 And they accidentally put up a graphic. 00:22:08.594 --> 00:22:19.799 That's just not quite accurate well, or or, or, you know the wrong mic is activated and, yes, you know, for a few seconds you're hearing, you know, some random talk on the field, you know. 00:22:19.961 --> 00:22:34.065 I mean that's not uncommon at all so, but it was interesting they took one or two of those things to actually make a highlight, you know, in the post and actually have the headline talk about how it was a failure. 00:22:34.065 --> 00:22:35.074 So I was just like all right. 00:22:35.369 --> 00:22:37.737 Well, everything has to be sensationalized. 00:22:38.157 --> 00:22:39.101 Yes, it does. 00:22:39.101 --> 00:22:43.640 Now here's another one Quote the buffering issues left millions frustrated. 00:22:43.640 --> 00:22:45.355 Well, that's a guess. 00:22:45.355 --> 00:22:50.439 You have no idea that millions were frustrated, so you should not be writing that if you're a member of the media. 00:22:50.439 --> 00:22:56.299 Another one said there were more than 35 million expected concurrent views worldwide. 00:22:56.299 --> 00:23:07.563 According to whom who's that 35 million expected by Netflix didn't put out any numbers beforehand or talk publicly about any numbers, so I don't know where that came from. 00:23:07.563 --> 00:23:11.885 There was some talk about Open Connect Fortunately not too much. 00:23:11.885 --> 00:23:15.096 I did see a couple of people saying it wasn't built for live, it's not going to work. 00:23:15.096 --> 00:23:27.270 Well, there's this interesting sort of just couple threads going on on linkedin to mark where people are talking about open connect as if, because it was vod, can't also be used for live. 00:23:27.270 --> 00:23:31.965 Yeah, this is all about how you segment out a server and what it's doing exactly. 00:23:32.729 --> 00:23:40.803 so someone actually said well, isp should be charging netflix for the cost of having to install all these new live appliances into their network. 00:23:40.803 --> 00:23:47.801 What Netflix wasn't installing live only appliances in their network? 00:23:47.990 --> 00:23:48.632 It's not how Open. 00:23:48.653 --> 00:23:49.275 Connect works. 00:23:49.896 --> 00:23:51.490 Exactly so. 00:23:51.490 --> 00:24:07.280 I think it was an Axios and, in all fairness, I don't know if this came out of their piece, but I was just reading something on LinkedIn and it was referencing Axios and it said that Netflix clearly upgraded their servers. 00:24:07.280 --> 00:24:17.684 Well, I can assure you that, logistically, there was no possible way that they upgraded between the Tyson fight and the game. 00:24:18.029 --> 00:24:19.978 Well, well, depends how you define upgraded. 00:24:20.770 --> 00:24:30.276 Well, okay, okay, that's yeah, that's true, dan software Okay, yep, absolutely Um, you know, but hardware no. 00:24:30.737 --> 00:24:32.279 Well, so part of the issue too. 00:24:32.279 --> 00:24:44.497 Here's the wall street journal pushed out a piece and they they talked about what happened during the Tyson fight and and then, when I reached out to them I said, by the way, you got a couple of phrases in here just inaccurate, right described they're like they were. 00:24:44.497 --> 00:24:44.938 Nicely. 00:24:44.938 --> 00:24:49.175 Replied said yeah, we didn't really, you know explain technically what the problem was, and I go. 00:24:49.696 --> 00:24:58.895 Yeah, you explained that netflix had a quote jam on its network, right, but here's the issue I had with that mark was a whole bunch of people were now using the wall street journal thing. 00:24:58.895 --> 00:25:05.538 Tech meme made it the number one article on streaming on netflix and post boxing because it's from the Wall Street Journal. 00:25:05.817 --> 00:25:07.663 Yeah, sure, and it's like you're describing as jam. 00:25:07.663 --> 00:25:08.875 My printer has a jam. 00:25:08.875 --> 00:25:11.998 I mean, that's not the same thing as Netflix. 00:25:11.998 --> 00:25:17.602 So unfortunately that got picked up on a lot by mainstream media. 00:25:17.602 --> 00:25:28.694 But even some people in the industry were like well, according to Wall Street Journal, they say that Netflix is working to improve their connections with ISPs in the US like Comcast, and I'm like well, what do you mean? 00:25:28.694 --> 00:25:31.074 According to Wall Street Journal, they're always doing that. 00:25:31.074 --> 00:25:32.659 That's not new. 00:25:32.659 --> 00:25:38.895 Now they have all the telemetry data to see what's actually needed after the boxing event to figure out how to have better QOE. 00:25:38.895 --> 00:25:40.715 So that's not news. 00:25:40.715 --> 00:25:45.635 But you know the Wall Street Journal because it's Wall Street Journal and gets all the publicity. 00:25:45.635 --> 00:25:50.404 So for anyone wondering, yes, there was a jam after the boxing event. 00:25:50.404 --> 00:25:53.395 That's the technical term you should use with your parents. 00:25:53.395 --> 00:25:54.640 Yeah, yeah. 00:25:54.759 --> 00:25:55.602 Or other people. 00:25:55.602 --> 00:25:59.059 They can relate to that, they can. 00:25:59.150 --> 00:26:01.739 That's why I'm saying use jam Traffic jam Yep. 00:26:02.451 --> 00:26:03.455 Now let's go into metrics. 00:26:03.455 --> 00:26:10.980 So here's what we have as of Saturday, december 28th, because again some people are taking these numbers out of context. 00:26:10.980 --> 00:26:20.443 So Netflix said US US only viewership of the games peaked with the Beyonce Bowl at 27 million viewers. 00:26:20.443 --> 00:26:20.984 Peaked. 00:26:20.984 --> 00:26:25.721 The Ravens Texas game had 24.3 million AMA. 00:26:25.721 --> 00:26:29.403 Chief Steelers was 24.3 million AMA. 00:26:29.403 --> 00:26:30.670 Chief Steelers was 24.1 million AMA. 00:26:30.670 --> 00:26:35.381 So global ratings and additional US insights are going to be released on December 31st. 00:26:36.931 --> 00:26:47.526 Netflix also said that Christmas was a record-breaking day for NFL and Netflix, with an unduplicated audience of nearly 65 million US viewers. 00:26:47.526 --> 00:26:49.297 Now, this is, of course, according to Nielsen. 00:26:49.297 --> 00:27:12.696 For Nielsen, what they put out so far, it's AMA average minute audience which are based on this is important fast national live plus same day data, which also includes out-of-home viewing, cbs local market viewing, along with mobile and web data from Netflix, nfl Plus mobile viewing as well. 00:27:12.696 --> 00:27:17.619 Now we're going to go in comparison in a minute. 00:27:17.619 --> 00:27:19.556 But a couple things. 00:27:19.556 --> 00:27:27.854 Just high level here the reach of 65 million for what everyone's calling or Netflix is calling historic record shattering. 00:27:27.854 --> 00:27:34.984 It's down 20% from last year's 81.4 million for Christmas Day games on CBS and Fox. 00:27:34.984 --> 00:27:38.070 It's actually not a larger audience Now. 00:27:38.070 --> 00:27:44.377 It's the largest streaming audience for an NFL game in the US, but it's down from broadcast. 00:27:44.377 --> 00:27:47.998 Also, last year there were three NFL games on Christmas. 00:27:47.998 --> 00:27:48.840 This year there were two. 00:27:50.770 --> 00:27:54.291 Netflix media put out a quick tweet Also. 00:27:54.291 --> 00:28:02.840 One of the things they said was nearly one-third of Netflix global concurrence are watching Chief versus Steelers. 00:28:02.840 --> 00:28:09.974 Media picked up on this and said one third of Netflix subscribers were watching the game. 00:28:09.974 --> 00:28:11.617 That's not the same thing people. 00:28:11.617 --> 00:28:15.050 I don't know why the media can't get this right. 00:28:15.050 --> 00:28:18.778 Another thing Nielsen reported 65 million US viewers. 00:28:18.778 --> 00:28:21.864 I saw headlines running with households. 00:28:21.864 --> 00:28:24.233 Households and viewers is not the same metric. 00:28:24.233 --> 00:28:36.569 I't get this approach mark of why we have this problem after every live event where people change the actual terminology that the companies put out. 00:28:36.569 --> 00:28:37.272 Like I don't get. 00:28:37.272 --> 00:28:43.334 Why would you just, of all the things you change, you just change the word viewers to subscribers or viewers to households. 00:28:43.334 --> 00:28:48.683 I don know why, but it completely changes the accuracy of the message. 00:28:48.683 --> 00:28:59.875 Now, the total amount of viewership time is unknown, but a viewer was considered a viewer and the way they were counting it was they had to have watched at least one minute of the game. 00:29:01.539 --> 00:29:05.154 Okay, that's interesting. 00:29:06.238 --> 00:29:22.590 One minute Now. 00:29:22.590 --> 00:29:24.556 Last one here is NFL media said that Netflix quote eclipsed peak current concurrent. 00:29:24.556 --> 00:29:27.382 Let me start that over again Quote eclipsed peak concurrent viewers of any Christmas in the past four years. 00:29:27.382 --> 00:29:28.285 Now think about that quote for a minute. 00:29:28.285 --> 00:29:28.986 It's actually very helpful for us. 00:29:28.986 --> 00:29:38.131 So what we just learned is that Christmas day during the peak of COVID in 2020 had higher traffic on Christmas on. 00:29:38.151 --> 00:29:45.737 Netflix in the past four years and think about how many people are at home during Christmas during the peak of COVID. 00:29:45.737 --> 00:29:47.898 So interesting. 00:29:47.898 --> 00:29:53.423 We can see from that one stat higher peak because they use the term four year. 00:29:53.423 --> 00:29:55.825 So I thought that was pretty interesting. 00:29:55.825 --> 00:29:57.686 Now let's do some comparison here. 00:29:59.128 --> 00:30:09.484 We still need some complete data from Nielsen to truly do potentially what might be apples to oranges depending how they look at out-of-home viewing and whatnot. 00:30:09.484 --> 00:30:18.501 But what we know is that last year CBS Sports and Nickelodeon averaged 29.1 million viewers for Raiders-Kinsey Chiefs game. 00:30:18.501 --> 00:30:22.659 The game peaked at 37.1 million viewers. 00:30:22.659 --> 00:30:28.538 Ama, the Peacock-exclusive AFC wildcard game early in the year. 00:30:28.538 --> 00:30:34.990 That was the most streamed event ever in the US and it had an average minute audience of 23 million. 00:30:34.990 --> 00:31:01.141 But keep in mind that's 23 million across Peacock, nbc stations in Miami and Kansas City and NFL Plus, so $23 million there compared to the peak at $27 million, and then the average of just over $24 million for each game, so a little bit larger than Peacock. 00:31:01.390 --> 00:31:14.182 But we have to see exactly how Nielsen measures this once we have the full data, because will it include additional information from Netflix? 00:31:14.182 --> 00:31:14.724 It will. 00:31:14.724 --> 00:31:20.101 It'll include their additional information just, or data just like they're using with Amazon for Thursday Night Football. 00:31:20.101 --> 00:31:30.559 But overall viewership, total viewership, if we measure it based on the way we did here. 00:31:30.559 --> 00:31:36.487 In terms of US viewers, it was down from broadcast. 00:31:36.487 --> 00:31:38.777 Now, does that surprise anybody? 00:31:38.777 --> 00:31:39.660 I don't think it should. 00:31:39.660 --> 00:31:44.221 Broadcast just has more coverage in the US. 00:31:44.221 --> 00:31:45.635 It's obviously easier to use. 00:31:45.635 --> 00:31:50.121 You also have to take into account the teams that we're playing. 00:31:50.121 --> 00:31:52.135 That's important as well. 00:31:52.430 --> 00:31:55.400 Also, last year you had three games, this year you had two. 00:31:55.400 --> 00:31:58.357 So were people more interested in watching more games last year than this. 00:31:58.357 --> 00:31:58.959 I don't know. 00:31:58.959 --> 00:32:05.502 Yeah, but you can't do a complete apples-to-apples perfect comparison. 00:32:05.502 --> 00:32:09.269 You have to throw in some other caveats. 00:32:10.353 --> 00:32:12.298 I like your point even about the teams. 00:32:12.298 --> 00:32:18.056 You know the teams that are playing and Netflix has no control over whether you know. 00:32:18.056 --> 00:32:23.585 One team just simply has a much larger, stronger fan base, you know, than another. 00:32:23.585 --> 00:32:29.856 Or if there's more energy behind you know, behind a particular Major market teams. 00:32:29.936 --> 00:32:30.441 Big market teams. 00:32:30.569 --> 00:32:31.555 I mean it's just yeah. 00:32:32.750 --> 00:32:37.375 If that was Giants versus Cowboys, if that was New York teams, that's a whole different story. 00:32:37.569 --> 00:32:39.551 A whole different story, maybe not. 00:32:39.731 --> 00:32:40.615 Giants aren't very good. 00:32:40.955 --> 00:32:42.681 Yeah, yeah, same with the Jets. 00:32:43.131 --> 00:32:49.854 So maybe it would actually make the numbers lower, but larger markets tend to draw a larger audience. 00:32:49.854 --> 00:32:51.637 So that's what we have as of now. 00:32:51.637 --> 00:32:54.723 Let's see what data comes out on the 31st. 00:32:54.723 --> 00:32:58.359 Let's see if we can truly compare it apples to apples. 00:32:58.359 --> 00:33:15.518 And then finally, just for next year, because again people got to get this right Netflix, legally, contractually, I should say has the rights to do two NFL games in 2025 on Christmas and two NFL games in 2026 on Christmas, but they only have to do one. 00:33:15.518 --> 00:33:18.838 They have right of first refusal on the second game. 00:33:18.838 --> 00:33:26.762 So when are we going to hear from Netflix that, hey, we like this so much that we're definitely going to do two games next year? 00:33:26.762 --> 00:33:27.183 I don't know. 00:33:27.183 --> 00:33:40.156 There's also talk online of based on how the scheduling works, and it's far too complicated for me to understand without having to look it up but there's a possibility that a third game will be added on Christmas next year. 00:33:40.156 --> 00:33:47.538 Oh, if I'm reading the scheduling right, yeah, because the scheduling changes based on the day that christmas actually falls. 00:33:47.538 --> 00:33:50.523 Yeah, god, terms of how many games they can have. 00:33:50.523 --> 00:33:53.510 So we'll see what happens for next year. 00:33:53.510 --> 00:33:57.519 Um, and you know the importance here. 00:33:57.558 --> 00:34:03.974 I think the key takeaway mark that we should all think about here is because of what happened with the boxing event. 00:34:03.974 --> 00:34:10.965 Netflix had the entire world watching, from both a viewership standpoint and every single media outlet. 00:34:10.965 --> 00:34:15.197 So this was a big day for us all. 00:34:15.197 --> 00:34:27.431 For anyone in the streaming industry, this was a big day because if they had technical issues impacting a large enough percentage of the audience, what's the media going to write about? 00:34:27.431 --> 00:34:29.135 Yeah, it's all going to be negative. 00:34:29.135 --> 00:34:30.398 Yeah, that's right. 00:34:30.398 --> 00:34:33.152 The the other thing that you know it's. 00:34:33.152 --> 00:34:34.474 It's not often mentioned. 00:34:34.474 --> 00:34:36.219 It's a super critical point. 00:34:36.219 --> 00:34:40.152 Netflix boxing event was initially scheduled for july 20th. 00:34:40.152 --> 00:34:57.423 Now, if that event hadn't been postponed to november 15th due to tyson's injury, netflix would have had five months between the boxing event, the NFL games on Christmas to tune, open Connect and just the entire pipeline video pipeline. 00:34:57.423 --> 00:35:03.983 But due to the scheduling change, netflix's five-month window to prepare for Christmas dropped to 45 days. 00:35:03.983 --> 00:35:07.711 45 days, that's a big deal. 00:35:07.711 --> 00:35:08.726 To prepare for Christmas dropped to 45 days, yeah, 45 days. 00:35:08.726 --> 00:35:09.324 Yeah, that was my point. 00:35:09.344 --> 00:35:13.226 Yeah, that was my point about the upgrading their servers. 00:35:13.226 --> 00:35:25.643 There is no way there was hardware going into, literally, you know, tens of thousands of servers in 45 days. 00:35:25.971 --> 00:35:27.998 More than 18,000 is what they currently have. 00:35:27.998 --> 00:35:29.516 That's the last number they gave us. 00:35:29.516 --> 00:35:37.596 Yeah yeah, yeah, the other thing here too, anyone who's done live events before to tune stream performance, you need enough real-time telemetry data. 00:35:37.976 --> 00:35:54.235 Yes, so the building data Now they didn't have that going into Tice and Boxing because all the data they had was obviously, up to that point, really focused on VOD for large scale because the other live events previous to that were too small and I don't know how many logs they were processing. 00:35:54.235 --> 00:35:55.858 It's a lot. 00:35:55.858 --> 00:36:08.612 But now that they actually had telemetry data QOE data from the boxing event, that really is what they needed and again, normally they would have had five months to prepare. 00:36:08.612 --> 00:36:22.543 So to go from having that data on QOE to 45 days later pulling off an event with 27 million AMA in the US, anyone who doesn't give them credit doesn't understand what they did. 00:36:23.231 --> 00:36:27.000 Yeah, yeah, exactly, I totally agree with your. 00:36:27.000 --> 00:36:28.503 A uh grade. 00:36:28.503 --> 00:37:06.731 You know, dan, I I mean, first of all, you know, even if there wasn't, uh, the rebound that was needed coming off the tyson, you know the tyson situation, you know, if this was just their first like it, it was a very good, very high quality streaming event Seemed to be yeah and yes, as we've already pointed out and covered, of course there's going to be issues and of course there were users who did not have a good experience, and that's not to say that, oh, too bad for them, but it's just to say that, look, this is hard, it was done at scale. 00:37:06.731 --> 00:37:21.565 And, as we said, coming off the Tyson, I remember the interview, the episode that we did right after Tyson. 00:37:21.565 --> 00:37:32.320 Very smart people inside Netflix who are working overtime right now to ensure that Christmas is not going to be a repeat of the Tyson experience. 00:37:32.320 --> 00:37:33.541 And guess what? 00:37:33.541 --> 00:37:35.164 It wasn't a repeat. 00:37:36.172 --> 00:37:37.335 I think they did a great job. 00:37:37.355 --> 00:37:49.117 The other thing I'm always looking at Mark is how many inquiries I take from people I know Because, even though a lot of people don't know what I do, it's just like, yeah, dan's something streaming related, I'll just text him when something doesn't work. 00:37:49.117 --> 00:37:58.717 Yeah, so during the boxing event, even though I was away, man, the amount of text I was getting I think I looked the next day and it was over 30 different individual people. 00:37:58.717 --> 00:38:09.795 Now a lot of these guys too who were texting me are deployed in countries all over the world, so not only am I getting feedback from actual, real users, but I'm getting it from people in different countries. 00:38:09.795 --> 00:38:13.693 That's an interesting perspective for this netflix game. 00:38:13.693 --> 00:38:21.577 On christmas, I didn't get a single text from anyone I knew talking about like, hey, this isn't working, or what do I do, or I'm getting this error. 00:38:21.577 --> 00:38:22.197 Not a single one. 00:38:22.197 --> 00:38:32.222 So to me, that's just for me as a sample size having people all over the world giving you feedback and then going to where you're getting none in terms of errors that says something. 00:38:32.222 --> 00:38:34.715 Yeah, so this was important. 00:38:34.715 --> 00:38:37.021 Shout out to the netflix team for pulling this off. 00:38:37.101 --> 00:38:38.032 It's big for everybody. 00:38:38.032 --> 00:38:46.141 You know, I remember what it's like to help produce these large-scale live events a long time ago, late 90s, early thousands. 00:38:46.141 --> 00:38:55.750 Many people don't realize the number of hours involved, the dedication and commitment to the task, let alone on a holiday. 00:38:55.750 --> 00:39:00.105 These folks had to give up their time on a holiday and many days before that. 00:39:00.105 --> 00:39:03.175 The problems that must be solved under pressure, and it was a global event. 00:39:03.175 --> 00:39:06.204 You know, in the nineties and two thousands a lot of the stuff we were doing was okay At that time it was considered big and it was a global event. 00:39:06.204 --> 00:39:09.233 You know, in the 90s and 2000s a lot of stuff we were doing was okay. 00:39:09.253 --> 00:39:18.902 At that time it was considered big but it was really small scale yeah sure so their success as a company equals success for everyone in the streaming industry exactly I. 00:39:19.101 --> 00:39:23.318