Netflix Adds Vertical Video, Runway AI Does Animation, and Gamepad ComfyUI

Netflix unveils two major platform changes that could reshape how you discover content. Hosts Addy Ghani and Joey Daoud dissect Netflix's new ChatGPT-powered search and vertical short clips feed. Plus, they examine Runway's impressive hand-drawn animated pilot and a practical innovation that lets creators control ComfyUI with a gamepad instead of text prompts.

Transcript

0:00

Like a vertical short video clip is what TikTok is, is what Instagram Stories are. YouTube is now that. Yeah, I mean, it is the discovery engine for finding new stuff. So yeah. I'm curious how this plays out. In this episode of Denoised, Netflix tests out a short feed and AI search, Runway gets into animation with a cool short film, and controlling ComfyUI with a game pad. Let's get into it. Welcome back, Addy. Good to see you. Hey, it's heating up here in LA. It's, yeah, the clouds have broken and it is, uh, now getting toasty. Yeah. And we've got AI a coming up. It's literally heating up figuratively and li Literally. Yes. So what do you got? All right, so first thing on my radar was a couple updates from Netflix. And so one of these we talked about previously where they were debating or experimenting with rolling out ChatGPT in their search. That's right. We covered that. Yes. And so now it is officially in the beta. You have to opt into it, but it is in their beta. And so it is a new search feature. I believe it's just on the app currently. Not on like Apple TV or anything, but it is basically like a more conversational way to try to search for something, to find something. So one of their examples is, you know, I want something funny and upbeat, and then getting movie recommendations based on just right conversational movies. Although Netflix has been really good at associations with search in the past, so like let's say you are searching for a show like Star Wars, you type in Star Wars, it'll show up. Like Star Trek will show, show up, especially when it's like, we know we don't have this in our library, but we know what you're searching for, but show and knows something similar, similar things. That is an original production, right? Yeah. So they have been really good about that, and I know. That there is a large ML team in Los Contos where headquarters is. Mm-hmm. So this makes no surprise. Like this is perfect. Yeah.'cause like the movies already have, they already have this data around the movies. Yeah.'cause also they have that famous whatever, 30,000 different like genres, niche genres. If you remember, there used to be, you could, there was like a hack where you could figure out the code. Yeah. To like find all the different genres that was like, you know, action movies from the eighties, like that take place in, uh, China or something. Yeah. And, uh, there was also a hack to use a VPN to get around their geographic, uh, sort of categorizations as well. Yeah. But did they, uh, crack down on that, on that VPN hack or not Sure. Having gen AI search, it's a good way to help find stuff. And this has also been a big issue for Netflix throughout the years of. Just improving. Finding things that you wanna watch. Yeah. At the right time that you wanna watch them. I remember too, wasn't it a number of years ago, didn't they have like a million dollar reward for someone who could develop like a better algorithm really to recommend? I mean, it was like a while ago, like when they were first getting off the ground. That makes sense, right? Because so much of their business and the sort of watch time comes from a good recommendation, right? Yeah, of course. And finding the thing you wanna find. And we mentioned that stat before that Ted Sarandos had mentioned on the earnings call where he was like, top 1%. Of their content or the top 1% of like their most famous titles is still like 1% of the viewing. Yeah. It's nothing. Yeah. Yeah. And there's just like such a massive library and an international library of content, right? That there's just so many different things that people are interested in watching that. You know, there's no one size fits all for Yeah. A, a massive library watching habits that they have is a double-edged sword. On one hand you have every single conceivable genre and show kind of covered. Mm-hmm. On another hand, good luck finding it. Yeah. How do you surface that? How do you find it at the right time? Yeah. I mean, I feel like the next thing from, you know, a gen AI search, 'cause you still have to, like, it's a bit active. You have to get on it. You gotta like, kind of know what you're in the mood for. Yeah. You know, I'm thinking the future is maybe like a gen AI channel. Not a gen AI channel, a generative playlist channel. Where? Oh, yeah. Like in, like Disney's been adding this, Spotify has that. They have this for, they, they have a DJI that is just uh oh. Right, right. You're auto automatically recommend. Hey Joey, welcome to, I'm your Spotify. I'm X your Spotify DJI. Should I do? Hey Joey. Welcome back man. Today we've got hip hop from the nineties. I'm mixing things up for you. You know what I'm talking about? Yeah, yeah, yeah. Maybe not as annoying, but certainly Right. So right. That with songs, something that, but like something that sort of feels like it's already on. So like Disney+ has been doing this where they've had, I mean we're basically like reverting back to cable TV with channels. But just like channels that are just old, you know, movies playing where Yeah. I mean, this was a better experience. Well, not a better experience, but it was a different experience in the nineties and early two thousands when you're on cable and you're like, I don't really know what I wanna watch. And just channel surfing. And then your mid play of a movie. Yeah. Then you just leave it on'cause it's already playing. Yep. I'm thinking, you know, the next phase of like Netflix is a generative channels based on your interest, but it's like already playing for sure. You know, it's customized to you and your interest, but they do do that with the kids accounts a little bit. The kids accounts are obviously, you know, because they're kids, you wanna be careful about what they watch and also fine tune. No pun intended, what they watch. Mm-hmm. They have a more like specific, uh, algorithmic output. And then the cool thing about the kids channels is. The parents will get a report every other week on what their kids watch and what the big categories are. So one of my girls, she loves to watch stuff that's about bravery, uhhuh, and about courage. And you're like, oh, that's a, that's an interesting underlying theme on all the stuff she watches. That's cool. Yeah, I did know. That's cool to know. Yeah, yeah, yeah. Do they tell you if they tried to search for anything, like what they searched for? So the kids, when you go into a kid's account, it's very much restricted. Kind of like you two kids. Like you can't really get terrible content out at that point. Yeah. But if you tried to search for it, like, like your kids try to search for Chucky. Red flag, red flag, they're looking for psycho. They heard about that. That's cool. That's interesting. Yeah. With the channel thing, I'm thinking back to the like nineties stuff. I'm thinking like. I've probably seen the actual beginning of Terminator 2, maybe like once or twice. Oh. But I've seen the, like I've seen so much of the second half of that movie. Yeah. Like a million times. But I've like maybe seen the beginning a few times where he walks into a bar, grabs the clothes from the guy and walks in naked? Uh, yeah, yeah, yeah. I've maybe seen that a few times. Okay. But it's like, you know, the river scene and the, the bus chase and like Okay. You know, falling, sinking into the pit lava, like, I've seen that a million times. Yeah. But because, you know, I'd always catch it on like TNT or something and so like I've seen the second half of that movie a million times, but Right. Maybe the beginning, once or twice. Yeah. That's, that's an awesome example. Yeah. That's the nature of when you were just catching stuff on cable. And so the other interesting thing with the Netflix update mm-hmm. Is they are pushing into, uh, a vertical feed. Yep. Short clips. And so very useful, very, very relevant to today's media landscape. Yeah. So this is not UGC type stuff. This is short clips from their shows. Yep. More of as a discovery engine. But I think this is really smart because Yeah, there are a number of times that I've been on TikTok or wherever and I see it's not the studios that are putting them out or maybe they're, you know, secretly putting them out, but it's like a scene from a movie is playing and you're like, can you kinda get sucked into it? Ten second bite. Yeah. And it's like, oh, one of the good dramatic scenes. And it's like, oh, interesting. And then it cuts to like a, a montage of like shots from the movie. And they don't even tell you what the movie is. You like go in the comments and people are like, what movie is it? What is this? And then so someone comments like, oh, it's this movie on, on this streaming platform. So it's almost like a auto automatically generated trailer that's vertical's like a trailer, but not, it's more like a clip or sort of like if you're on Netflix too and you scroll past the titles Yeah. And then you like let the, let it hover and then it starts playing. Yeah. Like a clip from the film. Right. Like that, but in a vertical feed. Yeah. I gotta say like at this scale with the millions and millions of titles that they have. Incredible feat in deploying this. Mm-hmm. Mm-hmm. Having said that, I hate to bring up Spotify again, but they have done this already and it's I think a year or two into it. Yeah. Where if you go to like Kendrick Lamar's page, right, you'll see a bunch of shorts and then you could swipe through each of the songs. And each of the songs is, is in a video format. It's a video, maybe it's the music video itself or some kind of visuals. And then when you like the song, you click on it, it goes and plays the song, right? Yeah. And yeah, I can see how that works. I feel like it makes more sense on Netflix 'cause you're like. It's visual. Yeah. And you're like looking at visual stuff and so you see a video clip and you're like, well let me add it to my list or watch it now. Yeah. Spotify, it's like, I'm gonna Spotify'cause I wanna listen to something. Not like scroll, but that's the thing like, because you know we're on our phone so much. Yeah. And our screen time used to just so high, like having that ability to scroll through visuals for a music product. Mm-hmm. Isn't that a podcast product? Yeah. We put up shorts now. We have shorts up now. Yeah, we do now. And, but I mean, that was another reason why. Spotify did such a big push into the video podcast and hosting video podcast. Yeah. For this exact reason. Yeah. But, uh, I will say like, you know, like a song is what, three, four minutes versus an episode is like 30, 40, 50 minutes. And like there is such, so much more data to churn through to create these shorts on the Netflix side. Yeah, yeah. And a lot more discovery and, and yeah, it, it makes sense. And uh, but you know, I've seen the show with the vertical clips come to Netflix reminds me of the. You know, ongoing conversation of like, you know, who's real, real shorts, not even that, but sort of that, I guess. But like the ver the, the vertical soap operas. Yeah. But I was thinking more of just like, Hey, you know, is Netflix gonna become YouTube first? Or YouTube gonna become Netflix first? And this like continuing ongoing Yeah. Battle between the two where like they're the two dominant players for like most, for screen time and owning screen time. Especially on a tv. I will say, I think that the line. Has already been drawn and Netflix will never fully become YouTube and vice versa. Like they know their market, they know where their revenue is, they're gonna be a full UGC, right? So there's always gonna be a yeah. Uh, you know, an acquisition or a content wall. But it's interesting that Netflix is sort of, you know, tipping the hat to the format that is working for all of the other social media formats like a vertical short. Video clip is what TikTok is, is what Instagram stories are. Mm-hmm. YouTube is now that, yeah. I mean, it is the discovery engine for finding new stuff. So yeah. I'm curious how this plays out. I haven't updated my app. I don't know if I have it yet, but, uh, I'm, I'm curious. I'm curious too, if they do special edits for these clips or if it's just gonna be like, I. We took a clip. Yeah. And we just framed it for vertical also, their framing process. Are they running this manually? Are they, maybe they bought the Quibi thing. Yeah, the Quibi Framing algorithm. The Quibi tech ended up, Hey, you never know. Yeah. I mean, I'm curious, like, are they running, uh, using AI models to like track the faces? To like auto reframe? Yeah. Or are they doing this by hand? I mean, it would probably be cheaper to just have a farm of, you know, artists in India somewhere. And just pay them to get this done, at least for the initial rollout until they figure out what the automated process is. Adam, I think the automated stuff already exists. That's been something that's like, been doable for a while. You know, just track the face and just make sure the face just, you know, stay center framed. Yeah. And this is, uh, not new news, but Netflix's app, at least on iOS, on my phone, has games and stuff built into it already. Yeah, yeah. The games are, I've used the games. Yeah. They're fun. Yes, exactly. Like they had included, I think I played the, uh, Squid Game game. I haven't played that one yet. Yeah. So I like the infinite run ones. They had a bowling one. Yeah. That was just like an infinite, like, you know, oh, bowling ball. Like you have to jump the, you know, it was like a speed run thing. It's like temple run. Yeah. It was like temple run with like bowling balls. Awesome. Yeah. Definitely has been doing that push into gaming. Yeah, that's always been a bit like. Where's this going? And now they have live events and now like, what are they not doing? You know? A bit of sports, but yeah, like yeah, sports is probably the next sports and news. Yeah. I, so, uh, on the flip side, on the competitor side, uh, I think Amazon is like two steps ahead on the sports front. Hmm. Like now they have, I think NBA, they've already had an NFL. And now they have NHL and NBA, and so they're really pushing into that end. Yeah. They've also built themselves into more of like a, like a platform Yeah. As well, where it's like you could have be on Amazon, Amazon Prime and Prime video, but then also subscribe to other streaming services. Mm-hmm. Through the Amazon hub, like subscribe to STARZ or Paramount+, yeah. Through Amazon. So it's kind of interesting. More of like a hub for like Yeah. Engaging with other streaming services. For sure. Uh, and then, uh, finally, I think we covered this as well. Uh, Netflix is to Doom, which is their u sort of their UGC platform where a lot of like show fandom exists. Yeah. Like maybe there's potential there to have a YouTube like creator output. Yeah. I mean, they do have a bunch of YouTube channels around without YouTube, without utilizing YouTube. Well, what do you mean? Like a, so like if somebody is a huge fan of Squid Game. Yeah. Right. And they just wanna make a review for each episode as it comes out. They have to now put it on YouTube. Right. What if they were able to put it on to do like more of like an in-house? Yeah. Kind of like to doom creator. Yeah. So it goes back to like when, do you remember when Chernobyl came out on HBO? Yeah. And they had the official podcast. Yes. And that was so successful. They do that pro pretty much all of the shows now. And that was the first show that was piloted with Yeah. Yeah. And it did so well. Yeah. And I'm thinking that was what, when you mentioned this, that's what that reminded me of.'cause yeah, like pretty much all the HBO shows now have a dedicated podcast. Right. And yeah, it's like from. Usually it's someone that was like, had a separate fan channel or something. Yeah. Paired with, I don't know, some other like podcast producer, and then they like, yeah, bring on guest. There is so much a guest show stuff. There's, there's just so much you're leaving on the table if you're not extracting out of that ip, like a podcast or other things, you know, because that's all engagement. Engagement numbers. Yeah. The same reason why there's like the behind the scenes stuff afterwards. It's like, yeah, you have your, you know, a lot of people just watch the show and that's it. But then you have like, you know, your, your bigger fans where it's like you watch the show and it's like, I wanna process that. I wanna like learn more about that or dive deeper that I wanna buy the toys. Yeah. Yeah. And this is just another way to kind of further that engagement. Yeah. And a podcast is pretty low lift. And if it was a, we could do it YouTube video. Yeah. Like that'd be pretty low lift. Yeah. And yeah, I don't know, I mean, I'd be curious if that made sense to build out some sort of extra streaming thing on to Doom. Or I feel like all roads lead to like YouTube. Threat on YouTube. YouTube because for the pure discovery, it'll fly discovery aspect. It'll just fly. Yeah. It's like, why wouldn't you?'cause you just, you want that top of funnel discovery Yeah. Aspect to get more people finding out about the thing and get 'em on platform and then watching the actual show. The only reason for putting it on to Doom, a very good reason is you could insert ads. Yeah, yeah. Into those, uh, UGC stuff and then just make money that you would otherwise not make from YouTube. Yeah, yeah, for sure. Yeah. Yeah. All right, next up. Interesting stuff with Runway. Yeah. Runway, uh, killing it lately, uh, after the Gen-4 release and mm-hmm. Uh, Cristobal's latest, uh, sort of speculative audio tool there where you just kind of. Tell the AI engine what to do. Uh, you know, bring me a building, bring me a sky, and it just kind of does it for you. So they're really thinking in terms of how m and e would use it. Mm-hmm. And the latest from them is, uh, animated series. It's called Mars and Sieve. And we'll play it back for you here. The first thing you'll notice is that this is very stylized. This is, uh, obviously like a Phil Noir. In the, in the way that the story unfolds and in the mood it has a very like, kind of the hand drawn Yeah. Animation vibe. So visually it has a hand drawn animation vibe, very much like a 2D look. Mm-hmm. And these are things that you generally don't associate with ai, right. Ai outputs typically Pixar style 3D it get prompt it Pixar style. Right. But like, that's what most people do. Yeah. Or, yeah. Yeah. You think it's a, you associated with an existing style already? Yeah. It's style that you. You know, for sure is gonna be output correctly. So in this instance here, I'm sure there was a big post process and an additional process on top of the generation, right? Yeah. So they have a, a nice interview in depth blog post. And so it seems like it's from Runway ai, their studio arm. Yeah. Uh, which has been tied in, but it's sort of a separate thing where it's like. To produce content and use Runway for like producing Oh, professional content. That seems like a fun department. Yeah. And Runway. So, yeah. With the creators it seems like, I mean, and they have images. The everything was hand drawn, storyboards, and then it seems like the bulk of the process was a hand drawn locations, frames of characters. Yeah. Hand drawing the elements and then combining the elements. In Runway. Yeah. Creating the first frames and sending it to Runway, and then using Runway for that in between animation. Perfect. Which is stuff we've talked about. Yeah. Before. Well, you know me, I'm an animation fan and when I saw this, like other than a few technical things, I thought this was really effective and it really worked where I think, uh, traditional animated. Thing would really defer from this is there would be a lot more set pieces, a lot more complexity in each of the shots. Mm. Mm-hmm. Where here it's just, you know, background character, background character. Yeah. Um, and I think that's, this is a good step. They were probably focusing more on lip sync, character animation for some of the Runway work, and then just compositing everything in post. That's my guess. Well, yeah, I mean, because Runway did have the whole, like the act one Yeah. With which was. You know, putting a character and taking a human performance Yeah. And being able to like match the emotions and stuff. It doesn't seem like they use that at all for this. Yeah. There is a tool called Adobe Character Animator. I think they reed it to Adobe Animate now. Yeah. That does really good Lip sync o off of like a iPhone camera. Yeah. You like right. You, uh, rig up. Yeah. Your animated character. Mm-hmm. And then it'll like sync with your iPhone as a Yeah. Like, uh, face capture. That could be like, this is exactly how we envision. It's all just tools in the like, it's like there's no one quick fix tool for this. It's just tools to speed up the process. Yeah. That's how movies get made. There is no, you know, magic, silver bullet, you know, it's just what do you need to do? Let's do it with this thing. Move on, do it with that thing and go on. Yeah, and this is a great example too, because I mean, they're like hand drying elements. They're compositing it. Yeah. And probably just traditional compositing software and then they're, you know, using Runway. To speed up the in-between process and like bring, you know, animate the actual shots. Yeah. The last time we saw a short film out of one of the big AI companies was, uh, do you remember when OpenAI released that balloon head thing with Sora? Yeah. Airhead. Airhead. Yeah. Yeah, yeah, yeah. So I, I think there was a big article on it after it came out that said something like, uh, one out of 300 or one out of. 3000 shots was used like it was a, a large number of generations. It was also heavily composited. Yeah. It would be like, you know, they'd get one element outta one shot. Yeah. Another element of another shot. And yeah. And that was sort of, especially last year in 2024, that seemed to be the common. Way to make something that looked realistic, like quality, like just toss the dice many times. Good enough. Yeah. And then get the elements you want. Yeah. And composited, and it was like after effects seemed to be the like glue for so many AI generated shorts to that were of a, you know, high enough quality that looked professional, uh, caliber. And most of these were created from media professionals who like work in VFX or animation. And so they know what they need to get and it just turned into. Using generative AI to just get the elements out of it that you need and com it later? Yeah. Really. Uh, I'm really surprised at the level of output and also the stylization and the creativity that went into it. It gives me a lot of courage that, you know, AI is not gonna ruin everything, if you will. No, I, you like use it, you know, correctly. It could just help you do more and Exactly. You know, for budgets where it just might be cross prohibitive Yeah. For something like this to exist. Yeah. I mean, we talked about before recording like this reminds me of the Cartoon Network. Yeah. Stuff like Dexter's Lab and stuff before in the nineties. Yeah, like classic adult swim cartoon network. Yeah. And I'd be curious. Like, I don't really know much about the making of process from those, like how big the teams were. Yeah. And like Sure. What the timeframe was to make those and how that would compare with something like this. Yeah. I think I remember reading into that a little bit. This is like a decade ago. Those were big teams. So Dexter's laboratory, powerpuff girls and like I remember, um, family guy, I mean basically similar process with how they're using Runway. They would do the key frames here in the States and like write the scripts, send it to Korea. Right. I think it was India. Okay. Because also if you looked at the names Yeah, it was like very Indian names. Yeah, Indian names and the credit. And they would do the inbetweens. Hmm. And so pretty, it's ai. Well, that's the thing. Now AI is so good at Inbetweens, right? Yeah. So that's, you give it first frame, last frame, and middle frame, which, yeah. Gen three can do and, and, uh, yeah. So yeah, I mean, this is a great example of, you know, combining human creativity, you know, and having originality. But using AI to speed things up and make I film that, you know, I don't potentially would not be possible otherwise. A hundred percent it, it still doesn't take away from the fact that you need a human storyteller with genuine mm-hmm. Creative idea. Mm-hmm. Creativity. Yeah. All right. Next up. So new news on ComfyUI. Yeah. What do we got? So. Look ComfyUI is this ever pervasive tool? You know, we've seen people at the AI hackathon use it. Mm heck I run it on my computer here on a very slow GPU. Yeah. When you want the most control Yeah. Out of your AI outputs and what you're doing with it. Yeah. All roads lead to ComfyUI. Exactly. And you're not just limited to your own, uh, you know, crappy GPU. You can connect ComfyUI to AWS or wherever and access a bunch of different, you know, high powered GPUs. Mm-hmm. Or run it off a cloud instance. Yeah. Or just run it off a virtual machine in the cloud. Yeah. But it is like. The most common language and the most common tool for AI builders today, Uhhuh, I could definitely confirm that. You know, even within the walls of AI manufacturers, they're using comfy y Yeah, yeah. To prototype things and sometimes to deploy custom solutions and so on. So ComfyUI. Is, uh, is a node based tool, but it's still not very user friendly. Mm-hmm. So, looks like you found something that's really interesting here. Yeah. So this ties into another thing we talked about of the lack of input or control in controlling ai. Yes. Outputs and using more traditional methods than like text prompts. So it seems like someone built a controller. Where you could use a game controller to control poses of characters generated in ComfyUI. That's interesting. Yeah. So in the quick video demo, there's, they're showing with, uh, they're running ComfyUI and they're using just a Gamepad controller Right. And hitting some buttons, and it's like they're creating a character. And it seems relatively in real time, you know, they're like hitting the button to the right and just changing the, the pose in the face to the left, to the right. Yeah. So it's a good indicator. I mean, I, my personal, what I would love to see is doing this with a camera in a 3D space. Right, right. But this is like in that right direction of just like a relatively real time way to control an AI output. Yeah. With something that's more intuitive than a keyboard, A game controller. Yeah. My 3D brain instantly goes to, well, why not? Like that woman's face, why couldn't it be a 3D character? And then the controller just controls the rotation of the head. Mm-hmm. And then you take that, uh, or if you can orbit around her. Orbit around, yeah. Right. Yeah. And then take that feed and do a neural rendering pass in comfy. Yeah. I would love to see that. Yeah. Nevertheless, you more about comfy than me too. You should download this. Lemme know. Uh, I'll definitely download it if it's available. Oh, that's another thing. Uh, for those of you who have not messed around with comfy, yet, comfy runs on what it calls workflows. Which are just a giant text file, uh, typically JSON files. So if you see a workflow you like, you can download it, load it, and that loads up all the nodes and all the connections to the nodes. All you have to do next is then load each specific nodes with the right models or the right. Yeah. Where I always hit issues. So Comfy now has a feature called, uh, install Missing Nodes. Yeah. That also hits hit or miss, but it, I'm, it is like, it's almost there for like the average deal. It's getting there. Yeah. I'm like, I see a mc Muppets tutorial and I'm like, I want that. And I download it and I install the JS O file. Yeah. And then it's like load missing models and then it's like, can't find this one. And uh, I've noticed that a lot of creators hide that under the Patreon page, so you have to pay and to get access to it. Yeah. Yeah. Yeah. Makes sense. Yeah. I paid them. I still, I subscribed to Mc Muppets. I wanted the missing, I wanted the advanced, uh, nodes. But yeah, uh, talking to, uh, JD and Skylar who run a Playbook and they've been doing a lot of stuff with Comfy and, uh, they, they have a cloud comfy instance where you can load their models and run on the clouds so you don't have to have a PC or a beefy, uh, machine. And we've talked about this and it's like, they're like, yeah, that's like when people bring in workflows, that's like the number one thing that breaks is like. Just finding the models that the person who built the original workflow has and then reconnecting them properly is like, yeah, just the kind of basic, but like the easiest part where just stuff falls apart, where you're like, yeah. And also like the models are so specific, like you can't just drop in the version of model. Yeah, yeah, yeah.'cause some of they're gonna output float 0.8, some of 'em are gonna output flow point 16, and if you mix up the two, then everything down chain is messed up. Oh, it breaks it all. Yeah. Yeah. Yeah. So the, I don't know the name of developers behind this, but the website company developed. It is vrch.ai, vrch.ai. Mm-hmm. So I just wanna give them a shout out. But yeah, this was on a Reddit post and so we'll link to it in the show notes. But yeah, I mean, uh, this reminds me too, of the hackathon I covered with, uh, Rob Legato. Yeah. That who is now at Stability as an creative. Chief pipeline architect. Right. And this was his number one request too, where it's just like, you know, gimme the controls that I'm used to Yeah. To operate in this AI space. Right. And yeah, I mean, so what, what do you think in order to like get to the point where we're talking about like, let's generate a world and like move through it in real time or relative real time. Like what, what else has to happen? It's happened already. It just hasn't happened at a, a consumer ComfyUI level. So if you look at startups like in world ai. That's exactly what they're doing. They're building a game engine that is rendering nearly. Okay. So you can take traditional, you know, uh, controls like a, a dpad mm-hmm. And just walk through this world. I guess that ties into like that, uh, the doom Yeah. The generative doom game Exactly. World that they built where you just keep infinitely walking through doom and it just generates it in real time. Yep. Yep. Look, having the capability aside, I think just the notion of having, uh, human interface devices like a keyboard or a, you know, dpad or whatever. Mm-hmm. Into something as complex as ComfyUI. That in itself is encouraging. Mm-hmm. That means that this is a barrier that is very soon gonna be crossed and somebody like Rob Legato or myself or you can easily control all of these complex nodes easily. And then also being able to save, I'm thinking too, that's the other issue of like you're moving, you're navigating, maybe you're changing stuff, but then making sure you're able to like save it and it's persistent. So then the model, it needs to be spatially aware. And have location tracking or some kind of coordinate system in it, which doesn't really lend itself to how a neural network. Actually computes it. Like, it's not like a traditional, how's the, is the neural network remembering anything? Or is it just generating and then it rem So memory is tricky in a neural network, it'll, you know, when it's trained on like a trillion images mm-hmm. It has quote unquote memory of like a cat, a dog, a human, a tree, and individual things. Right. And you wouldn't even really call it memory. It has like a part of the neural network attached to that. Association of that idea. So when you prompt it, what the prompt does is it will, and this is where attention comes in. What I, so you say, you know, I want a cat on a tree. Yeah. And it'll figure out I need more attention on the tree. That is the thing that the cat needs to be on. So secondary intention on cat, it'll go find in its memory what a tree looks like, what a cat looks like, and then it'll put those two things together. Based on what it was trained to be a realistic image. Hmm. So if you, if you're really thinking about how a neural network computes it, it's nothing like what we're actually used to. So how do you actually build in a system with coordinates and 3D spatial awareness? Yeah. Or I'm thinking of like generating a space with like the cat in the tree and a garden, and then I'm like navigating my camera and I pan to the left away from the cat in the tree. When I pan back, I wanna make sure I see the same cat and tree that I just saw before and it's not like regenerating. Yeah. So that, that would have to be built into like a reference. Mm-hmm. Add-on like a fine tune of a reference of that tree. But if you think about it, then you need it for every single object that's in the scene. Mm-hmm. To be consistent when you put the camera back on. Right. Yeah. So it's a, it's a tough problem to solve. But having said that, uh, we're just in such a early stage of all of this that I think it will be solved. You know, it's like the early days of compute when we couldn't figure out that, um, our computers would one day have 24 gigabytes of ram. Yeah. Who were running 16 kilobytes of ram. We're like, there's no way silicon could be packed that densely into have that much memory. Yeah. I was in San Diego over the weekend and I took the tour of the, uh, the USS midway. Oh yes. Uh, which is like a Yeah. Decommissioned, um, aircraft carrier. Been there. And yeah. Fun to cool to walk around. But they had their like computer room or their like radar room. Yeah. Yeah. And it was, uh, what was he run with? The original computers? Yeah. So I think the USS midway is like World War II era. It was, but I mean, they didn't update it'cause it, it, it, it operated in, um, the First Gulf War. Oh, okay. So they did so like floppies. But this was, I don't, I mean, I don't know what era they were gathering. The what? I think they gathered stuff over its operation over the years. Yeah. But it was a like, yeah, univac. It was an old, it was a UNIVAC computer that was like a massive, huge cabinet. And it had like, I. 32 kilobytes of data. Of of, of storage. That's amazing. Yeah. Well, off topic, but uh, I heard that this is one of the main reasons why our, uh, ICBMs all the nuclear missiles in silos, uhhuh are more or less like useless or close to being useless is 'cause they're running on. Ancient machines, Uhhuh, and they don't connect to the modern day internet and none of that stuff. Right. Yeah. Or like our, um, FA Yeah. Is that running on like, um, Fortran or these older languages that haven't been Yeah. Updated in 50 years. Exactly. Exactly. Yeah. Well, all right. Random note, but, uh, we reminded me of just, yeah, the, this was 50 years ago. Yeah, it wasn't that long, you know, uh, it wasn't that long ago, so we're in that moment now. Yeah. We're in the 32 kilobyte moment now. Yeah. We can't imagine. Yeah, exactly. What this is gonna be. Yeah. Alright. Good place to wrap it up. Yep. Thanks for everything we talked about in the show notes. And we'll link to the, uh, ComfyUI workflows if you wanna mess around with it and let us know if, uh, how that works out for you. And if you found any cool controls of how to use AI with alternate controls, always interested in that. And if you're just new to the show and tuning in, guess what? We've been doing this for a while. There's over 30 episodes. Uh, go back and watch the catalog. We cover all things AI media and entertainment tech, so yeah. Yeah. See you soon. Thanks for watching. Catch you in the next episode.