AI Filmmaking Hackathon, Reuters' AI Copyright Victory & Stability AI’s Hollywood Move

In this episode of Denoised, Joey Daoud and Addy Ghani break down an AI-powered hackathon tackling camera control, scene consistency, and AI-generated actors. From real-time cinematography to innovative AI filmmaking tools, we explore the groundbreaking developments that could change Hollywood forever.

Plus, we discuss:
✅ The latest on Thomson Reuters' copyright lawsuit against AI
✅ Stability AI joining the Academy Software Foundation & what it means for the industry

Subscribe to Denoised in your podcast app of choice!
https://vpgo.lnk/denoised

#############

📧 GET THE VP LAND NEWSLETTER 
Get our free newsletter covering the latest news and tools in media creation, from virtual production to AI and more: 
https://ntm.link/vp_land

#############

📺 MORE VP LAND EPISODES

Rob Legato on AI: "Nothing New" [Virtually Everything!]
https://youtu.be/h_YXyW2MWCE

The Best (and Worst) AI Commercials at the Super Bowl. Plus, Will the Oscars Require AI Disclosure?
https://youtu.be/TSdpEdtHH7A

The Tech Behind Cosm, ChatGPT Deep Research, and VideoJAM
https://youtu.be/pjGmMxFrEXc

#############

📝 SHOW NOTES & SOURCES

FBRC.ai
https://www.fbrc.ai

Creative Control: An AI Workflow Hackathon (and the winners!)
https://www.fbrc.ai/creative-control

AI on the Lot
https://www.vp-land.com/p/cinema-synthetica

TwelveLabs AI Hackathon
https://www.vp-land.com/p/twelve-labs-ai-hackathon

Rob Legato
https://www.youtube.com/watch?v=h_YXyW2MWCE

Luma Ray2 
https://lumalabs.ai/ray

Playbook
https://www.vp-land.com/p/playbook-3d-precise-camera-control-for-ai-generated-content

Matt Workman
https://www.youtube.com/user/mattworkman

Mod Labs
https://modtechlabs.com

Dylan Ler
https://www.linkedin.com/in/dylanler/

Comfy UI
https://www.vp-land.com/p/comfy-ai-desktop

Opus
https://youtu.be/gmfFhWGCl8Q

Mickmumpitz
https://www.youtube.com/@mickmumpitz

Thompson Reuters vs. Ross Intelligence AI case
https://www.wired.com/story/thomson-reuters-ai-copyright-lawsuit

Academy Software Foundation
https://www.aswf.io

Raynault VFX
https://raynault.com

Stability AI
https://www.vp-land.com/p/stability-ai-stable-diffusion-3-5

MaterialX
https://materialx.org

OpenColorIO
https://opencolorio.org

#############

⏱ CHAPTERS
00:00 Intro
00:26 AI Camera Control Hackathon
01:37 Rob Legato
04:57 AI Tools
08:17 Creative AI Camera Control
18:00 Creative AI Scene Control 
19:03 Dylan Ler
24:42 Creative AI Actor Control
25:03 What Are LoRas
26:39 Mickmumpitz
32:00 Thomson Reuters vs. Ross AI 
39:08 Stability AI Joins ASFW

#DenoisedPodcast #LumaAI #StabilityAI

Transcript

0:00

In this episode of the Denoised Podcast, we're going to talk about what we saw at an AI camera control hackathon. What happened with the Thompson Reuters AI copyright case? And Stability AI joins the    Academy Software Foundation. Let's get into it. All right, welcome back to the Denoised Podcast, Joey Daoud here. Addy Ghani, hey, welcome back on this rainy day here in LA I know, another rainy day in LA unusual. We'll take it. Let's jump on the first story. Uh, there was a hackathon that I went to this weekend. Nice. Um, and I wanted to kind of break some down, some of the interesting things that I saw. Uh, so the hackathon, it was put on by FBRC, uh, same group that puts on AI on the Lot. And they've done some other hackathons before and they work with a lot, kind of. Similar to what we're doing, that intersection of AI, new tech and, uh, creative industries. Yeah. And shout out to Todd Terrazas, uh, our mutual friend and he's such a strong advocate for AI, especially in LA. And yeah, it puts on such a lot of great events. Yeah. And you know, there's a lot of effort that goes into putting these things on. Yeah. Yeah. And these things are, these things are wild. And so I've been to one in the past, uh, that was a Twelve Labs hackathon, which we kind of mentioned as a joke last time, but, uh, or one of the previous episodes, but not ElevenLabs, Twelve Labs, which does AI, uh, video analysis. Um, but this one was a hackathon around creative control with AI, which is something that is always. It's been the biggest frustrating sticking point with AI. It gives you stuff. It gives you sometimes cool stuff, but it lacks a lot of the control. You can't have cinematography. Yeah. It's an output. Uh, as Rob Legato, who was one of the judges of this thing, he says, it's an output. It's not my output. It's not my creative vision. Um, it's someone's creative or it's something's creative vision. Let's back up. You said Rob Legato, as in. The Rob Legato Academy Award winning VFX. Yeah, who right? So Rob Legato, what he created the parameters of this challenge, and he was one of the judges for this challenge. Rob Legato was the VFX supervisor on Titanic. Uh, he helped develop the original virtual production. We're talking about virtual, like virtual cameras in a virtual environment for Avatar. Did the virtual camera work for Lion King? Groundbreaking. Yeah, every one of them. Someone has been around the industry for a long time and seen all of these huge changes in VFX. Exactly. And Rob is no stranger to having new tools on his hands. And he has for a while talked about AI being just another tool that's in our kit and like, we just gotta learn how to use it. I just want to take a step back and just really do an LA appreciation minute. Only in LA would you have Rob Legato doing an A. I. hackathon. That's specifically for cinematography and film. Yeah, I feel like this is probably the only spot you can pull this off. Aren't you glad he moved out here? I know, I'm glad I'm back. There's fun stuff like that. Yeah, I actually interviewed Rob last year at the Vu's Virtually Everything Summit after he gave his talk and similar concept of, um, yeah, A. I. as a tool. But this was cool to kind of see. Stuff in action of based on like what he's saying he wants. Yes, and seeing what people come up with That's incredible. And these events are cool too because it attracts a good handful of like tech oriented filmmakers But also just a lot of people who are I say this endearingly like tech nerds. Yeah Don't know you but no like Computer nerds who don't know anything about filmmaking, but are wizards at the computer, and like getting technical things to work. And so it's really interesting to see how they approach these challenges. Right. There was a script that Rob created that was sort of, they didn't have to reproduce the script. It was sort of just like, Uh, a foundation, an area to play in, in these three challenges. And there are three different problems or areas that a team could focus on addressing. Okay. And so high level one was a creative camera control. Uh, the other one was a creative scene control. So kind of creating a scene, a 3D space is interpretive, but a consistent scene environment. And then the third one was creative actor control. So people could kind of form teams if they wanted, uh, of like up to five people. And then they had 24 hours. To tackle these challenges, which is obviously not a lot of time. Yeah. And I mean, just for getting like a hundred renders or, you know, whatever inferences through generations, generations. Yeah. That's not a lot of time. Yeah, no. And, um, and the main sponsor slash tool that they had to use was Luma. They did have access to their new. Image to video generator with Ray2 their new their newest model, which, uh, it just came out Monday to the public, but they had access over the weekend as like a pre release thing, because at the time it was only the new model. You could only do text to video and then they added image video, which is a huge improvement because text videos can only get you so far. Yeah, correct me if I'm wrong. That also has API integration. It does have API integration so they can so you can they can use the web interface if they want, or they could use python. A lot of teams use python because they're More familiar, comfortable with that, uh, Comfy. Why? Yeah, you could probably run multiple sessions at the same time. Exactly, exactly. And that's how you get to do things in 24 hours. Yeah, I mean, there is still a speed, you know, even when some of the teams were demoing stuff, it's like, I'm gonna send it and then it's like While we wait, let me talk about something else, because it still takes another 30 to 60 seconds. Yeah, it goes up to the cloud. Your video to come back. Yeah. So every team was required to use Luma, uh, and then they had access to using, uh, additional tools. A lot of that we've talked about, uh, before ElevenLabs, which is text to speech generator, uh, Mod Tech labs, uh, which has a variety of tools for virtual production, but they also. Do which I wasn't aware of they could do a depth map of an image to kind of create help create a 3D Yeah, or a 2.5D kind of space. I don't know what Tim calls it He he doesn't call it 2.5D anymore. But yeah, it is similar to that Yeah, you take a flat image and give and help extract where the layers should be and to give it some depth So you have some yeah latitude to move your camera around Playbook which Yeah, I've used a lot and talked about a lot, but they do, uh, basically a 3D interface space to kind of build scenes with 3D models and have more of that camera control. And then you send it to a, uh, image model and you can get back an image, but have that control over, like, I want my scene to look like this and placement to be here. So more control over your generative image. okay, so for  Playbook, you're building a crude 3D environment. Yeah, no texture. You can just bring in whatever models like cones and spheres and that could be as basic as that. Or you could bring in existing, uh, like we used a diner model set, but it was like a diner, but it was completely, uh, untextured. So it's just like gray blobs that look like diner, like a diner booth and a diner counter. I think, uh, Eric Geisler is showing you can bring in, um, a photogrammetry asset. If as long as it's one of the 3D model types that they'll import. Yeah. I think you can import any. 3D model file. Okay. So from that point and then you have kind of a locked off camera. So it's a 3D space and it's a 3D movable camera. So you can move the 3D camera. Hey, that's like a very crude version of blender. They do have a blender plugin, which they've also said, like, it's probably the, if you're more inclined in with 3D tools, probably the better way to use it. That sounds, yeah, that sounds way more native. Yeah. So let's say you could build a blender scene that's just shapes and of gray shapes. No, no environment, no texture. So then that goes into their model. Yeah. Generates a new 2D image based on where, and then you could prompt it. So like going back to the diner example, you can have it say, Oh, I want it to be a 1950s diner. Or you could be like, I want it to be a steampunk diner, or I want it to be a futuristic diner. But you have that, you line up your camera. So you're like, well, I want. My diner shop, but I want to make sure that the booths are here on the right side and the counters on the left side, and every time you're generating it, you get that. Usually you get that same consistent shot layout. How long does it take to generate pretty similar speed to just any generation? So a few seconds once the servers are warmed up, um, 10 to 30 seconds. Yeah, I mean, can you imagine when that's real time? Yeah, I know. Yeah, that's 24 times a second. And I think that's what we're, you know, that's where we're going. That's where we're going is, is the, uh, aesthetics of the render is AI, but then the, um, construction is 3D, traditional 3D. Exactly. And yeah, but being able to get that real time feedback, there's a very crude version with like, uh, prayer that has a real time generator and you can connect it to a webcam and you can move stuff around and it'll like rapidly generate scenes in real time. Yeah, it's just very, I'm sure the resolution is not great. No, it's not. Okay. So. The teams that I'm going to run through, I'm just going to call out the teams that either one or stood out to me because I didn't agree with some of the choices. But, um, okay, the first one, creative camera control track. So this, the goal with this was to provide more kind of what we were just talking about with  Playbook, more ways to replicate how filmmakers really work. And Rob kind of gave a whole speech in a demo using, um, his, uh, camera wheel controls on his computer. He's like, I like to do this, you know, when I'm in Unreal to like work on my camera movements. And, um, Yeah, if you're good with that, you don't need a V-CAM. Yeah, that is the most precise way to control a camera. Yeah. Um, that was like, you know, that says like, I want something like this. The team that did win, um, they Basically connected a MIDI controller, so you know, like a little, yeah, bunch of buttons. A little USB board with like a bunch of buttons. Some pads. Some pads, yeah. And knobs and programmed some of the knobs and stuff to basically, I believe it was like running a Python script to adjust to prompt, but to like have adjustments in camera movements. So like one pad would be a pan. One pan would be a tilt and Yeah. Or I think if you turned a wheel, it'd be like pan plus three this way, plus five or something like that way. That's neat. I like that. And um, Luma Labs does have. Camera control kind of prompts and stuff and understand some camera control movements. It's all in the, uh, method in which you prompt it. Yeah. There's a certain language. If you're on their web platform, it'll pop up if you start saying camera and then it'll pop up with these specific word to use like camera, orbit left, orbit right. Right. Um, I assume if API, there's a, as long as you use this specific word. Yeah. It'll understand it. The camera wheels. It feels like Etch a Sketch a little bit. Yeah. Have you ever tried to use it? A long time ago. Yeah. Yeah. It's, it's difficult for somebody that's never, it's not into it. Yeah, I mean, it takes, uh, a while. Yeah, and if you're listening or are not familiar, I mean, it is using two wheels. It's a tripod with two wheels, one on the left side, one behind. Right. One adjusts the tilts, one adjusts the pan. The pan. Yeah. And. It's yeah, it's not the most intuitive, but it is extremely precise because you can also just how much speed or movement happens with each revolution of the wheel. So if you want to have like very slow shuttle details, you can like adjust that if you want very fast movements where like one spin is like it rotates a bunch of degrees. Yeah, so yeah, that's the highest level camera operators. That's that's how they're operating. Yeah. Do you know who Matt Workman is? Oh yeah, with a digital um. Virtual database digital. Yeah. Yeah. So he does a lot of cool Unreal development work and uh, one of his videos I saw that he took one of those things that has uh, like a usb input into a computer. Yeah And he did a whole plug in in Unreal. I've said yeah, I saw some of his videos, but that's pretty cool. That's cool Yeah Um, team that it was a Team Bold Control. It's I don't have everyone's name, so I'm just gonna go by team names because that's what was in my notes. But we'll linked out to if they post the winners will link out to the final sheet with everyone's name. Um, so they won with their midi controller. So it was cool and innovative. But I felt like I mean, I felt like it didn't go super far beyond like something you could just kind of prompt yourself because it's not real time. I think, you know, I mean, they definitely took the challenge of like adding a different. Way to interact and interface with it, but, um, but those are the, those are the micro innovations that lead to something big for sure. Yeah, it's a good idea. Right. Um, all ideas are welcome at this point, right? Um, but the, the one that came in runner up in that category, uh, I thought was really impressive and sort of all the bits and pieces they put together. So their team was Team CBB And first off, they kind of built, uh, this was all 24 hours. So they built sort of like a full web interface, uh, like a user friendly interface and you could give it the script and then it would analyze the script and do location breakdowns of like kind of figuring out what the locations were in the script. And then it would generate like a mood board concept board of like different ideas based on the script and the script descriptions of the scenes. It would make images and then you can kind of pick the images and then it would. Um, upres them. You could do some inpainting to like modify some of the images. It would turn it into a 360 panorama. It used the ability relight to like modify the scene. Um, and then they ran it through Mod Labs, which would make a depth map. And then they would pull that into either Unreal or they actually pulled it into, um, Fusion and DaVinci Resolve, which I was really interested to see that, you know, yeah, into a traditional non linear editor. Yeah, they pulled into fusion with the node base the effects pipeline and then brought in some like kind of actor stand ins and we're like now then you build your environment and then you can start planning out your shots and kind of do like a cinematography, virtual cinematography and camera planning. So bringing You create this 3D environment that you could either use for prep or you could put it on a wall if you have a wall and, um, it's a 360 space with some depth so you can kind of move the camera around and turn around and you have the same consistent environment. So Team CBB, in my opinion, like hit a home run. I thought, yeah, I thought they really did a great. Josh, especially for 24 hours and like all the tools they connected and everything that they were able to do and make it. I mean, a lot of this stuff. Sure, you could do it if you know all the pieces, but they really kind of built a, can you tell me a little bit about the actual actor in the scene? So you would still have to shoot the actual actor with a camera. You could, I think there's use case was still more of like, this is a prep planning tool. Okay. So The actor demos they showed were like, um, silhouette kind of cutouts. Um, or I think some like generate their tool might've also generated based on the script, like some rough character, uh, outputs based on the character descriptions in the script. I don't remember exactly, but I remember the demo video they put, uh, they made it was the environment they created and then they had the actor in it. And then you could move your virtual camera around to like. Virtually shot plan in your new generated 3D space and the actor was a separate asset from the background. It was separate Yeah, so I mean I think it is encroaching into the LED volume ICV effects territory, right? Yes. How do you see this? I mean, this is I mean their stuff was definitely in the like this is for prep. I think they mentioned But if it's good enough, you could load it on a wall. Yeah, if it's good enough, you could just go final with it. Yeah. Yeah. And the fact that this plugs into something that a normal editor or a normal VFX artist would understand, like Fusion or Resolve. Yeah, or Unreal. Unreal, yeah, that's the cool part. It's like, it's not just this new thing, it kind of plugs into things you already know. Your existing pipeline, yeah. Right, you could use this as their web platform to start building your environment. Someone who's non technical can build the environments is what I want. And then it goes into the pipeline to how was the quality of the stuff they were generating? Uh, I mean, you know, good for previous, definitely good for previous. Um, and like all of these things, uh, it depends what kind of shots you need, you know? So sure. If you're trying to do a couple inserts or close up shots and you need to like a environment for the background. Uh, yeah, I mean, I think it would work. It looks good enough. You know, when you get wider with a lot of these shots, that's when everything falls apart. These hackathon is where it's at. Like, yeah, that's where you see, that's where you see the film and the tech combine. And in like a forced 24 hour period of like people kind of doing that was the whole point of hackathons, you know, just what can you whip out if you just have this intense focus for a day or two? Well, also because, um, and I go back to sort of like the bigger companies that are trying to do this, right? Your, you know, uh, Autodesk, your Adobe, your Netflix or whoever. Like these big studios that are trying to build tool sets in sequence, one that interconnects to the other and it eventually outputs really usable stuff for film. You get a really good glimpse of that. Maybe even an early glimpse, six months to a year ahead of where this technology can be. At these hackathons. Yeah, because these, these guys don't have to build it for at scale. They'll just build it for 24 hours for this thing and it's gone. Yeah, these are all proof of concepts. Yeah, I mean, I don't have to keep developing these, but exactly, but there's IP there. Yeah, I mean, the stuff they've done is really impressive, even for I think it's also a good indicator tool of, um, we haven't really talked about it much, but all of the. No code or A. I kind of coding tools like bolt and cursor and some of the other ones where you can kind of build apps just with A. I, um, there's a lot of that, uh, and websites and all sorts of stuff. That's just you can kind of just generate. I don't know if they use them, but I would imagine for because like a few of these. Teams had like full fledged, like landing pages and like, like, like some teams were just like, you know, we built a kind of a working prototype, but like our demo we're running in python in code and we're just, you know, there's no, it was not user friendly, but it was like, you didn't have to be for this. But some of these other teams, it's like we built a whole like sass platform to you, you know, that you can make an account and you could upload your script and you can, uh, you know, it's a very user friendly interface. AWS accounts set up with their own thing running. I mean, I, I sort of pictured this as like a high school championship game where the college, uh, agents and scouts are just sitting in the bleachers looking at, like, Which high school athletes to pick up. I mean, that's exactly what this is. You see, and it's not just the talent, but also the ideas and IP here. And I would imagine, I mean, if I was part of a big studio that are, that's trying to innovate and build an entire pipeline, this would be the place to go for just Unique. I agree. I mean, these are the people that kind of know how these things work and know how to know how to connect the dot. Yeah, exactly. Which is a big part of, of this. Wow. Yeah. Amazing. So yeah, so they, they were second place runner up, but I thought they're what they built was really impressive. Um, okay. For the second challenge, if a team and remember this was The teams would pick what they wanted to focus on. So it wasn't like everyone was competing and everything, uh, creative scene control track. So this was more about even though I know it sounds like the, maybe that's why they didn't win that category. Cause the one I just described sounded more like a scene consistency. Sure. Um, and some of the teams admitted they're like, yeah, you know, just because camera control and scene control kind of are two different things. Well, but sometimes they kind of based on what they're trying to build, they kind of hit both things. Cause it's like. If you're building camera control, you kind of need a scene to operate the camera and so and you need the same scene. Yeah, because then it kind of defeats the purpose. Um, so this one was so creative scene control was more about building a consistent environment. Big issue with a I make an output and it's like, okay, cool. Let me make another one, you know, with change the character, but put me in the same scene, but it'll be completely different scene. Um, yeah. So this was like, how, you know, how can you get better continuity of being in the same physical space and the Team or person who won because this, uh, Dylan went solo is a Dylan, Dylan Ler and, um, talking about football and, and recruiting and the scouts. Yeah, Dylan's been on my radar for a while because Dylan at the last hackathon with Twelve Labs. Dylan also operated by himself and single handedly won that hackathon. Wow. Uh, first place. We should have him on the pod, man. I'd love to talk to Dylan, yeah. I've never talked to, actually, I talked to him after that. He's just like, well, it's not that complicated. You just like, plug all the things together. It's like, what? I know, that's the thing. People that are really, uh, at a proficient enough, yeah, they're like, wait, you can't do this? You don't see all the connections? Yeah. Um, so he had, uh, his team name was Team A Thousand Hands because He was like, I basically just built robots that gave me like a thousand hands. And I think his example is also a really good indicator of where agentic AI is going, which is like the big buzz word from Nvidia, from Jensen's, uh, keynote and kind of like the new direction everyone's like talking about the next step with AI is agentic. And he just built, uh, all of these little platforms and connections and we can show, we can link to, um, his flow chart that he, he built. Um, So I'm seeing three tracks that are working in parallel and then coming down to a final video block. Yeah, and let me show if I try to remember what he did basically would have chat GPT go run off and sort of create a prompt describing the environment and then reuse that environment prompt to describe different elements in the scene and then. I think he also had sound effects being generated to and would sort of combine all of that and then combine it with like a second scene and a third scene. I don't remember how he did this, but he basically would combine all of this into one video output. So by having a single video output, it's doing one generation. So if you have a shot looking one way, a shot looking the other way, This environment is consistent because it's it's a single draw of the dice. Exactly. Yeah, so it is generating it. So it was generally needed in the same output. Yeah, and so his scene environments would be consistent. And he was running that scale so he would like eat the script and then he just let the AI kind of run off and interpret it itself. So there were outputs that, you know, the visually, some of the things were just kind of crazy. Yeah. But his challenge was like seeing consistency. And if you look in the background, it's like it would show like a wide shot or a close up or something. And the environment, the scene would look the same, even though maybe there's like crazy stuff, like it's not like a final pixel output type thing, but like 90 percent demoing what he was trying to achieve and he's running these at scale. So he wasn't really having any input of his own or doing any of the prompt generation. He was having the AI 1000 hands do his work. Yeah. And he would run like, you know, have 10 20 videos processing at once. Wow. Uh, and he's just like, yeah, you just kind of keep Running and running and running to get all these outputs. And so he probably has like 50 different chat GPT sessions generating the prompt. And then, uh, it looks like after that there is a video generation in between video generation layer. And then finally that video goes into the scene generation. Yes, and then it adds the sound effects and then all of these like in this depth map It's like three scenes are combined into a single video. That's amazing. Yeah, so yeah, and it's a really way to like attack the problem Yeah, like with a thousand hands as well as CBB. I'm noticing that there is a trend of Layering and sequencing things in order to get what you want. Yeah. It's never like one shot in, one shot out. No. And yeah, this was, I don't know how he built it. I don't think he used Comfy, but a lot of the other teams use ComfyUI. And some of them showed their node graphs. And it's just like extremely complicated, like lines and stuff connecting everything. And um, yeah, I'll talk about one of the other teams that use that in the other challenge. But yeah, it's not a single tool. It's not a single tool. It's connecting a bunch of things together. I mean, that is what a hackathon is. But just stepping back and looking at how they're approaching the problem. It's a classical way to approach any engineering problems is to break it down into subcomponents until it gets more and more granular. And then you can actually solve for that individual thing. And once you have a solve for all these little things, then you can go back up and solve the big problem. So that's, yeah, I mean, they're approaching it like engineers. Yeah. And I mean, he isn't, he's like, I don't really know much about filmmaking, but this is how, but this is how I code this problem. Yeah. Yeah. Yeah. Uh, and so, yeah, so he won. Yeah. Super impressive. He did a similar thing with Twelve Labs where the challenge was building a, uh, there were a couple of challenges, but one was like a highlight reel generator from like sports clips. Yeah. And his was like. Sort of had the basic web interface, but it would pull the sports clip and then like transcribed it and then like found really good highlights and, um, edited it into like social videos and it was like all like automatic and kind of like Opus, kind of like Opus, um, but on, on his own, on his own, but I think I'm trying to remember it was a while ago, but it did more than what Opus was. does. I think you could also chat to it and ask it questions about the what happened. And this was before an AI years, like a million years ago, this was before like a Notebook LM or all those things. So like a year ago, less than a year ago. Yeah. Yeah. So yeah. Impressive stuff. Um, and then, uh, the third one was creative actor control. So this was about creating actors, but how trying to have more controlled before. How they look, the AI actors look. And so the one, the team that did win, I think their name was Team bldrs, it might have been something else, and the name got mixed up in the, uh, transcripts. They took an interesting approach. So they, they use LoRas? Yeah. Do you wanna explain? Yeah. Yeah. LoRa, I believe stands for, uh, lower order ranking adaptation. So you can take a existing model and then, uh. Pre train it enough using a LoRa, so it will kind of, you're kind of taking this thing that is shapeless and giving it enough of a shape to get the output that you want. And so like, use cases would be like, if I gave it like 20 images of myself, I could make a LoRa about, a Joey LoRa. It'll be like, uh, so the main engine would still be something like Flux. Uh huh. Uh, but that Flux with the Joey LoRa would only generate Joey's. Yeah. Yeah. So it's a good way to train to like, if you want a specific output of a person or a product or a few people. Yeah. Yeah. So the challenge with LoRas is with people is where you're, if you're trying to get different expressions, it tends to sort of just replicate whatever the training images were. Right. So I think, and I think if I'm understanding this right, what they did was they went around to different people and were like, act shocked. And they took a bunch of. Pictures of people at the challenge that day. Yeah. With different facial expressions. Yeah. And then they trained LoRas on, not on a same person, but on the same expression. Oh, that's a very cool approach. Yeah. And then when they're like, oh, we wanted to have this, uh, you know, we wanna modify this, uh, AI generated character based on the script and you know, what this description was, but we want to have them, you know, have a shocked reaction. Yeah. They ran the shocked LoRa. And got way better outputs of like expressiveness from these AI generations. This, this approach is not new. It's actually from the 60s. Uh, during the, uh, sort of figuring out what all of our faces do, there was a thing called a Facial Action Coding System, FACS. Okay. And I think there's about 200 different shapes our faces can make. And so the scientists in the sixties went around, surveyed like a thousand people, had them do shocked, do laugh. I think I've probably seen these. Yeah, wink your eye, wink your left eye. And, uh, in facial capture, even like, you know, 15 years ago, I remember that was the basis of how all of the facial capture systems were built on. So when a computer vision model was built to detect a wink, it would go back to the facts of what a wink is for a thousand people and the average of that. And so I feel like facts is back all over again in, you know, consistent character control. The other thing, uh, I think a couple of the teams may or may not have used, that's actually quite a, it's quite a powerful tool is, uh, open pose and pose net. Mm, these are Comfy. These are Comfy little blocks, and they help train the LoRa. Oh, okay. So a LoRa, uh, could be trained on, uh, a depth map. It could be trained on poses, it could be trained on a bunch of different things. Like you can give it a gradient if you want. You know, how would you use that? Like if you were like training it on a po or, I understand a post, but like a gradient. Yeah. Um, so I don't know. You want gradient outputs or something? Like, you want something like, yeah. So. A LoRa is essentially a mini trained model that's having a big influence on the major model. So you can give it things to kind of train it. And, um, here, PoseNet would be how you would get the character sheets. You know, I'm sure you've seen like characters in a T pose, characters Yes, yeah. That's actually the, the, my honorable mention one, but yeah. Yeah. Um So with open pose, you can have a face do a wink or a smile and you just have the outline of the lips. To get more of that. You have an outline of the eyebrow. And then feed that into the LoRa and then you would get emotions. But this approach of training specific emotions takes me back to fax. Yeah. Which is like early days of facial capture. Yeah. Yeah. Uh, yeah. Full circle. Full circle. Yeah. Uh, yeah, so they won the character control and I thought that was, uh, and the outputs that they demonstrated were like impressive of just like being able to, you know, take the AI generated image, but then give it that, that actual like more human reaction and based on actual human reactions. Were you impressed? Yeah, I was impressed with this one. That's cool. Um, and then, yeah, the other one, I don't know if they won anything, but they're called rough cut. And they were another one that kind of built a full fledged, like functioning website, like web interface. Um, and it did a bunch of things. But the one that stood out, there were ones where they kind of built their entire back end on ComfyUI. And these very elaborate, they kind of showed the whole ComfyUI model. And it's like very elaborate, complicated, but the they built the easy to use Web interface where you can just give it an image of your character. I think part of the process was it could also generate the image of the character. And then you give the image and then it runs this complicated, Comfy workflow and it gives you a character pose sheet. Yeah, um, that is, that is like the, the golden thing. Yeah, because it enables consistency. You can use that character sheet to train a LoRa. Yes, exactly. Get consistent character. That's all LoRa needs, really. It's like 10 to 15 different images of this character in different angles doing different things. And that kind of ties into going back to Dylan's project where he's like, because he's just doing Single video output, but by doing the single video output, you can have those different shots with the same scene. Yes. So with the character sheet, you give it this one input and it's generating a bunch of outputs, but it's the same. It's running the same generation. So you get that consistent character sheet. Yes. For the output. It's kind of a similar idea. Character sheets are image based and then Dylan did something that's video. Yeah, exactly. Exactly. Um, Yeah. So, yeah, I thought that I mean, if they, uh, because I've gone down the rabbit hole of YouTube tutorials and there's, um, I'm blanking on his name, but he posts really, really good ComfyUI tutorials. But every time I try to replicate them, uh, they're very complicated. And I always hit a snag. Yeah, I follow this guy named Mickmumpitz. Yeah, he's German. Yes, yes, he's amazing. Yeah, yeah, he was excellent tutorials and I tried to download his workflows and then I always hit bugs. Um, so this thing kind of does what he shows, but it's just a web interface. And I'm like, I would use that right now. Yeah, that's, and uh, going back to just everything happening on hackathon like, this is where the big enterprise tools will end up in six months to a year. We're just seeing early glimpses, and of course, they're pulling away at the races because they're so much nimble and faster. Yeah, I know, they can, right, it's hard for a company to They don't have to worry about internet security or legal contracts and things like that. They could just go for it. Uh, so yeah, overall, yeah, it was really fun to see what these teams did. And, um, it's a hackathon 24 hours. Really impressive what they built, but like all these things, it's like Not just what they built now, but like what they can do now, but like where it's going to be six months from now. Yeah. The next hackathon, when that comes around, give me a shout, man. I want to be there. Okay. All right. Uh, our next story. Yeah. So Thompson Reuters, uh, a big media firm, uh, they do a lot of news, uh, Reuters news, you know, covers worldwide coverage of everything. There was a, AI company called Ross AI that supposedly used Thomson Reuters you know the gigabytes and petabytes of news and Information that they have to train a model and then Ross AI was essentially spitting out Different versions of what Thomson Reuters spits out, like a version of the news, if you will. Yeah. Um. I think it was just training, it was, I think it was focused on their like legal coverage. Legal coverage, yes. Or their like, yeah, their uh, law vision. Thomson Reuters obviously took offense, went to court and uh, filed that, you know, uh, this is uh, not fair use of our property. Mm hmm. And that um, Ross should not have the use to use any of Thomson Reuters data. And the judge ruled in favor. And the judge said that this is indeed not fair use of Thomson Reuters data. So this is not in our industry. This is obviously in the news. Yeah, but like the law journal or the law. Yeah. Yeah, the legal. The legal side of things. But it has. Here's the thing about. This is one of the first. AI. This is one of the first AI decisions, right, from a company suing another company over using their data. Yeah, and that's why you and I covered the Oscars AI ruling, and that's why we're going to cover lawsuits and the results of it. Yeah, because this is the big factor of these companies holding off on going all in on AI. Because it's like, well Look at that. Can we, right? Are we going to be Is it protected? Are we cool? Is it copyrightable? Are we violating copyright? Yeah. End of the day, a business is here to generate revenue, and lawsuits will significantly impact the ability for you to generate revenue. Speaking of lawsuits, I mean, when I saw the story, because I think the lawsuit came up in like 2020, and when I saw the story and I was like, wait, Ross? Yeah, intelligent. Like, what is that company? They went out of business because this lawsuit buried them. That's it. Like, yeah, they went bankrupt because of this lawsuit, right? So whether they were found again, now they're also found whatever liable. I'm not sure what. And they were, uh, the, the thing that I found interesting and that definitely applies to our industry and media and entertainment is they were going under the cover of, uh, parody law and some of the fair use, right? So fair use for potential criteria. That's something to be covered under fair use, uh, educational. Uh, parody, which is where like a lot of like Mel Brooks stuff kind of falls Saturday Night Live, uh, Saturday Night Live. Yeah. Um, why they can make fun of like existing things, but it's not violate copyright because it's covered. It's a parody. You're allowed to make fun of people. Yeah. We're aware. It's a joke. Yeah. Yeah. Trying to represent them like in a, in a. Yeah. I don't know quote legitimate way. Yeah, you're not replicating them. You're parodying them. But yeah, all was rejected. So I think going back to all of the Legal sort of questions that arise in our industry Some of if there is ever a lawsuit in M&E I think the the lawyers and the judges will go back to this case and kind of look at the Use of language and things like that there. Yeah, there are two things about this or I think two maybe kind of specific separate things that were part of the case. And I thought were interesting, uh, because part of it is they trained on Thompson Reuters data and was the training part of violation of copyright. That seems a little fuzzy because the other part is the output, the inference, right? And I think of one of the articles, uh, there was a quote from the judge's decision in this article and it says, uh, the decision says, quote, It's undisputed that Ross's AI is not generative AI because the program does not quote, generate, generate new text in response to a question, but rather regurgitates verbatim quotations from published judicial opinions. Yeah, so it seems like an alphabet soup and not a neural network, right? Yeah. It seems like it was not sophisticated, uh uh, search engine and that kind of tracks because 2020 was still not. When generative AI really took off. Was that even with GPT 3. 5 even out? Or was it still like 3? Yeah, I mean, I think it was the days of AlexNet and uh, some of the stuff that Oh, was there even GPT yet? Yeah, it was like the OpenAI founders were working on that stuff back then. But uh, yeah, absolutely. If it was to generate, I think that perhaps the judge would say, well that is a new thing and it has nothing to do with it. And that's where I'm curious in the future because Uh, yeah, I would say the outputs we get now are new transformative and that's another kind of classification of a fair use is, is the new output transformative enough that it's different from the original. Maybe I'll clarify. We're not, this is not legal guidance or we're not lawyers in case you haven't noticed. Don't take anything we say as a legal opinion. Um, yeah. But yeah, having worked in a lot of documentary space, which uses a fair use a lot for editing videos and stuff. Um, is it transformative enough? Are you, is it, are you explaining something enough using just enough to sort of explain and make your point? Based on what someone else said, right? And not overusing your the source material of someone else's car. Absolutely. A little bit different from generative AI. But yeah, I feel like the outputs today are it's something new and different. Yeah, even if you like if it's an image to image and you give it, uh, let's say an image of a beach, like a photo that you took. It's not going to generate that exact same thing. It's still going to go through the computation and then it's going to generate a beach, but it's not pixel to pixel the same as your input image. Exactly. Yeah. So it's generating something right. It's new. Yeah. So this is an interesting one. I feel like it doesn't really settle much on. Yeah, current kind of more of the creative space, uh, copyrights, but at least it, uh, lets people know that there is definitely law and order in this space. You can't just copy and steal. It takes a while. I mean, this is four years for a decision to come out. Um, yeah, I think the other two factor too of, of the decision was, was, um, Ross was like a, Building a direct competitor product on the same exact thing and taking all their data and like spitting it out. So it's like very much like it was predatory. It seems a little bit more, yeah. Like kind of like when we're just copy and pasting your stuff, building our own thing. No bueno. So yeah, they're not really not a lot of transformative stuff. Yeah. And again, uh, I just want to go back to. Why we cover this stuff on the show, because I think the ultimate deployment and use of AI in film and TV is going to be these little steps where we see the ethical stuff progress and get it get worked out. However, it works itself out. Yeah. So we'll just cover it along the way. Yeah, for sure. Because, yeah, I mean, yeah, even though this is not nothing to do with media entertainment, it's AI and training and that's going to ripple and effect because there's a bunch of lawsuits and other things happening right now that we're everyone still waiting to see how it plays out. Yep, absolutely. Raynault VFX and Stability AI joined the    Academy Software Foundation. Uh, so the one that stood out to me most here was Stability AI, which is the company behind Stable Diffusion, which is a huge generative AI, large language, or how large language model is it? Yeah, remind me who's on their board again. Uh, James Cameron, uh, just recently joined their board. Yeah, and Sean Parker from the Napster, Facebook. I didn't really, yeah, until we looked that up. I did not realize that Napster, Sean Parker. Yeah, the best line in the, in the social network, you know, what's cool? Not a million dollars, a billion dollars dropped the cleaner. Uh, so yes, that stood out to me the most. Uh, well, first off, what is the    Academy Software Foundation? Yeah. So, uh, the    Academy Software Foundation is an open source body that. Enables the cross communication and the cross adaptation of open standards. So part of the academy, the academy, like the Oscars, yeah, the academy, uh, it's headed by, uh, one of my friends, David Morin, amazing, really nice guy. He invited me to one of the meetings once. And so I got to attend the ASWF meeting. Um. A couple of the things that came out of ASWF, uh, MaterialX, so they work on USD, which is an open source file format for 3D. MaterialX is an open source material definition for 3D shaders. So kind of trying to have like open source file formats and other workflows, so not one tool or company. Owns like a proprietary format like things can kind of communicate between it doesn't help anybody. Yeah. Well, I think over the last 20 years VFX has seen enough proprietary formats to now where they're realizing that. Okay, an open standard actually helps everybody because then you don't have to develop that thing. You could Yeah. Build on top of it. You can do stuff. That's way cooler than developing a file format. Just take the file format that everybody's using and then do cool stuff with it. So, uh, there's a couple of things in our world. Uh, I think, uh, open timeline IO is from ASWF if I'm not mistaken. Uh, so that's for like editor timelines and stuff. I never even heard of that one. What is that? What is the open timeline? I mean, cause like, I mean, they're edl or xml. It's it's yeah, it's I think it's the way to define your edl and then also how a software will build how the timeline is done within. I mean, that'd be great because yeah, it's always a issue when you're like, this was cut in premier. It's all defined. Okay. Yeah, that's one of them. Um, not to be confused with OpenColor, although I think OpenColorIO is represented by ASWF, but that was originally developed by Sony. Okay. Uh, all the major two was Raynault was one of the developers behind OpenColor. Yep. Yep. So it's interesting that stability. A. I joined such an, um, industry wide body because that just means a I is coming full force into mainstream digital content creation is seeking to integrate its generative AI models into existing production pipelines. Yeah, I mean, there's a big indicator of like where generative AI yeah, yeah. In the Hollywood industry. Yeah. And, uh, it's a good move on their part, right? Not making stability AI a proprietary thing. No. And they've always been one of the kind of open models like stable diffusion, stable fusion XL. It's always been open, available, uh, to, to use. And they have products that are not just image. They also have 3D products and I think sound products as well. Relighting one that we mentioned in one of the, the, one of the teams use, which I wasn't familiar that they had a relighting. Um, Uh, model or way to, to relight scenes. Yeah. Um, yeah, I mean, they've been building a, a, a lot of different tools. Yeah. Interesting to see them here. And there are the I believe they're the first AI, full AI company. I mean, there's other members. There's Nvidia, there's Adobe, Intel, Microsoft. So they have obviously have parts of AI stuff, but I think it's the full one where it's like the company, the entire company is built around generative ai. Yeah. That is a member of this. I would compare it to like, uh, United Nations, like you go there, everybody from every studio is there. It's not free. Like, I think you do have to pay for a seat to be there and then, uh, especially to vote. I would compare it to sort of like SMPTE a little bit, where SMPTE is more focused on television and broadcast side of things. ASWF is more film. It's the Academy. Okay. Yeah. Yeah. So I'm curious kind of to see what. Comes out of this by . Yeah. I mean, look like, uh, here's a quick example. Like, uh, you know, ASWF, uh, controls, uh, I'm guessing ais, which is the academy color encoding system, all the color spaces. Mm. Okay. ACEs color spaces. So. One of the big issues with AI is it doesn't generate color space compliance stuff. Yeah. It's like you don't know if it's, oh, that red, is that Rick Seven nine red? It's, I mean, it's giving you, yeah. Less than hd, not the, yeah. Great. Of course you can conform it and just say, yes, that red is that. Mm-hmm . But what if stability AI takes it a step further and you can have color gamt controls. I'm just giving you an example. Yeah. I mean, that goes into the whole hackathon kind of idea too, of just like more control. Over, I mean, more camera control and all those other things are like top of radar topics, but color pipeline and consistency is equally important when we're getting into the high level. Absolutely. And then a lot of the AI generation is still 8 bit, you know, and we're in a 10 bit world moving to a 12 bit world. Like, where is that at? Yeah, for sure. There it is. Super important. Uh, well, cool. I think that's a good stopping point. There's a good, uh, a lot of good topics today. Yeah. You know what I realized, Joey, is there's so much going on that we can't cover it in an episode in a few days. There's a lot. So, uh, I really encourage you guys to join and, uh, give both episodes a listen per week. So, yeah, I mean, so we're aiming to have a consistent schedule. So that's going to be, uh, every Tuesday and Friday morning. Uh, you can check out whatever your favorite podcast app is of choice and subscribe to the Denoised Podcast and we're still in an early stage. So every like follow and every thumbs up is still highly appreciated. Thank you. And, uh, show notes and everything else we talked about. You can get at Denoised podcast.com. All right. Thanks a lot, everyone. We'll catch you in the next episode.