Creativity Jijiji

Kristopher Eng on Adaptive Music and Gaming Narratives

• Chris Mchale

Share episode

Join us for an inspiring conversation with Kristopher Eng, a founder and composer at House of Elias, as he shares his journey and insights on adaptive music in video games and beyond. Kristopher's pioneering work in adaptive music for video games is not just about sound; it's about transforming the gaming experience into something engaging and authentic. We'll also hear about his fascinating path from the Royal College in Stockholm to developing the Elias 4 platform and how adaptive music can play a crucial role in storytelling.

We discuss the challenges of integrating advanced music technology, the importance of setting industry standards, and why game studios embrace these innovations for more immersive gameplay. Don't miss this exploration of how music complements and drives the gaming narrative forward.

Thanks for listening.

Check out our Linktree

https://linktr.ee/studiojijiji

Speaker 1:

Hey, welcome to Season 2 of Creativity's Jijiji. And the world has flipped since I did Season 1. Since I ended Season 1, it's a new year and it's a flipped-out world. I mean it's crazy, right? I mean the crazies have taken control of the world. I mean it's it's crazy, right? I mean the crazies have taken control of the world.

Speaker 1:

What do you do? What do I do? I write and I compose music and I produce and I paint and I cook and I do anything I can think to do to express my creativity and to push back on the motherfuckers that are messing with us. You know these guys aren't going to be around long. That's just the way it works. You know politicians, they come and they go, right. Who is the mayor of Vienna when they dumped the pauper in the grave? Right? You know who I'm talking about Mozart. You know Mozart. Who's the mayor? Who's the mayor? Who's the president? Who's the king of the world? Nobody knows, but everybody knows the pauper, the composer, the brilliant, shining light. That's who I follow. The artists, they are our leaders, they are the soulful ones, not these motherfuckers pointing fingers and blaming everybody, saying fuck you, fuck this, forget it, dig into your creativity.

Speaker 1:

This is Chris McHale and I'm excited to be back. And when I jumped in with both feet on this podcast, I was clueless. I'm still clueless. I just try to be as creative with this podcast as I can. I listen to a lot of podcasts. I try to do something different. You know just the way I am and we have some fantastic lineups coming, some guests from all corners of the creative world and I'm going to do a couple of solo shows and I'm going to do a couple of shorter shows, because people have been saying like oh man, chris, you're going on, so okay, and we're going to be asking our creative visitors what inspires them, how do they work, what techniques drives their crafts, how does one carve out a career in creativity in the first place? And is that important? No, it's not important. Actually, it's not important at all.

Speaker 1:

Today we have with us one of my favorite guys to talk to, christopher Ng, who is the founder of the House of Elias, a composer, an entrepreneur, and Chris and his team of software geniuses on the verge of releasing Elias 4. If you know the video game world, you know the musical platform Elias 4. It's innovative, it's intuitive. It actually has a lot of applications beyond video games. We're going to get into that a little bit and we're going to dive deep into how it works, how Chris works, and the creative process behind his thinking. You know the future of adaptive music, and if you don't know what adaptive music is, well, look it up or just listen to this podcast as we get into it with Chris. I started off by asking Chris, you know the question I love to start with, which is you know, where did you go to begin? Where did you begin finding your way through the maze, through the labyrinth? How did you do it? And here's Christopher with his answer.

Speaker 2:

Having been writing for a lot of theaters and things like that, coming in directly from the Royal College in Stockholm. After a couple of years I got this call from my friend that said you really need to be a part of this awesome game, and it really was awesome. My role actually was to record new sounds that hasn't been heard before, and also I sampled all the percussion in the Royal Orchestra in Stockholm and I sampled everything I could find and we used these sounds together with a real live symphonic orchestra and it was really, really, really cool to be able to aim for whatever we needed. No, we didn't need sound libraries and things like that.

Speaker 2:

We did everything from scratch and authentic and I started to write down some ideas of my own for the new tech that would be more suited for music and games, and I actually got a patent for that in the US and I started a company and after that I became kind of. I was the CEO of the company and I worked with that for many years like eight years or so with only you know raising money, go to different conferences and try to make this work, and the company actually got quite. I think we were like 20 employees or something like that. And but then you know the first we had the pandemic and then the war in ukraine started. Everything just collapsed.

Speaker 1:

That was actually the best thing that has ever happened to me you created a software you know how would you put it a software engine. Yeah, that brought a more musical experience to composing for video games, so it sounded more, more musical right.

Speaker 2:

Yeah, to me the thing that I thought was lacking in my first game project was people talked about adaptive things in other games, but to me it was not really adaptive. For me, true adaptive music is when it sounds linear, it sounds written, it sounds like it's supposed to sound, and to be able to do that you need to solve a lot of things. You need to make transitions sound right and also be able to use motifs on top of the music, and you can't have it work with just a stereo track. You have to work with stems and things like that. So a lot of these things I put down into Elias, Because even today I get a little bit allergic when people only strive for adaptive music. You have this bubbling music here and there and then some drums comes in and it's not linear. It's not what we. We have like a 3000 year of tradition, what music is. So we have to keep that in mind when we talk about adaptive music. You can't do adaptive music to the sake of really quality music, game music.

Speaker 1:

It doesn't just accompany the story. You know, it's not like a film score, it actually has to live within the game.

Speaker 2:

Yes, so your, your challenge is to do that as a composer to write music that is accompanying the story but is live today I'm working with the new company called House of Elias and we brought in the tech again and we're actually going to release a new version of it, and it's also going to be for everything that sounds in the game, so sound effects and everything. But in this new company, my role is actually to compose music. I, of course, use the tools and so on, but I'm back to being a composer and that's the important thing in my life.

Speaker 1:

If I gave you a film and said, chris, score this film. You would take a different approach, I think, than if you're sitting down to compose for a video game.

Speaker 2:

So the biggest difference is composing music for a game. You're composing for a story that isn't written yet. You're're composing for a story that isn't written yet. You're actually composing for a lot of things that could happen, and that's number one. And the easiest way to do that would be like doing a bunch of music and you cross, phase between them and and you think you're done. But that is what's. That's the kind of common approach to do that and it doesn't really work. We need to, as composers, we need to realize the power we have, even for a video game, and not only just make music and crossfade between them. We need to know that if you're really detailed, as you are in a movie, in a game, you can take this game to a completely new level. You can make a small indie game feel triple A experience only with really good music.

Speaker 1:

You know you said something like you're composing music for a story that hasn't been written yet, which which is true but that that is becoming an even more complex challenge with the advent of AI storytelling, which, you know, some of the NPC in the games are now going to be fully participating. I mean, how are you going to track all that?

Speaker 2:

You need at least to have ambition to try to do it. As you say, it gets harder and harder the more intelligent the game gets. It would be the same thing if an NPC gets sad the way a real person would get sad. So you could prepare for it. You could prepare for a lot of things. You can also use layering If it's complex. Feelings you can have a little bit of both feelings mixed together and things like that. You have to invent really good systems for it. The way I do it, I'm actually doing a kind of a game like that now. I'm not trying to do it more technical, I'm just trying to look back and see what tools we already have. So you could use music theory, like I'm doing, a foundation that could be both sad and happy, okay, and because of the layering on top, it will decide what it is Right. So I have these kind of broad brush strokes and then I can have really sharp tools with, like the stingers and the motifs and things like that. I'm trying to prepare differently in this game.

Speaker 1:

That sounds like an insanely complex project. That approach that will take you many, many hours to work that out yeah, it depends.

Speaker 2:

you could say like back in the day I was telling a game studio, I looked at their game and in this game there were like 12 different villages you can enter and the approach was to to score new music for every village and to me me it was more like it's better we do a really complex village music that has a lot of variations. It will never sound the same and then you put all the effort on that and that will. You can reuse it all over the game. So even if there's now we have four new villages, well, use the same music, because it's going to sound different every time.

Speaker 1:

Okay. So it's like you're putting, it's like you're doing a powerful sonic montage technique, you know, and creating colors that you can layer. It's like you have a palette of colors and how you mix them creates the effect. What is your composing process for video games, you know? Because it sounds to me like you kind of take that approach but then you've got to kind of deconstruct it in a certain way to be able to reconstruct it. Yeah, it's a little bit. I mean, when I did it Takes Two.

Speaker 2:

For instance, there were I don't know how, was it like 200 cut scenes or something like that, even more I think. And then it becomes that's a film score almost right, the cut scene. So I had a lot of these as well, and me and gustavo did the music there, treated the music like in the cut scenes. We can be really linear and really make the music as it should be, in lack of another word. And then, when it comes to the gameplay, it's it's more important that things are happening because you're doing something in the game.

Speaker 1:

But what I'm looking for is, like you know, where is the base inspiration, and how do you build from that? You know.

Speaker 2:

Yeah, I've thought a lot about that. My studio that I work in, I collect synthesizers and instruments and I have a percussion and I have everything In my computer. I have so many libraries and so many, and this is very common these days because libraries are very cheap nowadays, yeah, and good, by the way, and good, yeah, good and cheap, yeah, and so we get the new. I think everyone experiences this problem is that you have too many possibilities, so my inspiration comes from limiting myself. That's the first thing I do, and we have this limitation and that, for me, always gives birth to inspiration. Like there was one part in this tree that I really want a big brass, but I couldn't because it was the rule or else I couldn't do it. So I ended up dubbing, I think 16 bassoons or something like that to achieve the same thing, and I wouldn't do that if I didn't have the limitations.

Speaker 1:

It feels like the top is blown off the video game industry in terms of pushing the boundaries of sound, design and music and how those work together, and it just feels like the last five years have been a real revolution. What have you heard that you think is really pushing the boundaries of sound, design and music and video games?

Speaker 2:

Yeah, a couple of games Sadly I don't have. I'm not a gamer really I couldn't say that today because I have kids and everything. But I try to play the games my kids play and sometimes I get really happy. Tears of the kingdom uh, I think they're.

Speaker 2:

I think the music is fantastic, uh, because and it's typical nintendo, I think to find the balance because it's so there's so much air in that soundtrack, it's it's like and, but it works so well when it ramps up and becomes something more and so on. So I think that it's a really good adaptive score in in that game. And also, I just heard of a really funny game called untitled goose game. I'm not sure if you've seen that. It's just a goose and you're gonna do pranks on on people and I don't know the name of the composer. I should do. But there's this piano piece, classical music, but it's really connected to the game. So it feels like playing piano to a silent movie, that kind of feel to the game. It really feels you have a piano player next to you playing it. But I also like what's her? She's called sarah sachman.

Speaker 2:

I thought she did the music for call of duty 2, if I'm not mistaken, and um, I was a judge in uh, listen to a lot of game music and the first time I heard this music it was like okay, the same thing again, all over again. It's it's this kind of you know, bombastic war music for the way it sounds in any game. But then I stopped for a while and I really listened to the music and I realized what you've done with the music here. It's it's like fantastic. It's like like this genre. I didn't think you could do it. It's done. How do you do it again? But she managed to do it. I haven't played the game so much, I don't know the adaptive level of it. I was really amazed of how you can reinvent a genre that you've heard too many times.

Speaker 1:

Without joining the player. You know, is the whole artistry of adaptive scoring. You know, it's that balance between this cohesive musical narrative and allowing players to feel they're in control of their journey. I mean, to me it's really I can see why it really seduces you as a composer to kind of get in there and figure these things out. And you said something in passing which I think is one of the keys to the whole thing is you mentioned a score. That was simple. What I wanted to talk to you about next was you've been in this business long enough and you have watched the evolution of technology.

Speaker 2:

I started playing on a Commodore C64. That was my childhood, actually, when I got older and learned a little bit more of how they did, how they composed this music. It's so fascinating when you have, like in a C64, you have three voices, so one voice has to take care of like the bass and the snare or something like that, and the other has to be an arpeggiator to even be able to mimic a chord and so on, and they managed to do this fantastic music. Then we came into maybe I should say like PlayStation 1, and everything died because you had a CD for the first time in the console, so you could actually play real music, and that was a cool new thing. I can hear this is a real guitar, this is a real vocal, and that was was a cool new thing. You could I can hear this is a real guitar, this is a real vocal, and so on. But when I came in with my ideas of adaptive music, that was kind of dead back then, just eight, just 10 years ago, and no one really talked about it. It was still in this, you know, I I use london symphonic orchestra game. Wow, that was the thing. But then, as you say today, I think everyone is talking about adaptive music and the importance of technology and so on. And when you're talking about immersiveness, you can't only say that well, I'm going to use London Symphonic Orchestra. That's not enough. You need to have more. You need to be connected to the player, like at least come close to how connected a film score is.

Speaker 2:

When I went at the royal college, we were at the film music class and our teacher said I'm going to put on this, this uh scene here from the friends, french lieutenant, something, um, and you have jeremy irons, you have Meryl Streep in a dialogue, and then we're going to talk about the music. And we looked at this scene and he turned it off. And I was embarrassed because I couldn't remember if there was music in that scene. And then he said, okay, anyone, the music. And no one had anything to say about the music. And then I said, ok, now you know there's music, so let's listen again. And it was this like you have in the front, this oboe playing every thought that Meryl Streep's character had. She couldn't talk, so the oboe did it for her. It was so perfectly written so that you don't think of it. And that is the thing with scoring a film, you are actually enhancing the story and that's it. The music is really doing its job.

Speaker 1:

Yeah, I think Hans Zimmer said he looks on composing as the underlying. This is where you're supposed to start feeling yeah, yeah. And he said his breakthrough was when he realized he wasn't writing music, he was scoring feelings.

Speaker 2:

The knowledge we have 3,000 years of knowledge of how to make music and how to create or wake feelings, and that's a really powerful tool. So we need to be proud of that and talk to game studios and get them to know the importance of that and the power of it. That's the first challenge. The second challenge is okay, now we need to enhance the technology of this, because it's all doable, but without already made technology and we always have to build it it becomes too cost, it's not cost efficient. So we need to bring in some standards and there's a lot of people out there trying to do that, and that is a good thing. But I think the first hurdle is to make the game studios know what they are missing.

Speaker 2:

Because, if you think of it, what's been done now for many years? Which composer can do like really bombastic things and make this sound huge, this game? Well, today there's just one answer and that is AI. So we're not needed for that, so that you can do with AI easily. So why are we needed? Well, we're needed to do not only enhance the story in a video game, but give the music a story Right. It needs to be this is Chris McHale his music Right and I'm going to use him on the next project because all his ideas into this project were so great and you know we need to enhance ourselves and put us. It can't be about, you know, libraries and making things huge.

Speaker 1:

That is not a composer's job oh, so talk to me about the new. What is it? Elias four, or what do you call it?

Speaker 2:

yeah, uh, last yeah 4.5, I think 4.5 and that's coming out.

Speaker 1:

When's that coming out?

Speaker 2:

well, it's software, so I can't say yeah, right. But the last thing I heard from the programmers were a Q1 next year. Okay, so it's coming out soon. Yeah, so we are already using it in a couple of game projects, so it's already working. But you know you have to polish for the market and so on.

Speaker 1:

Can you take us through really quickly, I mean, a couple of features that you think are really going to address some of what we've been discussing here.

Speaker 2:

Yeah, I think if you think of Elias 1-2-3, we're like adaptive music in a certain style. That's it, and that was like a tool for people. So for me to make a studio want to use it, I first had to explain to them that they want to have adaptive music. That's hard, of course. So in this new version it's more a complete solution with audio and music, but with a NodeGraph system where you can do whatever you want.

Speaker 1:

Okay.

Speaker 2:

So I think the first versions of this new version so it will will be. You can do whatever you want, but it's going to be a lot of nodes, right, and then we're going to listen to to everyone and see what our own needs are like. Okay, people are using like a car engine, like this all the time. Let's do a combined node for that and so on, and it's also going to be a really open system so people can do their own systems and so on. That's a thing. And when it comes to the music part, it's more like, if you just open the new music part of this solution, you might think that we left out a couple of things from the last one, two and three, but we actually didn't. We just moved it to the node graph.

Speaker 2:

Before it was, we had this powerful tool that was called action presets. Using composer could, could, could, script a long chain of events that will happen and certainly like play these stems on the on this level, at the same time, play the stinger and when that stinger is done, play this and so on. You can do like that. So, as a composer, when you deliver the music, you know how it will sound in the game With our new system, you can do even more. You could do like well, I'm going to connect this certain aspect of the music to the health of the character of the music, to the health of the character, and I'm going to connect this certain aspect of the music to the area or whatever, or knowledge about the area, or so, so you can get even more granular.

Speaker 1:

What advice would you give to aspiring composers and sound designers anyone who wants to break into this video game industry?

Speaker 2:

Yeah, actually, I've changed my mind a little bit about this the recent month, because of AI, actually. So I learned that there are so many people out there and that's a good thing that are using the technology today and creating music, but that is not the same as composing music. They buy their chord packs and they buy good libraries and they, they put them together and it sounds great and then they get a feel that they are a composer, and I'm not saying that is a bad thing, but if you want to work as a composer, you can't do that because it doesn't matter how good you are, to just bring in the right technology. Ai will always be better and it will be better and better and better. So don't do that.

Speaker 2:

Go the other direction. That's my first thing to say. Go back and see if you can learn some music theory. See if you can learn some simple score, string arrangement or whatever. Learn things and combine that to your passion. I'm sure, let's say there's a I don't know, there's a ukulele player listening to this and the only thing you can do is play really good ukulele. But yeah, use that, that's your edge and use the rest of your time to learn the craft and learn the skills.

Speaker 1:

You want a music career, you want a professional music career, you need about 10 years of music study. You know, you got to pick your instrument your guitar, your piano and you got to sit down and you got to try to master it, because mastering that tool will open up doors for you. And it's just nothing. There's nothing, even close in the technology, to an artist actually putting their hands on something yeah, I totally agree.

Speaker 2:

And also talking about that, if I'm sure you have the same experience that I have, like, if I listen to something I did 20 years ago, it will still sound good today if I use real instrument. What you just said. It's so powerful to have your real instrument playing that because it will sound as good in 2000 years.

Speaker 1:

All right, Christopher, thank you so much. Thank you so much for coming on Creativity to Gigi and talking to us about video games and music and all that kind of stuff.

Speaker 1:

Thank you so much for having me and we'll talk again soon, okay, yeah, all right, man, take care. Christopher eng, talking to us from sweden, one of the more interesting thinkers I've met in the last 10 years while I was wandering through the world of video games Sort of never really quite connecting, but learning a lot as I did that At core. I think Chris is a composer. I met him when he was a CEO and he had lots of staff and fancy ideas, but you know he really just wanted to write music. I get it because I started Studio 2GG to get back to writing music too. So I understand where Chris is coming from. He's a musical composer, slash technologist, you know, and that fusion of art and innovation has led him to some really groundbreaking production tools and to push the boundaries of adaptive music and just to write some good music. But I think he's a guy who just really wants to pick up his guitar, pick up his bass, you know, get his hands on a keyboard and just compose. I mean, I know already that guy just shuts the door, says leave me alone and I am working. I always love talking to him and I hope you enjoyed this conversation. I hope you learned something.

Speaker 1:

This has been Creativity to Gigi. Visit us at studiotogigiio and subscribe to this podcast and maybe join on the website. It doesn't cost anything, but it keeps us in touch. Share, you know, help us out. Okay, we've got some good things coming up. Stay tuned for more inspiring conversations about the creative life, and I will see you next time on Creativity to Gigi.

People on this episode

Contributor

Chris Mchale

Host