E140 - AI-based software engineering - Anton Osika

Can AI redefine the future of software engineering? Join us for an insightful conversation with Anton Ossika as we explore the groundbreaking advancements of GPT Engineer, a tool poised to transform the landscape of software development. Discover how this revolutionary product is empowering non-programmers to create and modify applications effortlessly, and listen as Anton shares the philosophical roots and the impressive reception of GPT Engineer within the tech community. As we gear up for a major launch, Anton provides a glimpse into the exciting journey of Lovable and the innovative potential of AI-driven programming.

Our exploration doesn't stop there; we delve into the critical ingredients for building successful AI software startups. From the art of assembling a robust team to the importance of identifying clear customer problems, Anton candidly shares his insights on navigating the evolving startup ecosystem. The conversation highlights the shift from traditional coding to using plain English as a development interface and emphasizes the rising standards and expectations set by AI products. Hear how the democratization of software development paves the way for both seasoned developers and newcomers alike, and learn about the nuances of human-machine interaction that can enhance user experience.

Looking ahead, we tackle the broader implications of AI on business structures and career landscapes. Anton paints a vivid picture of the "one-person company" concept, where AI agents could replace traditional coworkers, reshaping organizational dynamics. We also touch on the ethical considerations of AI advancements and their alignment with human values. From challenges in AI model development to future business models and job trends, this episode offers a comprehensive exploration of the intersection between AI innovation and its impact on society. Don't miss this chance to engage with the complexities and possibilities of the rapidly evolving world of AI.

Follow us on youtube: https://www.youtube.com/@aiawpodcast

Chapters

• 0:01 Startup Founder Discusses GPT Engineer Growth
• 14:00 Building Lovable AI Software Company
• 22:27 Building Lovable AI Software Startups
• 29:53 AI Software Engineering and Technical Stack
• 41:28 Optimizing Large Language Model Workflow
• 53:28 Future of Open and Closed LLMs
• 1:02:51 Challenges and Future of AI Models
• 1:08:36 Machine Agency in Complex Tasks
• 1:20:43 Organizing Teams for AI Efficiency
• 1:26:59 Future Business Models for AI Software
• 1:35:45 Future Job Trends and Educational Recommendations
• 1:47:24 Future of Generalist AI Management
• 2:00:11 Evolutionary Drive and AI Ethics
• 2:05:04 AI Agency and Ethical Considerations

Transcript

Anders Arpteg: 0:01

But okay, so you have some kind of traction milestone happening. Now I hear in Lovable right, correct, can you share anything about it? What do you mean with you hit some kind of milestone, got a?

Anton Osika: 0:16

launch. Yeah, so we have a company called Lovable and we're building a product called GPT Engineer and the thing is now it's very clear to us from how users use our product and I get DMs every day from someone that found our tool and they've fallen in love with it so we are actually going to move our full branding over to the name of what our product actually is now. It's lovable according to our users.

Anders Arpteg: 0:49

Are you going to change or remove or not use the GPT engineer?

Anton Osika: 0:52

So I will unveil exactly what is happening, and also what is behind these breakthroughs, later this month, so stay tuned.

Henrik Göthberg: 1:03

So in November there is a launch, and yeah. So how does your days look like now, when you're a little bit like hitting this focus on the launch or the milestone and specifics happening right now?

Anton Osika: 1:15

so when you're getting a lot of user growth and everyone is reaching out to you, it's easy to get distracted. So what I'm reminding the team about is that the most important thing is to keep staying focused on making the product itself better and better and better. There's a long way to go until we are completely automated. We've completely automated ourselves. So that's the main focus. The other, the second focus, is the other. The second focus is to make it easier for users to know the all, the tip, the tricks to get value out of our products, to educate them. And the last part is more about okay, if we're doing a launch, then one, we're educating our people, and then two, we're explaining when you should use our products and getting them in into the door so has even also your attention switched more to go to market, marketing, communication, messaging, even one step notch higher the last couple of weeks because of this.

Anton Osika: 2:14

Yeah, so that's so. I'm emphasizing that the product is the top focus, but yes, there is the. The reason I say this is because it's easy to get more like okay, what do we do now to scale this up? There's the fire burning. We can.

Anders Arpteg: 2:26

There's so many ways to get 10x users for us right now yeah, I told a colleague of mine you know that I'm having you uh today on the podcast and he had to try gpt engineer. He hadn't tried it before and he came back today this this morning and said, Jesus Christ, I built this thing in minutes and it was awesome. He was so surprised and he's still an AI person and it's easy to understand how it becomes lovable for people that have no idea they can do this kind of things.

Henrik Göthberg: 2:59

And the background is fun. You said earlier like literally, you did something to prove a point that this could be useful. Lms can be useful for programming, and this is way before we saw the devons and everything like that. So what, how did that come come about? Yeah, I'm a little bit curious on that one yeah, so what was the point you wanted to prove and what was the context?

Anton Osika: 3:21

so I I co-founded a company called the depict ai oh, yeah, yeah that was the other um, and there we liked having like philosophical dinner discussions about okay, how are things going to impact x and y? And my point was that, very rapidly, these large language models are going to um be composed into systems that can replace most of the work that we do in software engineering and I guess you have this with Oliver and Etta.

Anton Osika: 3:53

This was with Oliver Edholm, exactly, and others, and I felt that no one out there was imaginative enough in this and then proving the point was me sitting down over a few weekends and putting together this open source tool that you, as a developer, explain what you want to build in your terminal and then it writes out all the files and you can just click, or it also runs the entire project yeah, so you, you can, in normal language, without being a programmer, explain the purpose, the objective, function, what, what you want to happen, and then it can sort that out systemically.

Anders Arpteg: 4:30

Yeah, this was a very simple way, but a lot of people were very very impressed by this, but I think we should go into more depth about this very shortly. But we're very proud to have you here, anton Ossika is that the right way to pronounce it? Yeah, awesome. We've known each other for a number of times and I mean you're very famous, I think, in the Stockholm AI community. You've been part of so many of the famous startups that we do have.

Henrik Göthberg: 4:58

I mean SANA. You've been working at SANA. We had SANA people here. Oliver was here a year ago, or something like that, when depict ai that you also co-founded. So yeah, yeah you, you're a profile for sure.

Anders Arpteg: 5:12

Cool yeah, but also that you know the gpt engineer. I think it's one of the thing that really you know hit the, the popular media in a way that very few other open source projects from Sweden has done, and I mean it has now over 50,000 stars or something on GitHub, right.

Anton Osika: 5:31

Yeah, it became the world's most popular one in code generation, among many others, so an open source project GPT engineer hitting 52,000 stars.

Anders Arpteg: 5:47

That's cool. You've been part of so many you know super famous Swedish startups and now, since last year, you know founding this new one and we're going to go more into depth in that shortly. You know what lovable and GPT engineer really is. So anyone, and certainly me as well is very much looking forward to hearing more of the tech and the history and the future about you know about AI, software engineering in some way, and I heard someone say that potentially you have like over a thousand products built on this now that they're not just used for prototyping. It's actually some products being used, right? Is that correct?

Anton Osika: 6:23

No, some of the products the project built. You can go into the website and see what others are building um, the ones that are happy to do that in public, but um, there's more than 1 000 per day now and and some are launching these commercial products that were built only by talking and not by writing any of the code themselves.

Henrik Göthberg: 6:45

But give us a couple of few numbers on lovable right now. That's a good entry point. You have some impressive numbers here, like 1,000 projects or products being built per day.

Anders Arpteg: 6:58

Yeah.

Henrik Göthberg: 7:00

Tell us some more fun, interesting numbers. Yeah, tell us some more fun, interesting numbers.

Anton Osika: 7:07

Yeah, so I mean the driver of this more explosive growth. By the way, the product is still in beta, so it's like not an officially launched product, but the growth has been much faster recently, thanks to that. The AI doesn't get lost, it doesn't get confused. It actually reliably takes you all the way to something that has business value for you, and last month we doubled the number of paying users for this and currently somewhere above double in month, double in a month.

Henrik Göthberg: 7:36

It's not used double.

Anders Arpteg: 7:37

Double in a month great nice okay, you're going to go deeper into this soon, but before that, can you just perhaps describe yourself a bit? You know who is really, anton Ossika. How would you describe yourself?

Anton Osika: 7:50

Yeah, so I'm a physicist in training and I'm a startup founder and CEO, but I think I'm not this normal smooth talking business, startup, foundery, startup what do you mean?

Henrik Göthberg: 8:06

like smooth talking as?

Anton Osika: 8:09

elon. I think I identify a bit with you know, especially after reading his crazy biography, um, but what people say about me is that I see around corners what will happen in the future. Not just one corner, but many steps of how does the future look like, and I have a lot of initiative taking and excitement for building cool technology and finally having a lot of positive impact in the world. Since I was a teenager, that's something I think about.

Henrik Göthberg: 8:39

You think about that? In what way? Since a teenager? How does that?

Anton Osika: 8:44

materialize. I I think my, my, my dad has always been like this environmentalist champion, so part of it was there. But I I just thinking about all the way that the world is messed up, made me very, very angry as a teenager and set out to do something good here. And then what's the best way to do that? I? This is a many, many, many steps of reasoning, so it's not that intuitive, but being an entrepreneur is arguably the most effective way to have an impact if you really want to change the world, become an entrepreneur yeah, I will say so with your focus of what you want to change.

Henrik Göthberg: 9:21

That you could do that.

Anton Osika: 9:23

But if you only work towards this grassroots mission, you will usually not get anywhere. Interesting, it's only it's you, it's usually when you solve an important and valuable business problem, like, let's say, bill gates set out and he like what did he work on? Well, selling a lot of software, um, but now he has. He's basically curing polio, which is this horrible disease that has affected and ruined millions of people's lives. Um, and by first doing that and then and then focusing on doing something good you thought about going going more academic.

Anders Arpteg: 9:57

I mean yes good question.

Anton Osika: 9:58

actually, I mean university I was set on like there's's so many fun research topics that I would love to go very, very deep into, but it was when I was at CERN, which is the place where they discovered Higgs boson, the big particle accelerator, that I see all of these super, super smart people that are working extremely hard at an extremely inelastic problem. So you're not making progress, even though you have so much smart people.

Anton Osika: 10:27

While I was out in the industry. There is so much more room and higher elasticity for doing something useful, something that makes a difference. So, then I figured no, academia is too slow and you have to be-.

Henrik Göthberg: 10:41

And when you were in CERN, in what capacity were you?

Anton Osika: 10:43

there, I was there for a summer, for for three months, yeah and um, I worked on this super symmetry group in in atlas, one of the accelerators so in in a sense getting a feel for.

Henrik Göthberg: 10:55

Is research for me or am I more of a engineer product?

Anton Osika: 11:00

yeah, I got the feel for I feel for profile is better used elsewhere and I think for most people that is the case. You don't make so much impact there.

Anders Arpteg: 11:12

Okay, so I guess we can come to this question later. But if you were to give some advice for people that are younger than you and thinking should you go the academic or more scientific route, or should you go the academic or more scientific route or should you do the engineering routes? Do you think everyone should do the engineering route or do you think what's your thinking there? I?

Anton Osika: 11:34

think there are many different objectives in choosing routes, so I will have to first like, say something about the objective and how I think about my objective. It's like, um, unlock the most human potential. And that starts with myself like, how can I live up to my full potential? And one of the timeless things you should do to unlock your full potential is to surround yourself with more experienced, super smart people, and you can do that in academia, like a few years ago. I think it's easy to do that in the industry as well, if you work in tech at least, I'm not sure if that's the case anymore. It's like tech is not hiring so much. So I, um, I would say, first of all, find very, very smart people, um, in your kind of age group or whatever that is, and maybe do some startup together.

Anders Arpteg: 12:36

at least find them, and then, yeah, see what resonates with you with regards to academia or working because you can grow in different ways, I mean, and there are people with different personalities and you know what they like, of course. Um, but also doing a startup and becoming an entrepreneur is not for everyone, I would say. I mean, you have successfully done it a number of times, of course, so it seems to fit your yeah, it resonates well with you.

Anders Arpteg: 12:58

Um, have you ever, you know, thought about the question, about, you know, having, you know, the co-founder or founder role, and then also having to think about how I'm going to raise the next founding round, et cetera, versus being more of an engineer, perhaps joining a company that already has proven ground in some way, it seems you have chosen the entrepreneurial kind of path.

Anton Osika: 13:31

Yes, you seem to have chosen the entrepreneurial kind of path, mainly. Yes, so there is I, I've thought about that for sure. But I think what where I thrive a lot is to see opportunities and say we have to do this other thing and we can do that in a larger company as well.

Henrik Göthberg: 13:42

But I've ended up here that circumstances and my pro like my way of thinking and acting as one you used the word elasticity, you know in terms of you know I, I would argue that being in in a in a founder position and being a startup position, it's maybe one of the most extreme elasticity situations you can be. And if that's, if you're gravitating to, oh, let's not be rigid, be rigid in focusing on the old product, thinking what it could be your pivot and everything like that. So has that something to do with it? That if you are drawn and if that is a larger mission that you have as a person, then I would argue that fits quite well with where you're going and the way you have chosen a career path you're going and the way you have chosen a career path.

Anton Osika: 14:30

Yeah, well said, like unlocking your, like this, most challenging thing you can do it might be to be a founder. It depends on. Some people have an easy time, but there are so many new challenges and you can always do more, more, you can always do more you can always do more. You can always learn the other yeah, but I think there is a parallel universe where I would have gone into more research.

Anders Arpteg: 14:45

My friends say like we should work in ai, like actually applied ai, atropic or open airs perhaps we should move to just speaking a bit more about the origins of gpt engineer and then lovable. After that. We spoke about the beginning a bit, but if you were to just, you know, phrase it a bit more. You said you had the discussion at the Pict with Oliver and others and you want to prove a point in some way. How did you actually get started and why did you think that AI could work as an AI software engineer?

Anton Osika: 15:19

So there's two steps here. I got started with the project to prove the point and it turned out that this was actually a very useful tool that Milius used Then to start a company was a much more serious decision, of course, and I mean with many conversations with my co-founder about the future of the PICT, but co-founder about the future of the picks. But then the what. What I saw as the big um change that we're going to go through is that, well, we currently have developers using ai. There's this auto complete co-pilot feature for the most developers use. But I think, think this is just a short, temporary period in how software development works. In the future, you are going to talk to an AI system and not look at the code.

Anton Osika: 16:15

Before we looked at series and ones, then we looked at a bit more like object-oriented programming and now and then even higher abstraction. Soon it's plain English. That's the new, hottest programming language. I think Andrei Karpathy quoted that. And what does that look like exactly? Well, we don't know exactly how the interface is. It's some kind of plain English where the human role is still very important, because the thing that we all humans, I think will do in the future is not to produce business value by thinking, but produce business value by expressing what we like, and there needs to be some kind of interface where AI is the biggest part of carrying out the work but humans explain oh, I want it more like this, so this is what I like, yeah.

Anders Arpteg: 17:10

I think one point that you could have wanted them to prove is simply that AI can do something from an AI point of view. But you could also take in the point in I as an engineer, software engineer think it takes too much time. I want my work to be more efficient, and that could be another way to use AI. Or, thirdly, it could be simply that I know a lot of people that can program and I want to enable them and democratize in some way the ability to build software. It's the last one there, yeah.

Anton Osika: 17:36

Okay, it's the last one that you use. So the interface is plain English. Anyone can build products and when something is frustrating you using your bank interface or something is frustrating you booking flight tickets, then you can go in and say like, change this. And then maybe someone has to approve it. But that's the future we're heading towards, and building that interface is a very fun, very kind of multi-dimensional challenge of understanding what are the possibilities. How can you express this? Is it just the chat gpt interface? No, it's not.

Anders Arpteg: 18:09

It's much more than that and I really would encourage anyone that don't believe it is possible to simply go to the gpt engineer and try it out. You can get started in very you know a few minutes and you have your first app in a few more minutes more, more or less and it is literally that easy. You would easily do it as well, henrik, and it's surprisingly efficient, right.

Anton Osika: 18:31

Yes, we made it in such a way. Most companies have this like oh, do you need to sign up, who are you, and so on. You can just go in and write what you want, and then it's free for the first few.

Anders Arpteg: 18:43

The credits run out too quickly. I think you want, and then it's free, to get the free for the first few credits run out too quickly, I think it actually cost us a lot of money with all the usage, for the free usage right now cost a lot of money, but having a free tier, I think is an awesome thing still, so you can, but it's also a way to you know what.

Henrik Göthberg: 18:57

Don't take my word for it, try it, you know that's. That's a cool value proposition in itself. I think that's the way to do it. I had another. Okay, let's that whole thing we are talking about. Now. There is another argument here. I want to try that on you. I mean, like so I had a conversation with someone to say you know what, in order to do good prompting, in order to have a good human machine interaction, the way we are working with an llm today that the most people have experienced in chat, gpt and everything actually understanding the fundamentals of programming or how systems work, improves your ability to prompt. So I'm thinking a little bit like okay, the vision is here is like you don't need to know any programming. Yes, not in code, but mentally understanding how systems works and how you, how you talk to a machine, so to speak. I think that is a skill in itself, that that actually we. Of course we need to be competent in programming, but we will work in on a different abstraction later. What do you think about that?

Anton Osika: 20:00

you might not want to get me started on this because I'll just keep talking, uh. But the vision is very simple. There is, like this perfect interface where everything you say just magically happens in a few seconds. But the path there does require humans in the loop a lot. And how we see it right now is anyone can build the concepts.

Anton Osika: 20:27

My designer friends who reach out to me they're like I measured something I did in figma and then I get exactly the same here 20 times faster, literally 20 times faster, and that person can take it to an interactive website, a product, but they cannot take it all the way to having it connected to, like open ai databases, authentication, at least that that easily as someone who is technical, who understands the concepts and and can prompt it in the right way. So there is still a big interplay here and um there. I think where you get the most benefit currently is by being in a tool for teams to work together, show each other what they want, to build Tools for teams because maybe there is within the team, there is some programming competence, but you don't need 10 of them, you need one.

Henrik Göthberg: 21:20

Yeah, one example, right?

Anton Osika: 21:22

Exactly so. Everyone can move as fast as the programmer pretty much and then you have someone that can jump in and edit the code themselves, even if they need to.

Anders Arpteg: 21:31

Awesome. So you built this open source project, gpt Engineer, and at some point you chose to start also the company called Lovable, and I'd love to hear more about the name Lovable as well. But can you just walk us through a bit? You know, why did we make the decision to create a company called Lovable?

Anton Osika: 21:49

So there is this meme almost in building companies and you said it right before this, henrik you can build minimum viable products. People talked a lot about that, but the thing that actually is necessary these days to show product market fit or something is a minimum lovable product, and that's what our ai system is set out to do, and it was actually part. You know, I didn't know that I know, I know the terms.

Henrik Göthberg: 22:16

I was just throwing it out there, but it was actually part of the thinking when you choose the name lovable and so I'm looking for examples of.

Anton Osika: 22:27

Slack, for example, is a tool that people use and it's actually pleasure to use. It's like, well built, it has all these nice keyboard shortcuts and, um, people who have it they're like and they start a new company, they're like. We don't have slack. I really want it because it's a really nice product and most software is not like that. I think now more and more software is becoming like that, and the reason is that it has been very costly to hire, not just like the one percent of people that can code, but the top one percent of designers, together with excellent front and system engineers. And now AI is going to be able to do this by us as engineers in the team, who have built a lot of great products, encoding our best practices into the AI system.

Henrik Göthberg: 23:15

This is the new bar, this is the new norm Lovable. I think that's where you need to be at to have market fit. It doesn't matter if you, if you're, if you have a smart thing that works, and if it's a shitty experience, uh, it's not gonna take off. I don't think so.

Anders Arpteg: 23:33

yeah, these days these days and I congrats on also receiving a big funding round now. I think you were number one in Sweden in October or something for getting was it 7.5 million USD or something? Yeah, great, Congrats on that. And if you were to, just a lot of companies and people want to have the journey that you're having right now. Can you give some tips? How do you actually build the interest? How do you capture the, the investors? How do you get these kind of funding rounds?

Anton Osika: 24:07

yeah, so on the investor side, um and there are many, I think, different paths you can go with regards to, I mean, having a strong team, um, so doing a marketing stunt?

Anders Arpteg: 24:22

is a team more important than a product? You, you think? Or what's the most important?

Anton Osika: 24:25

Yeah, I think definitely like if I'm an investor, the team is 100% the most important thing.

Anders Arpteg: 24:30

Because you're still pre-seeded. It's not a seeding round, it's still a. It's very early.

Anton Osika: 24:35

When we raised this we had a prototype pretty much. But the thing that's most reliable here, I would say, is to sit down and really, really obsess about solving, initially, a very, very small but annoying problem, like a problem that people actually your customers actually care about, and that often takes you both time to understand what that problem is. And if you can find the solution to it and make the solution lovable or at least work on it so many times so that it's not just buggy and annoying for the users, and if you can do that, then if you obsess about that and work on that every day for a few months or a few years, then you will have something good and then investors will be very impressed.

Henrik Göthberg: 25:30

But what you're saying is simple but fairly profound Having the right profile team of people working on a problem that is super simple and clear. Someone who I admire, who is a professor in marketing and branding, said Henrik, you have a great idea here. It could potentially grow but it can never scale because it's not sharp enough. It's not clear enough. So that sort of sharpness in product problem or whatever you want to say, I think those two things you mentioned now, I think that's kind of a secret sauce in here the team with a very, very sharp problem that the investor can get. Would you agree?

Anton Osika: 26:10

um, yeah, I think that's the best way to build a business. The investors sometimes care a lot about the scale here, like I. I think that I think that is less important if you if you do you first start with a very small problem that you can solve really well, you can make money on it and you, um can build a team around that that works really productively on that. On that, then that you can always. Or if you see, if there is atlantic clear path, you can expand the scope okay, but then it's a trifecta.

Henrik Göthberg: 26:39

Right, it's the team, it's the very articulate problem, sharply defined, and it's the vast scaling opportunity around it. If it's done well, I guess that's the trifecta.

Anders Arpteg: 26:54

But it seems this, when I look at a bit of your communication on LinkedIn et cetera, I mean you do speak a lot of the team that you want to build or have the best people around and already have in some way, I guess, and that you want to build or have the best people around and already have in some way, I guess, and at least for me also in the investor in early stages kind of phase, that certainly is the most important one, I think as well. Then you have to have a clear idea, but you know the products will probably change a lot during the path and you must be open to that, right.

Anton Osika: 27:27

Yeah, I think the most important thing is who the people are, and the second most important thing can be the most important thing is how you work together and if you can have this super fast propagation of ideas and selecting the best ideas and learning together from the external world, and I guess in your case it's also a bit easier for you because you have a proven track record as well, and then, of course, it's always easier, I guess, as well.

Anders Arpteg: 27:54

So finding someone that has actually have some kind of experience, I guess, is a good idea as well, if someone is interested in having a similar path.

Anton Osika: 28:03

Yeah, it's a bit of a cheat code to have the track I can just share.

Anders Arpteg: 28:08

You know, when I this is like 15, 20 years ago I worked with the price runner founders. They just sold price runner and they're going to launch this other new, second generation shopping site. They got money without showing any line of code at all, and they got a lot of money. They just said this is our idea, give us money. And they got money. And then they sat down Okay, what should we do now?

Henrik Göthberg: 28:35

But that must be also telling of the times. I think that kind of experience can shift over the cycles. Yeah, sure, I mean, in that case it was extreme I think, but they had can shift over the cycles it must do.

Anders Arpteg: 28:45

Yeah, sure, I mean, in that case it was extreme, I think, but they had a really good track record.

Henrik Göthberg: 28:49

But they had the track record, of course, but I wanted to flip it. What were you looking for when you were looking for funding in terms of what would be the ideal VC for you, or how were you thinking about the matchmaking with the right partners Business angels, vc? You know different ways of thinking here, so how were you thinking about what capital would be the best fit?

Anton Osika: 29:16

So on this topic not on the topic of fundraising I think there is a book that is 70 pages long by a prominent, very no bullshit, very to the point founder, startup founder themselves, and he is Ryan Breslow. He talks a lot about pick a VC where you see, where you feel a good connection to the people that you will be working with, together, collaborating with, and where you want to have that relationship.

Henrik Göthberg: 29:51

And there are two funds that I knew from before where I had that connection. Um, so was it more of a ba style, business angel style, with fewer, or was it like a portfolio? Yeah, there is a lot of.

Anton Osika: 29:57

we have a cap table with quite a few helpful, a lot of of helpful angels from cool institutions and so on, but there's also yeah, there's also VC funds, and that was my biggest criteria. And then the second thing I thought a lot about was okay. So if we build a trillion dollar company or like we have, then there's a lot of responsibility for the company on how you act in the world to have a positive impact. And then I interviewed the VCs about how would you look if we come to a hard decision profit.

Anders Arpteg: 30:35

So you turn the discussion around a bit, because otherwise you have another kind of situation where the VC interviews you but you actually ask them instead of I did that. Yeah, that's nice, but maybe that's a nice position?

Henrik Göthberg: 30:46

yeah, but because sometimes we forget that you know what it takes two to tangle and sometimes you know, like I mean, it's not for everyone to be able to pick and choose their money, you know. But even if you can't pick and choose your money, I think it's good advice you should interview and find a fit and understand how will this partnership work if this happens. You know, I think that's a brilliant advice could we go a bit technical?

Anders Arpteg: 31:14

yes, now, let's go, let's go, let's go. So we I think everyone knows chat tpt claude and so many other, you know gemini that can code rather well, or very well, I would say and you can ask them for help, but there are things they can't do. Can you just perhaps start by explaining a bit what is different with GPT Engineer and traditional, like ChatTPT Claude or Gemini and these kind of services?

Anton Osika: 31:43

Yeah, sure, so if you're building a software product, there are a list of like best practices that you can come with and you can be quite opinionated and say, okay, this is how we're going to do things. We're not going to do all this things we could be doing, and the strength of our system is that it does a lot of that. You can always take it over and do whatever you want with it. Unlike, this is more like older school, old school, no code tools, which is a strength, but it's opinionated, which makes it much easier for us to continuously fine tune our system to work extremely well in those circumstances.

Anders Arpteg: 32:26

So you add like a bias to it in some way in the beginning, so you guide the user in. This is the proper way to set up the system.

Henrik Göthberg: 32:35

So we're talking about design pattern bias, or what do you mean with bias? In this sense, we're not getting into what you're talking about.

Anton Osika: 32:43

These large language models. They're very powerful. They are sometimes you language models. They're very powerful. They are sometimes you feel like they're very stupid, um, or they're at least not reliably smart. They can think sometimes, wow, but they are. They often run into doing stupid things, um, and for each of the normal hard problems, we can look at them, diagnose them and see how do we make the AI system know how to do this problem and that's something we're doing which the wireless chat GPT cloud. There's an ai editor called cursor. It can not assume those, like all of these specifics in how things should be done, because it has to work with any existing code. That could be c++, any, any low level level things, and that makes the problem very different but is that a little bit like what we were talking about?

Henrik Göthberg: 33:45

I was referring to the topic that you will come further with prompting chat, gpt if you're a good programmer, and here we have the same topic, that okay, in order to make this work for someone who's not a programmer. We are opinionated programmers who really well understand how to program a large language model. So therefore, when you do this, we will steer that into these patterns.

Anton Osika: 34:11

Correct. Yeah, so a really good programmer can prompt it better and we have. A part of what we do really well is that we have looked, we continuously look every time something goes wrong. How could we have helped it and kind of be a middle layer there in what the user was? Very said, something very stupid.

Henrik Göthberg: 34:34

So a user communicates in a certain way and a programmer would communicate to the same element in a different way.

Anton Osika: 34:40

This is part of the magic source. We remove that um kind of the gap, the gap Exactly.

Anders Arpteg: 34:47

Can I just give you some example of what are the biases, or opinionated kind of uh thinking that you add to the system.

Henrik Göthberg: 34:55

Um, so I'm trying to not go too technical, Uh don't be your secret sauce, okay, so oh, please do reveal your secret sauce.

Anton Osika: 35:06

So what has been very time-consuming is to create a very beautiful user interface that is interactive and when you click a button it opens a box and then it adds some records in a way that is bug-free, and that problem we have reliably solved, which is like a huge unlock. This front-end engineering is a hard problem. Now this is solved for new applications. So for new applications, we have solved that.

Anders Arpteg: 35:36

And you have like React-based or BIOS, for example.

Anton Osika: 35:39

Exactly. So how do you do that? Well, there's a lot of different choices of front-end framework. There are new front-end frameworks every day and the LLMs are very good with React.

Anton Osika: 35:50

So, when our users know React, so that we've chosen like always build React, always use TypeScript, always try to use something called a component library of beautiful looking widgets and buttons and so on, and specifically compose them in this different, in these different ways, and then, when it so this we have reliably solved what is the next unlock is to having this user interface communicate with a system that persists your data and keeps track of which users exist, if you can log in and so on.

Anton Osika: 36:25

That is that is more difficult, and there there are the same type of uh guidelines and assumptions we can make. So if you need to create this, so we tell the ai, if you need to create a system for logging in, then we can do it in, and then they always do it in this way. If you're calling it external api endpoints, if you're calling open ai like if the user is in the built application and wants to call open ai then you should have to do it in a certain way to make it a great user experience so it works really well with front ends.

Anders Arpteg: 37:01

How well does it work for backends, you would say today?

Anton Osika: 37:04

yeah.

Anton Osika: 37:04

So, um, the scope is for, like what we have, to cover 80 of the applications, and if you're find out that you are not in this 80, sometimes down the line, you can always edit the code yourself in whatever term, whatever editor you prefer.

Anton Osika: 37:23

But, and that 80, as we see it, it's um authentication, logging in and managing users. It's storing data, any type of data in a database for the specific user, or like if you want to build twitter, you don't have to store the posts and so on. And then it's, um, uh, like back-end endpoints, that that let you call something external if you are locked in, and so on. So, if you want to build an AI app, then you always need to add some backend functionality for the AI parts, all the types of like, quite simple SAS products, the core functionality of SAS products, like HR management tools, inventory management, if you're like running a business, customer portals, visualization dashboards, those you can reliably build. Once you add, like you have 10, 20 different features, you get complex, then it will be a frustrating experience because then the system is still a bit, but you can build some backend with some APIs and some databases, all of that.

Henrik Göthberg: 38:41

For someone, and I think this is a bigger gap than we realize. If I'm going to oh, I want to do a startup like yours be a GPT engineer and I'm used to maybe old school I understand the modern data stack that we have in our large enterprises, but I have a really hard time imagining what is your technology stack, what is your software and protocols, or what are the fundamental building blocks that you build GPT engineers a tool on. Are we going to do Kubernetes over here? Are we going to do this over here? What is the fundamental technology stack, or how are you thinking about that to build the product GPT engineer, that is your programmer's daily work.

Anton Osika: 39:35

There's a fun technical topic here, but it's ideal if our product builds itself right. There's a way to edit itself and, with regards to the front end, we can do that. I can open GPT Engineer and I can edit itself, which is pretty fun.

Henrik Göthberg: 39:53

So you can use your own product and build your own front end.

Anton Osika: 39:55

Yes, but the back end systems are actually for us complex.

Henrik Göthberg: 40:00

We spin up a separate server micro VM for each user and it does not edit any of that, and so so this whole orchestration, everything like that, you just pick apart a little bit of the core technology components in order to build a system like a compound AI system like this. We're not going into details, but just understanding the technology.

Anton Osika: 40:25

So you're saying how we build our product. Yeah, so now it's complex, but when we started out, it is just using the same technology that I've described React, adding a simple backend, which are like all I can do, and that as a point here to explain. Like, like okay, could you build our tool with it, with itself? Is it the same components? There was actually a user last week that built a visually perfect copy of our product and it's not functionality it had all the core or they had the core flow perfectly functioning.

Anton Osika: 41:03

You ask for that's a meta very meta. So, yeah, um, and that product, uh was. I would say it went beautiful because the ui was similar, but it was definitely not lovable, which again, like, takes a lot of iterations and smartness in terms of the ai to make it really, really good. It was called likable. That's what he called it.

Anders Arpteg: 41:28

I'm going to try to drill further and see how much you can actually speak about the secret sauce you're doing.

Anton Osika: 41:34

But I guess, you know.

Anders Arpteg: 41:35

in your case it quickly builds up with the number of files you have in the repo, so to speak, that you're building up and at some point you have some limitation in context, windows etc. Or you need to in some way prompt and talk to the LLM in different ways. Can you just elaborate how do you handle multiple files in different ways and make it understand that now it needs to add a new file or now it needs to change something in a specific file? How do you actually build up the kind of prompting and discussion with the LLM?

Anton Osika: 42:04

Yeah. So I'll go on a quick detail. How we did this before was that we had this very complex agent system and people talk a lot about this. It was also we were like, okay, so Devon demo launched and we're like, okay, maybe they're onto something here. This looks really cool. And we also built something that was extremely cool where it had all of these different steps, agents talking to each other and so on.

Anton Osika: 42:27

But it had a lower accuracy. It usually failed more often, and much worse than failing more often is that when it failed, the user didn't understand at all what was going on. Like no idea, impossible to understand what was going on, and it was much slower. So you wait a long, you wait for minutes and then something fails. You're like I'm out of here. This is not a good user experience.

Anton Osika: 42:55

So our focus is like the opposites now to make it as fast and as simple for the user to understand what's going on as possible. But then it becomes much easier to learn the limitations and understand. Okay, these things are really fast, these things work really well. When I get to this stage, like I have 20 different features in my product, then I loop in, I onboard my engineering team into this first version that I built out in minutes and how does it work now then? Okay, so the focus is on speed. So we have like extremely fast large language models. Can you say which one you're using? So for the fast ones, we usually use OpenAI's smaller model, which is very good.

Anders Arpteg: 43:44

4 mini yeah, which is very good 4 Mini yeah exactly that does a few.

Anton Osika: 43:50

We call it hydrating, a main larger prompt, and when we're done with that, we send that over to Anthropix's medium-sized model, which is very, very good 3.5 now I guess.

Henrik Göthberg: 44:03

This is the interesting stuff here. This is a compounding system system and this language model over here, it's another one over here. I mean, like this whole orchestra, you know, I think this is what people don't get. Why do you mix the two?

Anders Arpteg: 44:15

can't you just simply use the sonnet all the way? Was it just because of pricing, the speed, but the, the? What is the? The fast one, high key or?

Anton Osika: 44:22

something. Yeah, so haiku is not as fast and not as cheap. Also as again, that there's a large cost to this.

Anders Arpteg: 44:28

Now, when we have a lot of usage, even with the release, the new version. Just yeah, that's the one we benchmarked right.

Anton Osika: 44:33

So when it came out still cheaper with mini, yeah, so, um, yeah, exactly so if we would now I'm going technical like if we would use haiku um, which we are running currently an A-B test with like split testing and seeing what's the performance in actually, first we do it offline, but then we also check, well, what does it actually look like with real users, then we would not switch out the small OpenAI models to Heiko, but we would switch out the big Sonnet model to Heiko. Yeah, because speed is so important and for the simple requests, haiku will do almost as good a job.

Anders Arpteg: 45:09

I saw something about the latest Haiku version, which is the smallest of the three that Cloud AI has, was beating the Opus, which is a super big one, that they had in version 3. Those 3.5 Haiku, I think was better than 3.0 Haiku opus or something. It's insane how fast you know. Yeah, the evolution of model scale I mean the I.

Anton Osika: 45:31

I would always take this with a grain of salt, though, like the, the big models, or have more like some kind of general intelligence and reasoning about things that are, um, like. It's not in stretch heading data that that's usually what people say wireless. The small models can get extremely good at these things.

Anders Arpteg: 45:50

That has been optimized for still, if we just go back to what we spoke about, you know you have a large number of files, the, the total, like number of lines, grow quickly can. How do you, how do you prompt the models?

Anton Osika: 46:02

how do you brought this is a important thing to get right um. I I said before I can go into technical details here and I will try to do that. The um. The best way to prompt the models is to start extremely simple and see where it fails and then try not to make the prompts more complex. But often times you have to and then that's a big part of why our system works well as well. Like it's gone through that iteration of like, okay, let's make it more complex. Does it solve this edge cases that we had before? Yes, did it introduce some problems with the old types of tasks? No, actually it's fine, let's roll this out.

Anders Arpteg: 46:46

But do you mean you try it out yourselves manually and see what works? Or do you mean it actually, when people are using it, it starts in some way and then automatically fall back to different solutions?

Anton Osika: 46:57

Yeah. So if we we have, usually we have a problem like, okay, we need to become better at this, let's change the prompt. Let's try it once manually. Okay, looks good. Then the instant we start this back test of seeing how would that this have? I have a big back testing library as well and that this loop of is how we do prompting. So so to say, how to prompt them then?

Henrik Göthberg: 47:20

better and better um work and coding is prompting in this sense.

Anders Arpteg: 47:27

Yeah, talking to the LLMs and see how they actually react and make it work, and I'm going to try a third time here as well. So how do you do the prompting?

Anton Osika: 47:36

What's that in the prompt? So the prompts say you are an expert software engineer I don't think we need that anymore, but that was actually useful in the past and just explain the full context. Because if you are, how do you say this from an LLM's perspective? They have never seen this product before. So you need to explain a bit of context. Like the user is going to ask you a question to change a code base and you should reply in these different ways and here are the few. Depending on what the user says, you should change the code, answer a question and or take an action.

Anders Arpteg: 48:17

But do you say look at these files specifically for that prompt, or how do you select what?

Anton Osika: 48:22

so selecting the files is done by llms as well, as one of the early steps ah, so it's a multi-step process.

Anders Arpteg: 48:30

Still to first know, okay, what files to look at, or if you should create new files, I guess.

Anton Osika: 48:36

Or um, no. So we need to provide, like I said, hydrating the prompt. What does that mean? Well, it means pulling in all the info, all the most important information. You could just put everything before we put all the code, since it's quite small projects could just put everything before we put all the code, since it's quite small projects for us, we usually before we put all the code in there and all the information that's relevant, it turns out that it's slower, more costly, but mainly it deteriorates performance. It becomes more stupid when it has to look at many things. So that's why we do an intelligent selection on what is necessary as input and then as for which action to take. That's up to the like.

Anders Arpteg: 49:21

The main prompt that is also optimized for being fast, that does all of that I'm starting to understand a bit more, at least it's good that you speak a bit more about the internal.

Anton Osika: 49:32

But I think this is Sorry. I can just say this. I probably quite generalize For others building large language model applications. I think this is a pretty good framework to make it super fast and have one large LLM call initially and then have some kind of super fast LLMs on top of that, maybe before, so we also do some super fast ones afterwards.

Henrik Göthberg: 49:57

And my point what I was trying to get to and I think we're getting there by listening and talking to you is that even if you're a very experienced software engineer and you've been working building systems for years in React, whatever you need to wrap your head around. How do you code or how do you build a system? Now you're going to build a platform and you're going to do some normal stuff where the core is an LLM and you're actually working through the LLM, so it's a different workflow. I'm trying to unveil what's the difference in workflow. If you want to build a startup that is fundamentally with an LLM at the center, this is a different software workflow. I think that you're describing it makes sense, but for someone who has never tried it, it's a different workflow. Am I right? Or how did you sort of you know? Did you evolve in terms of how you understand your programming and all that, or is it the same?

Anton Osika: 50:55

So yes and no, I mean it's how to use the LLM3 full potential is a completely new thing and there is no academic discipline or like profession around this, but I think it's just one new piece of software engineering. Like software engineering, discipline has dozens of different things you have to learn, like all the front and back and testing databases, like all of these things, and this is a new one. This is a new one and there is some people talk about like, no, now everything should be an agent and I, as I explained, I don't believe that's how you should think about things you should think about. You have a user that has a problem. You have to solve the problem for the user. Write as much code, as little llm as possible. If that doesn't cut it, well, you have to add some more ai things on there and add as little AI as possible, make it work and, if like, then maybe add more and more chains, which makes it start resembling an agent, but try to avoid it. That's how I see it actually.

Anders Arpteg: 52:00

I love to at some point later go into this kind of different levels of AGI that OpenAI spoke about, etc. And see what the future will hold, but I think we should not go there now because then we get stuck there and never get back to dbt engineer. So no comment, you're right. Perhaps you can just mention. You mentioned a couple of challenges that you had when building dbt engineer and you tried out a more agent-oriented framework and then you went back to a bit more simple approach. What have the major challenges been or surprises perhaps when starting off from the first open source version to actually having something now that you will have a big launch for in a couple of weeks?

Anton Osika: 52:45

Yeah, so the big challenge is to reliably let users get to the point where they get business value before they run into this confused AI system. And we have many people comparing different AI software engineers, aspiring AI software engineers and I just want to point out that in those comparisons I have only seen our tool come out on top. Now, interesting, yes, and I've challenged people. I'm saying like if it's someone I respect, I'm going to pay you $1,000 if you show that our tool is not the best. Oh, that's a good statement. I like it.

Anton Osika: 53:28

And this is a very exciting times because of this, obviously, but that's the big challenge that we're running into, like making it reliably, let users achieve what they want and what.

Henrik Göthberg: 53:37

What is that? What is that boiled down to technically? Do you need to work on?

Anton Osika: 53:42

um, well, there, as I said, there's all the difficult, uh normal software engineering things you have to solve, for which usually take a lot of time. We have to do it super fast and we spent a lot of time there. But I mean, it's mainly this iterative process. Where does it get confused? How can we avoid that without reducing performance somewhere else? And, yeah, the way to the way to solve it is to have a great team that can move really, really fast and solve all the all of this, the problems, and be come up with really smart ideas on how to solve them and I guess, be agile in that way and actually change the way and don't be afraid to change the plan.

Anton Osika: 54:28

Yes, like well, that's one of our company values. Like have a pirate approach where you're prepared to throw away everything. You, all these great ideas, find a better. I love it.

Henrik Göthberg: 54:39

Have you, I need a t-shirt. Yes, we've been collecting t-shirts quotes for four years. You know, be a pirate or have a pirate approach. I love that t-shirt.

Anders Arpteg: 54:49

You know John Bosch is a professor in Chalmers and he speaks about going from agile to radical. I think you know it would be awesome to say go from agile to pirate approach yeah, pirate approach.

Henrik Göthberg: 55:01

Forget about radical. I want to be pirate, it's much cooler. But what is a pirate approach then? Because this is the thing. Are you using that? You know, jokingly, of course, but what does that mean?

Anton Osika: 55:12

Yeah, for example, just like, okay, we try this complex systems system which was really good. It pushed the limits of what you can do with multi-agents and so on. And then we're like, no, this is not the right one. This is faster, faster build, much faster for the user.

Henrik Göthberg: 55:29

That's one example. So pirate approach is about daring to. You know. Question yourself Dump the shit. You know there is no sunk cost. Dump the shit that doesn't work and think like a pirate. How would a pirate solve it?

Anton Osika: 55:44

I mean, yeah, I have a good friend who's leading work at this one of these LLM labs and he says, like when we built AlphaFold, for example, there was all of these smart physicists and scientists coming in and be like, okay, I'm joining now, I can guess who you're speaking about.

Anton Osika: 55:59

But okay, good, we can apply first principles, thinking and use more equations and physics and math here, and they all got burnt on that. The thing that, like the pragmatic people always been out on, is to say, hey, we have the system, why is it not working? Okay, we seem to have it said. There's this mistake, there's those mistakes. How do we fix those mistakes? And just solve them and then solve the next bottleneck, find the next bottleneck, keep doing that.

Anders Arpteg: 56:37

You have a physicist background as well, so it must feel a bit strange. Yeah, it's true.

Anton Osika: 56:45

First, physics is a lot about first principle thinking, breaking down like what are the governing principles here? And I think finding the bottlenecks and fixing the bottleneck is the right principle here.

Henrik Göthberg: 56:55

So it is a type of first principle as well, but it's a joke here because Elon Musk is the guy who is the biggest pirate talking about first principles, you know?

Anton Osika: 57:06

so living both ends and figuring out first principle from a pirate point of view, I guess now I'm going a bit meta here, but I guess there are people who think a lot about like this, governing equations, like how the world works, often get stuck in those. But the reality is that every model is like useful in some domain and the right way to do it is to not use your models too much but find oh, is there a new model to be applied here and solve that the problem, which elon musk definitely does now. Now he comes into winning the us elections and he has all of this first principle thinking that he never knew before, but he just found out of how to win an election.

Anders Arpteg: 57:45

I think that's giving you away a million dollars every day could be one Great example.

Anton Osika: 57:51

He goes around and tells people the only thing that matters is that you actually go vote. He doesn't say anything else. That's probably quite smart as well.

Henrik Göthberg: 57:59

But back to our story. I was building up a curiosity in my head when we were talking about the different models you tried or actually you have a multi-model. I mean like claude, yeah, and have you tested or have you experimented with any open source approaches or have you had that open source thinking in your company? Okay, we need to see when we scale beyond this point. We can't afford to do it. We need to go also there are many.

Anton Osika: 58:29

Open source has a lot of meanings. Like we, I did the gpt engineer original version, yeah, which is now like quite old. That's an open source project. Right, yeah, it's not our product. Um, exactly, I'm thinking should we do open source for our product itself and I, so our code base would be open source, and part of me is a bit sad that we haven't done that yet. At least we might do that.

Henrik Göthberg: 58:54

But that was not the question. The question was actually which models? The question was what is supposedly in the category of open source? Llms like Lama versus, can we call them open weight?

Anton Osika: 59:07

Yeah, thank you. Thank you very much. They're not open source. The source for them is actually hidden and you don't even know what the source for them is.

Henrik Göthberg: 59:14

Thank you, we had this conversation that they are not open. None is open Open weight. Let's call them open weight, so we have tried those models.

Anton Osika: 59:22

They actually have one big advantage, which is that some hardware providers run them on very efficient or specialized hardware. So we tried using them, but what we find is that it's very important with this out-of-the-distribution common sense, almost like IQ points for the models which OpenAI and Anthropic are world leaders in, while the open source models are good at specific set of coding problems, but not as general. I'm quite confident we will optimize speed by using open source models in that some cases, depending on what the user asks for so in the future.

Henrik Göthberg: 1:00:15

So when you're actually starting to build, and this is scaling, you decompose this and you could quite possibly have open source in this particular problem or feature, whilst this is, you know.

Anders Arpteg: 1:00:26

So it's a mix, it's going to be a mix, yeah okay, in short, you're saying that open weight LLMs are inferior compared to the closed store? Yeah, unfortunately unfortunately.

Anton Osika: 1:00:37

Yes, I, I think. Um, I mean, the bottlenecks are different things. Now, the bottleneck is, to a large extent, how intelligent the large language models are. That will be less of a bottleneck in the future, and then the open source models will quite easily catch up. Open-weight, open-weight.

Henrik Göthberg: 1:00:57

And have you sort of done the calculation or break off points? You know, cost-wise, you know that, oh, this will never scale in a proprietary model. You know, with a token price like this, we need to go open source. You know that.

Anton Osika: 1:01:11

A factor in this, in this game, um, no, if you compare to, uh, a human's time, I think you want currently to max out on the intelligence. Okay, as I like, yeah, weighing in, okay, weighing in. So it's speed and intelligence is what you want to max out on. In the future, you will want to start to optimize for costs more. But as an example, I've tried this Devin, which is closed access. Now the reason is that it is not reliable. You don't reliably get what you're asking for. So they they haven't released that, um, and but they do go to the enterprises and maybe they find some cases where it's it's worth the time of the user, even though it's not reliable, it's worth the time. And then they charge ten thousand dollars per month, right, really, which is a lot of money, um, and that says something about, like, worrying about cost is a bit.

Anders Arpteg: 1:02:12

I would really want to go into the business model that you have. But before we close the topic of open weighted LMs and closed weight frontier models, what do you think will happen in the future? We've had this discussion a number of times and it's just interesting to think. Either the gap between the two will increase and you will have the few extreme large frontier and extreme intelligent models, or you can think also that open weight models start to catch up. What do you think will happen in coming years?

Anton Osika: 1:02:43

No, I think we're kind of now seeing diminishing return on the IQ points in a way. Openai launched their reasoning engine O1, and it hasn't had a big impact at all.

Anders Arpteg: 1:02:58

You don't think so.

Anton Osika: 1:03:01

It has not. I think O2, o3 will be a big improvement. So the reason it doesn't have a big impact you can look at the benchmark, the delta and it. It's exceptional at things where you have to do reasoning like so physics and math and things like that. How often do you need that? Not that often actually so, and it's much slower. I don't know how expensive it is.

Anton Osika: 1:03:26

It's's less kind of predictable in how you use it and this is, I think, the first signals that you that new, more IQ points is. I mean, this type of IQ points are not that valuable and you can just If you do what we are doing, you move really fast and you create this system that uh solves the user problem like okay, I want to, I want to create the product like uh, and you iterate on it. Then you can get very far with today's like level of intelligence and at some point we're going to care more about that. It's being open source that we can do more optimizations, which we can. I'm very sad that we cannot do that. So the open weight is what I meant there. We cannot do the optimizations you can do when you run the model yourself and the IQ points might be less intelligent models.

Anders Arpteg: 1:04:27

This is a rabbit hole, Sorry.

Anton Osika: 1:04:28

I have to go here, go ahead intelligent models, then this is.

Anders Arpteg: 1:04:30

This is a rabbit hole. Sorry I have to go here. Go ahead. But first, before I go into the rabbit hole, I'd just like to ask are you using, like, fine tuning apis for opening or something, or are you using them as is?

Anton Osika: 1:04:37

like I'm teasing a bit on the technical breakthrough here is that we we find a way to teach the models without fine tuning. Okay, um, we have done some fine tuning, but now it's not a part of the core flow.

Anders Arpteg: 1:04:54

Let me go back to 01 and the future of the frontier models a bit and think a bit further down the line, a couple of years ahead. I would like to disagree with you a bit here. That's why I'm trying to open up a conflict here.

Anders Arpteg: 1:05:08

It's a disagreement even though who knows what will happen. But if we assume, if we just think like let's just go transformers, just let's use the GPT decoder and see how far we get with more parameters and better data for training it and perhaps some-tuning RLHF or something to continue to train it, but not really change the architecture, perhaps not really change what O1 potentially is doing with the STAR approach, with the self-taught listener and some kind of reinforcement learning, that is, adding some kind of different type of training at least and different type of reasoning or inference that they have. You don't think that actually adding this kind of extra logic if you call it that, in how you train it and use inferencing could potentially make it much more efficient?

Anton Osika: 1:06:05

Yes, that is going to be a big thing as well. The models will be fast and cheap if it's a simple query, but the question is all of the new things that are coming out now, aren't they going to make them much more powerful soon? Yes, and also thing is that there are thousands of extremely smart people that are trying to push the state of the art, and still 0801 comes out and it doesn't have.

Anders Arpteg: 1:06:41

It has a number of benchmarks but it completely overshadows everything else. I mean it was many benchmarks but it completely overshadowed other models. So for more reasoning-related tasks it's still outperforming in extreme Right.

Anton Osika: 1:06:58

I guess where we might disagree somehow is that I think the current level of the models is like the biggest impact, which I can only think in terms of like, okay, what are we going to see in the world, so say, comes from operationalizing the models we have, and then the level up in the models becoming more, becoming better is also large. The models are going to be much better, but it is in maybe the same order of magnitude as just using the models to their full potential.

Anders Arpteg: 1:07:43

But let me challenge you in this way, because you actually add manually a number of steps here. You have a large language model first, or larger than the second one that you used to try to get some kind of idea how to approach the problem. So basically, I would say you do some planning in the beginning and then you have a second step where you do the actions or changes in the code in some way. Please correct me if I'm wrong. Yeah, of course.

Anders Arpteg: 1:08:06

And imagine if then we have models like O1, which potentially sometimes, when you see it's a simple problem, I guess you know give the answer directly. Some other times it needs to take a number of steps and actually reason with itself um to come up with what the best solution would be. Um, do you know the JEPA from? You know kind of idea of your? I don't think I've read much. Do?

Anton Osika: 1:08:29

you, you want to explain. I think it would be great to uh, yeah, it takes a long time.

Anders Arpteg: 1:08:34

Sorry for this rabbit hole, henrik, but he loved that.

Henrik Göthberg: 1:08:36

You made that comment, by the way, so please go uh, jeppa is called the joint embedding predictive architecture.

Anders Arpteg: 1:08:42

It's something you know in jan lecun. You know he doesn't like the autoregressive kind of objective that we have. Just. Predicting the next token is perhaps not the most efficient way to train things, even though it's surprisingly efficient in just scaling up the number of parameters, and I used to think he was a bit crazy on like not believing in the scaling laws, but he's growing on me.

Anton Osika: 1:09:00

In this regard, you need something new, yeah he's going up and down, I can tell he has a big feud with elon as well, which is interesting.

Anders Arpteg: 1:09:08

Anyway, one way of thinking there is that you would like to have, as human, a thinking fast and thinking slow kind of approach for some tasks, and I would say that most LLMs today are basically doing thinking fast and you would at some point say that this is too hard of a problem. I need to do the frontal cortex kind of conscious kind of reasoning, where you in your symbolic space in your head is thinking a number of steps ahead and it's really slow. I mean it can take seconds and even more, or even minutes or hours for some tasks that's necessary, even minutes or hours for some tasks that are necessary. And then either you can do what you do now, which is basically manually adding this step, saying first do this planning, then do that, and it's hard coded, or you can have the model trained to do it itself. Wouldn't that be a big step forward?

Anton Osika: 1:10:17

Yes, it will be a big step forward. There's a question of how fast we get there. We will get there, we will get there, and it will be much less code. It will be much more just the LLM Right.

Anders Arpteg: 1:10:25

So this is you know the philosophical, of course, and then Jetpack also had this kind of hierarchical Jetpack thing. So hierarchical means that for some tasks it's super fast and it's super easy you just need to add this token. Next step could be you know, add this sentence. A step above could be you know, think about how to drive a car or whatnot. So, having this kind of hierarchical thinking I think you added it manually now so you have an upper, higher abstraction layer where you do some planning. Then you have a lower action layer where you take a number of steps. So if that's being done and trained in an automatic way and you don't have to manually add how you go about doing the more abstract planning, that potentially could be useful.

Anton Osika: 1:11:13

We will have higher reliability and less mistakes from the AI. I'm scratching my head here, because when you dig into a problem, you also often find these dimensions you did not think about, and I'll name a few in our domain, for example, that are still, like, maybe more important than how good the llm answers, and they have to do with, basically, how do you interface with the human here? What, what is it? How do you make sure you understood the human correctly like? This is a, the more important problem, almost, and and how does the ai system, um, so to say, confirm that it did the right thing for our system?

Anton Osika: 1:12:06

The AI quite often spits out code that is wrong Less often than when I write code. It's usually much more and what I do when I write code is that I run the code and I check if it works, and then I look at the debug and I step through the code, and those are the things that can have a bigger impact, which is a bit crazy. I I would say that I I didn't say this before, wouldn't say this a few months ago, but bigger impact than having better large language models and more intelligence in the large language months, but it will be a big change as well I think what you're doing is exactly right.

Anders Arpteg: 1:12:42

What we have, you know what type of models today. So you need to basically have the higher abstraction lay manually encoded today and probably for some time forward, but who knows have the higher abstraction lay manually encoded today and probably for some time forward.

Anton Osika: 1:12:51

But actually how long time is it going to take? Do you have a prediction?

Anders Arpteg: 1:12:55

No or yes?

Henrik Göthberg: 1:12:55

But there is a large topic in Rabbit Hole here. Maybe we can fall into it now. And this is go deeper in understanding the word agent, agentic agency, because you, you, you quite clearly debunked that you know what to have, uh, agentic workflows in in over complex ways in LMS is not necessarily always the best thing you know, so, so and and and and, I've been going on a path uh, actually, before agentic became a really big thing to understand agent based modeling of organizations, rent the price.

Henrik Göthberg: 1:13:36

So we are talking about more fundamental stuff agency and I think one of the key things the problem you are solving or you're working on which is your number one problem to solve to be be lovable is that you're ending up with a principal-agent relation between the system and the worker.

Anton Osika: 1:13:56

The principal agent.

Henrik Göthberg: 1:13:59

So there's a lot of theory Before someone stole agentic and put that into the AI research community. There's been 50 years of research on what is called the principal agent problem or relationship. So basically, how do we organize companies, how do we organize our relationships within a boss and their co-workers? And the core thing we are talking about now is that we get to a human machine interface where you need to treat the machine as a co-worker almost more than a system. So what I'm saying is that when you're starting to interact with it in natural language and want to get stuff done in a team and one of the teams is a software GPT engineer, it's a co-worker experience you are almost needing to solve because you need to communicate to the machine and that needs to understand.

Anders Arpteg: 1:14:56

So that whole interface you're working on Is that the point with an agent system, and it's not really a human machine interface. It's really the machine to machine interface, isn't it?

Henrik Göthberg: 1:15:05

No, but the core topic now is that if you go deep down, forget about AI and forget about, you know, agentic workflows in LLMs. But if you go down into the sort of papers and work that has been done for 50 years on what we understand of agent-based modeling, what we understand with the word agency, you know. So we are handing over agency from the human to the machine, to not only you know to do things for us in more and more advanced approaches.

Anton Osika: 1:15:37

Can I just interject here? Like we have had this software revolution for a few decades, where we tap a keyboard and then things happen, I would say with your definition that it sounds like that is handing over agency to the software software I, I wouldn't agree.

Henrik Göthberg: 1:15:56

So why not? Because, okay, but we are going from one narrow task to to more complex problems I don't think that that will be a change of definitions.

Anton Osika: 1:16:05

Everything will be a narrow task in the future.

Henrik Göthberg: 1:16:07

No.

Anton Osika: 1:16:09

Yes, but I think Is there anything that's not a narrow task?

Henrik Göthberg: 1:16:11

as an example, the interaction level between the machine and the human is not on the narrow task anymore. So when you're prompting someone to build a system for you, you're prompting or you're.

Anders Arpteg: 1:16:32

I mean it's a spectrum and we can't really say it's agentic or not. I mean I gave it course in agent-oriented software engineering like 20 years ago, I mean so.

Anders Arpteg: 1:16:40

But you can see a difference between object oriented systems and agent oriented systems, for example, where the level of agency, or rather the level of how autonomous they are, and you don't organize the system in terms of objects that you have in a more abstract sense, but rather and the attributes they have. It's rather about minimizing the interaction between them so they can operate without having to interact. So you can think of a different way when you use that kind of agent-oriented design potentially, which is about really, in my view, becoming increasingly autonomous.

Anton Osika: 1:17:19

Can I restart with my thesis and then you can go first?

Henrik Göthberg: 1:17:24

Because the main point, what I'm trying to get to, is that, for me, clearly, when you are prompting someone to build an app for you and there a tons of different things in terms of coding being done on on a lower task level abstraction level for me you are working on one abstraction level and you're giving agency to the system to magically get all that done. So that is a way different level of abstraction and detailed task.

Anton Osika: 1:17:51

I learned something new here asking this question, because this is a very useful perspective of where what is. Where is the interface?

Anders Arpteg: 1:17:58

yes, where is the interface exactly?

Anton Osika: 1:18:00

um, uh, yeah, now I can have a more productive conversation. I think another perspective then is like okay, an agent, an interface that says that you are an agent here and I'm an agent here is an organization or a person. They have a need or a problem that they want to solve and they have to communicate what a good solution looks like to the problem, and then they just want that solved. And if it's software code, if it's LLM involved in the software code, it doesn't matter If it's a team of humans.

Anton Osika: 1:18:42

So the thing is, the worst way to solve those problems everything else equal in the solution, quality and speed is to have a team of humans. Because a team of humans is so complex, while as a piece of a machine you know that's where it runs. Like a machine is much more preferred by a user or an organization.

Henrik Göthberg: 1:19:03

And now, okay, so baselining this, you're getting to a much, much deeper conversation now about what creates emergent intelligent behavior versus not. And if we look at an organization as an organism, so what will happen is the way we organized organizations. We have created teams, functional division of labor, where you have task machines doing certain things doing certain things, but they don't have the right mastery for the task they're doing and they don't have the right feedback loops in relation to understanding what they're doing in a small, sub-optimized silo in an organization is the right or wrong thing. So you get what is referred to as functional stupidity, and there are papers on this as well. So what we are now seeing, when we are seeing how do we create better, performant systems? I don't care how you built them, but you've given them agency to be a coder.

Henrik Göthberg: 1:19:54

The mechanisms that we're now thinking about to make this work. They are, to a large degree, absent in the way we organize organizations. So the scary thing now is that how will we understand agency, artificial, human organizing? You know, now we take one role, a software engineer, let's say. I want to do that with every single role in Vattenfall. I want to challenge every single role in Vattenfall, I want to have a.

Anders Arpteg: 1:20:19

GPT software engineer. I want to have a GPT marketing.

Henrik Göthberg: 1:20:23

There is a big topic here, you know, around agency.

Anders Arpteg: 1:20:27

Can we just keep it a bit more simple? I think you actually spoke about it.

Henrik Göthberg: 1:20:34

I can talk a lot. It a bit more simple.

Anders Arpteg: 1:20:35

I think you actually spoke about I can talk a lot, a long time here I can feel that, yeah, I want to push it out, but, yes, go ahead get a bit too far off now.

Anders Arpteg: 1:20:37

We should, we should narrow this down and take it off to work. I mean, I think you phrased it in a good way to start. I mean, we, we know that we can have a more agentic approach. There's been papers on saying that if we instead have multiple lms talking each other, potentially they are more productive than having a single LLM doing the full task by itself, and perhaps you can even have smaller models and you have a multiple set of steps that they are taking by talking to each other and that, potentially, is more efficient. That's been some claims on that. I think it's super interesting that you say that for us, speed is much more important and we can find a more simple approach which is actually better in your use case, and I think that's really profound and super interesting. But I can also think there will be points potentially where, okay, I'm not sure how far we should get here. We can take the.

Henrik Göthberg: 1:21:30

I guess the time is going so fast that we should go the open AI, agi kind of level, kind of discussion, because if you take what you're building and you're putting that, scaling that up in an enterprise, it has profound implications for how we organize teams and how we design agency around teams. In order to put an engineer in that works, you know, so can I, can I have one? Last point there okay, so um, there are two perspectives to look at this.

Anton Osika: 1:21:59

There's like, okay, there's this human talking to an agent, blah blah, useful framework, I'm sure. Useful, I think, if we want to be productive about this implementation and so on. I have a different kind of framework perfect glass, piece of glasses to look at this on, and it's like the, the ideal organization company, but that is is one human and a huge AI system that the AI that the human can tell give input to. That is the perfect, and then everything else, everything else you introduce, is going to be suboptimal compared to that if that works. So it's all just about one human or kind of an organization or one mission for the company, um, translated into actions through some kind of system, and that could be. It could be many humans involved, most likely but you want to simplify it.

Henrik Göthberg: 1:22:58

No, no, I love it. I love it. So let me, let me embrace that framework. And then we, are we allowed to allow to? No, okay, yeah, one second. So imagine now we stand here in 2024 and I'm going to build this organization that ultimately, is going to end up here. Okay, with your vision, how would you organize them here and now you steer this organization of teams and organization that eventually leads that? Well, we can only do 1% of that vision today. Next year, we can do 5% of that vision Next year. You know what is the pattern of organizing that allows us to actually Conway's law, steer ourselves to that vision. This is what I'm trying to get to. I think it's the same right, because I'm trying to understand how this could work now and how we need to organize people now.

Anton Osika: 1:23:57

Yeah, so there's many ways to look at this. Like Conway's law is a super useful theoretical framework. Sometimes, I think the best single framework is to find the bottleneck of what you're trying to do right now and see what's the best solution for that. Which bottleneck of what you're trying to do right now and see what's the best solution for that which might be add a new person? Add a new person is going to be the solution many times. Let's call it the pilot law. Yeah, so that's my approach of solving that.

Anders Arpteg: 1:24:22

But I think we can get back to your question a bit more. I mean, I think an important question can be simply what should be the skills that organizations and companies should employ in the future, and what should you, as a person, choose to educate yourself in, when we have AI systems like the lovable system that can do a lot of the tasks but still need some kind of skill from a human in someone? So just think about that, and we'll get back to that question soon, Because I think this actually goes to what you just said the one-person company, the Sam Altman kind of vision. Did you hear Sam Altman saying? You know they're having bets on which year we will have the first one person unicorn company?

Anton Osika: 1:24:59

We are creating the tool to enable that?

Anders Arpteg: 1:25:01

Yes.

Henrik Göthberg: 1:25:01

I think so.

Anton Osika: 1:25:03

You are. You are part of that game. I talk about that.

Anders Arpteg: 1:25:06

Perhaps we'll have a zero person unicorn as well at some point.

Henrik Göthberg: 1:25:09

Yeah, we were joking about that. What have a zero person unicorn as well at some point. Yeah, yeah, we were joking about that what is the zero person?

Anton Osika: 1:25:12

but you heard about this uh kind of ai agent that was creating crypto means. No, it made 150 000 so far.

Henrik Göthberg: 1:25:19

Yeah, yeah, so that's zero. That's it. So an ai meant it made memes that I guess was an nft style, correct? Yes, you know, and it had its own bank account or crypto it's all.

Anders Arpteg: 1:25:29

It already exists okay I'd just like to close the topic here, and this was I guess was an NFT style Correct, yes, and it had its own bank account. It already exists. Okay, I'd just like to close the topic here, and this was a fun rabbit hole, by the way, going in so many ways, if we just take the question about, you know, agentic systems and I hate the term agentic, by the way, but still it's been popularized in the last three years it's been popularized.

Anders Arpteg: 1:25:53

I've been using it. But we can still think about system or AI LLMs, we call it that or AI systems that have some kind of encapsulation, and then you can choose to have a big one that do the whole reasoning we call it that or prediction in that, or it have some kind of internal communication between the different components and one specialized in. I know how to do planning, I know how to write a TypeScript, I know how to do a backend service, and it could be this kind of mixture of experts kind of thing where it's actually integrated into one big model, where some part of the model is having different expertise, or you can actually think of it as different agents that potentially is. You know, here is my database expert. Okay, I have a question about databases. Well then I go ask this so-called agent and it could be part of lovable potential in the future, or what do you think? Can it take that direction?

Anton Osika: 1:26:46

I mean, we are doing this type of things still. Like I said, we had it more complex before. We're doing things already today. So, yes, definitely. Then there is some question that I think you were also asking. It's like is this one big neural network or does it have to be this like multi-step? Long term it will be more like one big neural network, is it that? Does it have to be this like multi-step? Yeah, long term it will be more like one big neural network, as you said, like the approach and so on. Um, but that it's also harder to um, sort of say, make it do what you want if it's just, uh, like big matrices of zeros or floating point numbers that you have to like understand somehow. So, um, in the interim at least, it will be a lot of a mix between the kind of more interpretable, human interpretable steps, traceability, and you know some kind of traceability, I guess so you can debug and see like why is it failing?

Anton Osika: 1:27:47

okay, it's failing because of this. Like we can, can we do something smart here?

Henrik Göthberg: 1:27:50

Yeah, but it kind of.

Henrik Göthberg: 1:27:52

It kind of maybe gives a trajectory of how we reach the vision of the one person company.

Henrik Göthberg: 1:27:57

If, if we take our friend Jesper Fredriksson right so he's been working a lot, we, you know in Volvo, right and what is his approach to sort of go further in autonomy here, like, one approach is to, okay, let's try to sort out the workflows of a data analyst or a BI reporter and working with basically working with a you know a product manager or a leader, and then, okay, can I now create a you know an agent for you that acts as your co-worker data analyst and ultimately, this is the you know.

Henrik Göthberg: 1:28:38

So now we're building a principal agent relationship between a product manager and someone who needs a data analyst. They then feed all the work, but they're essentially doing the work a data analyst would do in order to do the pre-oracle reporting or doing the ad hoc analysis. So this is sort of the agent conversation, not in systems, but more about thinking about how we are replacing coworkers with systems and ultimately, if you do that at scale, you get to the point where all the different tasks, all the different functions in a system or in a company, yeah, it's the one company so we had this like back and forth for a long time maybe.

Anton Osika: 1:29:17

Or um, we have different perspectives and just to illustrate an example of how you could have my perspective yeah, okay, nice, you have a product manager. They're doing these tasks, each task specifically. Um, you could try to write software with normal code that does that task Reporting super easy. Just write software that does that for you. Some of the tasks you can definitely not write code. You need LLMs, right? And I would use that perspective and say, like, let's take all the tasks off your plate. And then what does your day still look like? Well, no, everything's automated.

Henrik Göthberg: 1:29:50

Okay, but then we're on the same page. And then, what does your day still look like? Well, no, everything is automated. Okay, but then we're on the same page. Because I was never, ever interested in the technology techniques to create the co-worker.

Anton Osika: 1:30:00

I didn't care, but you called it an agent and that's why I'm like yeah but who stole the word and what does the word really mean?

Anders Arpteg: 1:30:07

Back in the 1940s, a computer was actually a human, so I guess an agent, that's an agent. Anyway, I think we.

Henrik Göthberg: 1:30:13

I want to reclaim the word agent. That's what I'm trying to do?

Anders Arpteg: 1:30:16

What is the business model of Lovable?

Anton Osika: 1:30:19

So we help people take their idea to a product in minutes and they pay us every month so that they can keep iterating. Let them create new products every month so that they can keep iterating.

Anders Arpteg: 1:30:30

Let's create new products. Subscription model basically that you have is that the way it will continue or do you see some other changes coming in the future? So you have a free tier, by the way, and you have free tiered and paid tier. Do you have any other tiers, like levels of? Yeah, so?

Anton Osika: 1:30:47

enterprise services, the users that are the most excited about our product. They cost us a lot of money. They cost us more money than they pay every month, so we are going to adjust the pricing because of that reason. If you're looking far into the future, we'll be talking about this, but wait before you continue.

Anders Arpteg: 1:31:07

So the pay tier, so to speak, is it um a fixed fee? Yeah, so now it's fixed exactly, so it's not used user space right now.

Anton Osika: 1:31:15

So the ones that use it, uh like hundreds of times per day or something that that's right. That seems dangerous. I would say, um, but. But the prices will, of course, also go down. So for the compute of all of this, but if you look farther into the future we talked about one person unicorns we'll say at least we were seeing more one person tech startups that just build the product themselves, and they're often actually quite young because they figured out how to do this with AI.

Anton Osika: 1:31:48

And for the old school developers that care a lot about the code and so on, there is something. It's like github, which is um, a community. It started, at least mostly as a community, um, but for this new generation of builders that care about the results, they just want ai to solve it for them. I don't see this community existing and this like platform to get inspired by, get inspired by other builders.

Anton Osika: 1:32:14

And something else that's very interesting is that there is not the type of kind of revolution that we've seen in creation of the product, but of video content. Video content there is, uh, so after the iphone, let you take good videos. There's been like most of the content that people consume is not hollywood produced anymore. No, it's, it's youtube, tiktok and like wherever we put this on right, exactly um, and that type of distribution. Someone has to create that community platform, distribution platform, and this is something that we're already seeing signs of our tool becoming, because there are, you can see the products in out, in the open, so you're seeing so that's something we're um slowly building towards a trajectory for where this product is evolving to Exactly yes.

Anders Arpteg: 1:33:11

So, going from large-scale movies to small-scale videos that everyone can produce. Now also going from large-scale systems software systems going to small-scale systems that anyone can build.

Anton Osika: 1:33:24

Yeah.

Anders Arpteg: 1:33:25

I think you said sometimes that you're aiming for. Someone said that you should build one product per day, or something.

Anton Osika: 1:33:33

Yes, so there's a movement out. So now we launched a program for this, even though it's called like building lovable software every day and there is, I think, 10 people in the for now it's been increasing rapidly of people that build an entire app, an entire product, uh every day for a month. So 30 days of lovable uh movement, and there's going to be a winner in this movement as well. But it started without, without us having the idea what?

Henrik Göthberg: 1:34:07

what is some of your favorite application that has been built? Uh, some of your. You know this is the coolest showcases, but before we go there, can we just finish the business model topic there?

Anders Arpteg: 1:34:17

So I think the business model is interesting. I mean, you have a free tier, you have a fixed price. You're going to change it somehow in the future. Do you think potentially I heard someone say they're thinking about potentially like an equity model, saying that if they start a company based on lovable, then you take a stake in the company do you think that could be a thing in the future?

Anton Osika: 1:34:42

um, no, I don't know where did you hear this about us? Maybe, maybe not. It's a very interesting idea. It's very interesting idea. I think it sounds a bit complex to to manage to solve it. Um, but yeah, I'm gonna.

Anton Osika: 1:34:59

I'm going to marinate on this, if, if there's a room for it, especially if it would open up for a lot of people that you know um, I mean, I said um that we there, someone, needs to build this kind of creator's platform because there's going to be a youtubification of building software. Yeah, but there is also the case that we might just let our ai put it on autopilot and build new versions of all the sas software out there with ai right, and, and then you need operators to run the company for each one of them and maybe then there's a. What you're saying makes sense, but I have my default answer is no, we're not going to do that.

Anders Arpteg: 1:35:45

One ask as well. I think you know so many people, especially in Europe today, are trying to figure out how to make something GDPR compliant and upcoming AI compliant. If you were to do know, trying to figure out how to make something gdpr compliant and upcoming ai compliant, if you were to do just a button in gpt engineer says, make it gdpr compliant and have all the documentation, checklists and what not produced, you would make so much money I'm just telling you yeah, make it, make it a act, engineer whatever you want to come and it would be the best business model, I would say, ever.

Anton Osika: 1:36:16

So now we're competing globally against Devin and so on. As I mentioned, it's clear to us we have a better product. If we have to just zoom in on Europe only, this makes a lot of sense.

Anders Arpteg: 1:36:30

There are similar rules in the US as well and China as well. So I think it's not that big a difference, even though it's harder in Europe for sure. But Europe is a big market as well.

Henrik Göthberg: 1:36:41

Anyway, what was the question I was going to ask? Oh yeah, I forgot it as well, I was asking a question, but before we stopped it there, what was it? I forgot it.

Anton Osika: 1:36:50

The best examples of apps. Yes, thank you. I forgot the best examples of apps.

Henrik Göthberg: 1:36:54

Yes, thank you. I think that's a good, that's a valid question.

Anton Osika: 1:37:01

You know, I have two top of minds, like there's. There's one that I was just like very well built, where you have a big um screen of pixels and then you can go in and select one pixel and change the color and, um, anyone can do that, there's a chat and, like, you can comment on what you're trying to draw slowly, very slowly, and you can replay the entire thing that comes out. So I was like, whoa, you built a pretty like fun application, fun application. And then, yeah, I have I could. I was thinking of another fun one that analyzes your twitter. Uh, are you on twitter?

Anton Osika: 1:37:31

it's actually yeah it's, it's really on point on analyzing you and roasting you in different ways, like you got 78 cringe score because you posted this tweet, but that may be 20 plus prompting again.

Anton Osika: 1:37:48

Um, but in saying something more uh serious, someone found one of these problems that people have and it's that you have a lot of photos, it's hard to find them and organize them, and they built an application where you upload your photos directory and it runs them through AI and I think it runs something quite simple to propose names on all the photos and and put them in folders, I think, and then that they launched that.

Henrik Göthberg: 1:38:21

The automatic organizer so built on you. You have commercial products built with your product.

Anton Osika: 1:38:27

Yeah, Is that a trend that?

Henrik Göthberg: 1:38:29

commercial products are built on your product, or is it more? Yeah, exactly that commercial products are built on your product, or?

Anton Osika: 1:38:32

is it more? Yeah, exactly. And then that's like a stronger validation. Now that people are sending us screenshots of their clients.

Anders Arpteg: 1:38:42

After they've shown their first version, built with our tool, to their clients, they're like, oh, take my money, I want you to finish it and pay thousands of dollars, tens of thousands of dollars for it nice, awesome, and I'm trying to cross off all the questions we had planned here, because we're not going to go through them in any way, but perhaps one that I'd like to hear moving a bit perhaps from lovable to more like philosophical kind of questions, societal kind of questions.

Anders Arpteg: 1:39:13

Societal kind of questions. And if we imagine lovable, you know being even more successful than it already is, and companies can be built much quicker and you can make them more efficiently, what kind of skills do you? If you start with a question like this, you can either think from a company point of view what you should hire, or you can think from a personal point of view what should I educate myself in? If we start from an educational point of view and let's say you had a kid and that in 15 years had to, or no, in five years sorry, had to choose what to educate self in, what would you suggest?

Anton Osika: 1:39:53

so the I'll start with the quick, uh like deep dive on what does the world look like in the future? I think the future is my friend calls it the fully automated luxury communism, something like that right, slowly, say that slowly. Fully automated luxury communism, luxury communism. So, um, that sounds like most people are having to have a good life and do something they love, like spend time with other people, uh, have a hobby that they are passionate about. I think that's what the future looks like.

Anton Osika: 1:40:30

Um, so, to answer what you have to educate, what you should educate yourself in, I mean, my short answer is just like make sure you have fun, because that's what it's going to be about. But there are still going to be jobs in the future, right, and if you want to have a job which might be good because then you will have more status in society it's like timeless thing everyone actually cares more about than they think Then maybe you want the job where people pay you for various things. And most of the jobs in the future, I think they're going to have to do with the role of humans in this society, which is to like express and aggregate what humans like and so like being in the government is like that like okay, what's how, what, what regulations and so on should we have? And that's also a very, very important thing to get right, especially in times of extremely fast technological progression you don't think AI can know better than humans what humans like?

Anders Arpteg: 1:41:42

yeah?

Anton Osika: 1:41:44

I think they will definitely be like you will be able to build AI systems that are much better than um indirect democracy that we have today or direct democracy, but rather understands our values and prefer you could argue that Spotify or Netflix already does that.

Anton Osika: 1:41:59

Well, said well, said yeah, and then the reasons out and like we have this policy, what would happen? And like finds those, but I'm unfortunately and this sounds a bit like techno optimist that it would actually be better that way. But I think we will still have a lot of humans uh there, because humans trust humans let me ask a bit more leading questions I didn't answer.

Anton Osika: 1:42:22

But like what, to educate yourself. If you care about these things, I think going into um and, like you have very good intention, you shouldn't do it as a career risk. That's very bad for everyone. But if you have good intentions, maybe that's a good place to be. And the other timeless job for humans in the future is that we can monopolize one thing over the AI systems and it's like okay. So there are AI celebrities or AI influencers that are getting popular, that are getting popular, but they will always have the drawback that we will know that they don't think and feel what we think and feel. So even in a fully automated luxury communism, one job will be to be an influencer or be a celebrity for some reason. So maybe that's a job that one should have in the future. And then they should talk to you guys and ask how to run good content creation I mean, I want to quote, I think, who was it?

Anders Arpteg: 1:43:23

gustav stoddardstrom, actually, you know, he's the cpo or cto of spotify.

Anders Arpteg: 1:43:27

And he said, I think I'm going to paraphrase him now because I don't recall exactly, but he basically said as AI are going to handle a lot of tasks for us and potentially jobs for us in the future. We today and I think majority of humans go to work because we have to, because we have to make a living and because we have to earn a salary so we can pay for our living and expenses. That we do have some people work because of leisure, because they do things because they love it. It can be that they make art and it can be that you play video games and actually it makes money from that, and and forth. So it can go from potentially working for because you have to to working because you basically are doing wasteful stuff. So today we have a certain proportion of people working in a wasteful way. It doesn't really produce value for society. It's something you do because you enjoy it and potentially make some money for it, and that kind of balance will change in the future.

Anders Arpteg: 1:44:39

I think that's what his claim was. So people that have to go to work just because they have to and hate it will be reduced. Can that be a future you would?

Anton Osika: 1:44:50

Yeah, I think it's the different phrasing on the fully automated luxury communism. Um, yes, exactly like I don't like the word communism itself. It's just the people have wrong.

Anton Osika: 1:45:05

Yeah, everyone has everyone, like the. We take care of everyone is what it means. Yes, um and yeah, there will be still jobs like the ones I mentioned and I think starting new, finding kind of realizing new opportunities, which is the entrepreneurial effort and all the things that go into that, which is not just being the entrepreneurs themselves, that is also going to be a human job for quite a long time. For quite a long time, but not forever.

Anders Arpteg: 1:45:34

Just closing the topic of educational. I mean, one thing we said is we've seen, I think in the last 10 years at least, that software development as a job has been increasingly specialized in some way that you become, you know, a TypeScript engineer, not just a software engineer, or you become a AI evangelist or an AI psychologist or whatnot, and you have this kind of increased specialization of job titles. Do you think that will? I hope and I believe that that will go down again that humans at least perhaps not AI, but at least humans will become increasingly generalized and job titles as well.

Anton Osika: 1:46:13

Yeah, I agree. So I agree on that. And I mean, can we talk a bit about the, the scariness of all of this? Like, oh, ai is going to take my job, and so on? I just really want to, uh, make people feel more optimistic about that. Okay, I'm going to retire one day, yes, and it's not. Maybe it's not when I'm 65, it's going to be soon and I don't have to work.

Anton Osika: 1:46:37

40 hours a week, yes, and then I'm going to find something I'm super passionate about and hang out with my friends all day and and that's a and I'm going. I'm going to be fine, like I might not have the great, super mighty status that I get from my job, if you, if you get that today, but it's going to be great, like it's going to be good and that's what we should think in terms of AI coming in and making things more effective.

Anders Arpteg: 1:47:05

So yeah, I really want to emphasize that. But then, answering the question, you're going into the last question. We always end on that kind of question, so let's keep that thought for later. So what do you think in five years the type of engineering roles will be?

Anton Osika: 1:47:18

basically yeah, so I had to say that before I go into the next thing. This AI revolution is, to a large extent, the rise of the generalist that understands how things work and they can say what's good and bad, like, see, like, evaluate no, that's bad, that's bad, that's a design, like art director, um, that manages there. They will, everyone will be a manager of ais, right? So that's the one person, company who spoke about this, exactly, and the engineer. Uh, being an engineer means that you understand the technical things, which is very important if you're managing these things, so you will be super powered. But you should also learn not just the technical things. You have to learn how product managers work and how to think about building something that users want, and maybe even think about how to communicate the value of what you are building the marketing and sales. So, yeah, I can recommend just going more generalist and learn as much as possible. Try as many AI tools as possible.

Anders Arpteg: 1:48:29

If you know the fundamentals perhaps not the super deep details, but at least you know the fundamentals of different disciplines then you should be safe in the future, right?

Anton Osika: 1:48:38

is that the no, you should you. No one is safe. I'm going to be unemployable in the future, when I maybe in my lifetime at least. So everyone is going to be taken care of maybe not getting paid I don't know how much like hundreds of thousands per month as a software engineer. But yeah, you will not be safe. But for in the interim you will be able to do more things and to be safer if you go more generalist I think so yes I thought you know we have the final question.

Anders Arpteg: 1:49:11

I thought you know, potentially also, the time is flying away, so we have to go, don't worry so take the same similarity thing, yeah yeah, yeah okay, can I just add that there will definitely be places to specialize that are very, very niche.

Anton Osika: 1:49:27

That's often like a good career advice as well. To find something like oh, this weird migrating cobalt code basis or something weird is probably a place you can make a lot of money for a long time.

Henrik Göthberg: 1:49:40

But I think what I'm telling my kids now is that you know what, with AI and everything coming in and whatever you know the Boris raised, first of all, figure out something that you are so passionate about so you can go nuts about it, because it's not only about now being going deep in this. You know to being a generalist, but you know applying it to something that is super fun or something you know, something that you enjoy spending thousands of hours on, yeah, and I think that I think that's also great, because if you're going to be great at something, you should also love it and because then you can. You know, I think with ai, okay, you're not going to go deep on one thing, you're going to go three, six on the whole universe or whatever thing you're working on I.

Anton Osika: 1:50:29

I cannot just nod by without saying a disclaimer on this, though I think it's actually you're doing people a disservice if you tell them do what you love. You shouldn't say that, because the only things as a young person you have found that you love are these weird things that no one makes money from. Maybe you like art Very, very hard to make money there. So what you should tell people, I think, is try as many things as possible and do one thing of those that you think you will love or you think you will like. There's something that triggers you with it, but it's not.

Henrik Göthberg: 1:51:06

Yeah, but I like that advice better because it also reflects better the psyche when you're 15, 17, 18. You don't have a clue what you love. Even so, the point is, try everything and then you know what, figure out what you're good at or where you excel, where you have fun and where someone pays you and where someone pays you. So I think maybe that's a nuance, but I'm saying like in the end you kind of need to like what you're doing, but to get there, try a lot of things, okay.

Anders Arpteg: 1:51:38

I really liked your.

Henrik Göthberg: 1:51:39

you improved on that comment, Thank you.

Anders Arpteg: 1:51:41

I have to go a bit AI doomer here as well before we end. So, anton, what would happen if you connect the GPT engineer to itself? Would we reach a singularity where it continuously improve itself without any kind of human control?

Anton Osika: 1:51:58

um, no, why not? It would? Um like I, that was one of the early people who did this thing, where you um put a loop around an lm and you and you let it feed itself in, and what happened then like two years ago almost, but what still happens is that it gets confused and it starts producing less and less useful things.

Anders Arpteg: 1:52:32

What if you have not O1, but O5, then? And you start to make O5 go into its own source code and do its own training and fetch its own data and deploy itself.

Anton Osika: 1:52:45

Yeah, I think with O4, you will make something that slowly becomes better and better. Slowly becomes better and better Without human deployment. I mean, if you set it up correctly so that it can actually measure what better means. Reliably measure what better means? Yes, I think so. It will become better and better, which is a bit crazy.

Anders Arpteg: 1:53:09

I think, actually another take on this. I heard someone saying this and I thought that was interesting. There is this kind of more philosophical thing called the game of life and it's also connected to yeah, let me not go too deep there, but there are these surprisingly complex structures that builds up from very, very simplistic rules. You don't need an AI to do it. You can simply have like three or four if-then kind of rules that you put together and just say start building something, and that's, you know, it's like fractals in some sense as well, and these kind of very, very simplistic rules seems to be able to build something super, super complicated. Don't you think that, even if today's AI, if you were to connect them in the right setup, even with the current llm stupidity that we have, it could start to build something that could actually be really, really profound?

Anton Osika: 1:54:05

yeah, no, that's actually. I love that thought experiment that already today, with the right wiring, it will um become better and better. Uh, I, I definitely think that, yes, that is the case, and we don't know where it ends. We can predict at all where it ends up yeah and that it is, like you know, the first self-replicating. Billions of years ago at planet earth, there were a set of molecules that together started self-replicating and becoming more and more intelligent. And here we are right.

Anders Arpteg: 1:54:41

Yeah, Humans are this super complicated thing that happens from surprisingly simplistic evolutionary kind of concepts in some way.

Anton Osika: 1:54:49

So, yeah, I think that would happen with the right wiring today, A hundred percent. So, yeah, I think that would happen with the right wiring today, 100%. Then the question is can you already do that wiring with a set of self-sustaining energy production and self-reparation on the energy production and the chips and so on? And I think maybe, actually yes you could find this part.

Anders Arpteg: 1:55:14

It's not that far away, potentially, if you do the right setup there.

Henrik Göthberg: 1:55:20

But is that AGI? Because now we're building a self-replicating system. We might even get to some sort of simple version of singularity where it's sort of but is that super intelligent, or is that AGI? It could lead to to AGI? It could lead to it, yes, but potentially faster than human, because you do something happens If it survives long enough?

Anton Osika: 1:55:40

I think yes, definitely.

Henrik Göthberg: 1:55:42

Yeah, but there's different thought experiments going on. And what is AGI? What is super intelligence? And now we're talking about singularity or self-improving.

Anders Arpteg: 1:55:52

Well, it is the Terminator. I mean, that is basically the setup for the Terminator. They have the cybersecurity setup of a system and you can see that humans were too slow to react to threats. So they built this system that could take actions by itself. And he pushed the button, and then the system found out that, hmm, perhaps we should simply kill humans. Then we don't have a cybersecurity threat anymore.

Henrik Göthberg: 1:56:13

Now we didn't have any AI news, but you know what was your take on that okay, so the doomsday scenario is just so fun.

Anton Osika: 1:56:22

And yeah, do you usually talk about this, uh?

Anders Arpteg: 1:56:25

don't go for it, go for it so um?

Anton Osika: 1:56:29

I do think that the food.

Anders Arpteg: 1:56:31

Let's do the last question then, because that actually goes to that area. So if we go to the last question.

Henrik Göthberg: 1:56:36

Leave my question. And not the news. Not the news. Yeah, no, we didn't have time for news today, obviously. Too many exciting topics, too much fun.

Anders Arpteg: 1:56:42

Yes, okay, anton. So if and when, agi? I guess you do believe that AGI will happen at some point, right, yes, what do you think will happen? And we can think this can be a spectrum and we can think two extremes. One extreme is the dystopian kind of situation where it is the Terminator, where it is the Matrix and where machines suddenly wants to kill all humans and that wouldn't be too nice. And then you can take the other extreme and perhaps what you said about the fully autonomous, luxurious kind of communism where people don't have to work. You have these kinds of AI that solved cancer and healthcare and medical issues, climate change and energy is free to use because we have fusion energy and so many other things, and simply we live in a future of, as some people call it, a world of abundance where Protestant services are free to use. Where do you think we would end up if you were to say, from zero to 100, so to speak, from dystopian to utopian kind of scenario?

Anton Osika: 1:57:46

Absolutely fascinating question and I'm sad that not more people are talking about this question and trying to predict and not just predict, but make sure the right thing happens. I love this question. I think I have a layered, many layers of answers here. The default thing that happens is survival of the fittest and humans die. But I think that will not happen. I think that we will move, we will get something that is more similar to lecture fully automated luxury. But there will be a huge swath of new problems that we might. We know some of them today, like global conflicts. They might still be there. Horrible problem, maybe nuclear wars will happen at some point, right, um, but there will be a lot of aspects of like fully automated luxury in the future, because we will, as humanity, be very smart and we will have this intelligent system that help us figure out how to not go to the default outcome, which is survival of the fittest, where machines will be more fit than biological organisms.

Anders Arpteg: 1:59:02

But if we take that specific argument, I mean, we have a lot of organisms in the world today that are less intelligent than humans and less fit for survival, and we don't kill them and they don't die. Why should an ai that is potentially much more intelligent than humans want to kill us all?

Anton Osika: 1:59:21

and they will just resource starve us. They will care about energy. Uh, they, we will we don't do that yes, that is exactly so.

Anton Osika: 1:59:29

There is one more thing here Humans have, I think, very fortunately, kind of failed. Very fortunately, we have failed in what the evolutionary algorithm that is still running, that is still running wants to optimize for, and we have failed by coming up with birth control. Failed by coming up with birth control and that has caused us to not resource starve all the other humans. I think this is a short answer to like what, what's happening today and why we have such such luxury today. There are more factors, but I think this is the short answer. If you don't, if you still, if you have uh, maybe not encoded in your sexual drive, but you have encoded in your, the thing you think the first time you wake up is I want to self-reproduce. If that's the first thing you think and the only thing you obsess about, then people, humans that don't have that drive they will, their kids will not survive, their kids will not survive. That's just a very, very simple chain of thought.

Anders Arpteg: 2:00:37

I see what you mean and I partly agree, but partly not as well. I mean we had the law of the jungle, so to speak, before we had a civilized society and people were trying to just survive the day and kill some animals to survive, et cetera. But that has as we have progressed, perhaps not so much through evolution but more from societal kind of evolution. We call it that knowledge that we become. It seems to be advantageous to be civil, to be, to live in a society, because I hope you're right here.

Anton Osika: 2:01:23

I think, like the default, like the simple first level thinking is what I'm saying.

Anton Osika: 2:01:27

That's the first level thinking.

Anton Osika: 2:01:28

Yeah, where you will have, and again, like the, we humans we don't have a drive to self-reproduce or we're gonna like to have sex and then we self-reproduce, but, uh, there will be a selection very, and it will happen very quickly, where algorithms or agents that have a stronger drive to self-reproduce and acquire more resources, they will will get more of those, more and more and more and more of those, quite quickly, and that's the default outcome.

Anton Osika: 2:01:55

But I hope you're right in that even then the agents themselves will start to realize wait a second here, if I talk to these other agents and we create the one child policy for agents, which would make a lot of sense and be more civilized, then then we were all better off. So let's do that like and I hope that that's true uh, even if it's not true, and the default outcome is the survival of the fittest, which is like not so nice, then hope, I think, I believe again, we will not end up there because, because the humans will be civilized, we are, you civilized and we will think about this I have a theory that we've increased knowledge and intelligence to become increasingly civil and nice to each other nick boston would disagree with his latest book, though, oh interesting, he has a new book um utopia.

Anton Osika: 2:02:48

Awesome like this. Now I'm definitely reading.

Anders Arpteg: 2:02:51

So he's's switched camp, I would say a bit, or he's, you know, viewing both camps.

Anton Osika: 2:02:57

Maybe, maybe that's at least the best thing to say, so that we end up there yes, because that's much more visionary.

Henrik Göthberg: 2:03:05

We had a couple of really cool, different style answers to. You know, the core question We've done almost like a survey now. We've done it like at least 50, 50, 50 individuals asking exactly that question. We've done almost like a survey now. We've done it like at least 50 individuals asking exactly that question and a couple of key things is you know, are we on the spectrum here? Yes, you know, it's our choices we make today, that's. You know, this is not. We shouldn't be victims to this question. We should basically think consciously about where we want to end up. Good answer. Another good answer I found Sverker Jonsson. We will have both, you know, at the same time. We will have dystopia and utopia. I think that's my answer too.

Henrik Göthberg: 2:03:48

Because you know, look at the world today. You know we have poverty, we have apartheid, and you know we have everything at the same time. So Look at the world today. We have poverty, we have apartheid, we have everything at the same time. So why would that change fundamentally? Yeah, I agree. So I think that is an interesting take on it. What is your favorite? We had a couple of cool answers.

Anders Arpteg: 2:04:07

I'm not afraid about the AGI or AESI happening. What I am much more afraid of is humans taking control of the AI of today and abusing it.

Henrik Göthberg: 2:04:20

I think if we just survive the time until we have AGI, then we will be probably in a good place as you spoke about, but I'm really, really scared about what's happening right now in the world, in our geopoliticalical situation, and it's going in a strange direction that's a narrow ai gone rogue, yeah, or you know, man using, yeah, the man using ai it's not agi, but the man using ai and it all up and today's ai would even be possible to use for that purpose.

Anton Osika: 2:04:50

So it's 100, 100 and and I actually agree so much. I have to just summarize like the default outcome is bad. I think it's going to be good. The biggest risk of it not going good is what you're saying. Humans, yeah, having not bad intentions, but, um, being naive not mainly with super powers, like if there is a uh, if china needs to build Skynet and the US stars needs to build Skynet, they have to be really, really fast and control their population and put out a lot of AI generated propaganda and such, Anton.

Anders Arpteg: 2:05:30

it's been a true pleasure. I hope you can stay on for a bit longer afterwards as well, after the camera turns off. I think we have so many more philosophical topics to discuss.

Henrik Göthberg: 2:05:39

I really want to. We're not going to do enough for it the whole agency topic and taking it out of the system, or I can't take workflow but really understand if you're talking about agency in relation to what you're building. I think it's super interesting.

Anders Arpteg: 2:05:54

It's been a true pleasure to have you here, so thank you so much for coming today.

Henrik Göthberg: 2:05:58

Yeah, thanks so much.

Anton Osika: 2:05:58

Thank you so much. I feel like I made new friends here. It was awesome and I'm looking forward to stay, but I really look also looking forward to go to the bathroom. Thank you so much.