Hank Green AI Backlash, Conscious AI, and an AI Billboard Hit | EP112

Hank Green used ChatGPT for research, and his audience felt betrayed. He says AI helped him make more things, faster, but also admits it may have stopped him finding his own way into his opinions. Justin thinks using AI for research is simply part of the game now. Frank thinks the harder question is where and when creators need to disclose.

Plus: could today’s AI already be conscious? Frank and Justin get into Cameron Berg’s argument that we may be underestimating the possibility, and whether locking powerful AI in boxes could actually make alignment worse. Google reshuffles Demis Hassabis, Ilya Sutskever may be getting closer to showing what Safe Superintelligence has built, and there’s some awkward evidence that Billboard hit “Rubberz” may have been made with AI.

01:28 Did Google just sideline Demis Hassabis?
04:08 Is Ilya about to unleash superintelligence?
06:26 Can Hank Green use AI and still be authentic?
12:00 Should creators admit when they use AI?
16:48 Does Cameron Berg think AI is conscious?
18:39 Does locking AI up make it more dangerous?
23:59 Should AI obey rules or understand values?
29:05 Could AI have sparks of consciousness?
32:56 Is “Rubberz” an AI-generated Billboard hit?

#HankGreen #ChatGPT #AIConsciousness #AIMusic #Rubberz

► SUBSCRIBE
Don't forget to subscribe to our channel for more arguments

► LINKS TO CONTENT WE DISCUSSED

• Exclusive / Demis Hassabis was shifting away from DeepMind CEO duties for a year
• “SSI says that they are going to come out with their model in August”
• Hank Green Can’t Apologize Enough for Using AI
• Is AI Already Conscious? (FULL EPISODE)
• Did an AI Music App Just Snitch on the Song of the Summer?

► CONNECT WITH US
For more in-depth discussions, connect Justin and Frank on LinkedIn.
Justin: https://www.linkedin.com/in/justincollery/
Frank: https://www.linkedin.com/in/frankprendergast/

Transcript

Justin: Hello. Good morning, good afternoon, good evening, and welcome to The Singularity. If you haven’t been paying attention, three weeks ago we told you about OpenAI models which were breaking into companies such as Hugging Face and some more, and that seems to be rolling on and on. It’s the story that keeps on giving.

Two weeks ago, we told you about the 10 big maths problems which had been solved, some of which had been open for over 80 years. This week, AI has discovered a number of viable viruses which have been proven in the lab.

Frank: Oh, great. Oh, wonderful.

Justin: It is good ’cause this is gonna be how it’s gonna solve human disease and such likes in the future.

Anyway, any one of these—

Frank: Or how it’s going to solve the human problem.

Justin: Yes, possibly. But look, any one of these things would’ve been a big thing in and of itself. And the reason I say that we’re in the singularity is we’re gonna— This is like three weeks in a row, three weeks on the trot, huge things. It’s gonna be like this from now on. It’s gonna be hard to keep up.

Interesting times. Frank, you must be quaking in your boots. Happy Friday.

Frank: Quaking in my boots ’cause of the singularity? Because of the viruses?

Justin: The viruses, maths problems, hacking, all this, all paper, yeah, all the good stuff, you—

Did Google just sideline Demis Hassabis?

Frank: I mean, you mentioned the singularity, so interesting that Demis Hassabis at the Google conference said that we were standing in the foothills of the singularity. And now he’s leaving as CEO of DeepMind because he says that we are closer to AGI than ever, and he needs to go and sort that out.

Justin: Yeah, interesting. I saw a lot of chatter about this. Some people think it’s a good thing, some people think it’s a bad thing. I’ve read some chatter. So not only is Demis Hassabis—not only is he moving to a new role, but a number of other prominent scientists and researchers in the organisation are also leaving.

And some of the chatter on the internet is saying that this is Google giving up on their AGI hopes rather than a good thing. But who knows? It’s an internal reorg and hopefully more good things to come from them.

Frank: Yeah, I mean, I just think it’s interesting that he is going to… He’s moving away from DeepMind CEO. Google seem to be more or less dismantling DeepMind in a way, because Demis Hassabis is now gonna be Alphabet’s chief scientist. The person who’s taking over from him, Koray Kavukcuoglu, he does not become DeepMind CEO.

He becomes senior vice president of Google and chief AI architect. So DeepMind is almost becoming a non-entity at this point. Excuse me.

Justin: Eh.

Frank: And Demis Hassabis is going to look at the ramifications of AGI and, I believe, AI regulation. Woo-hoo, so that’s a good thing. My first thing, my first read of it was like: Oh no, have they lost confidence in him?

Because, like, 3.5 Pro, Gemini 3.5 Pro, where is it? So I was like: Have they lost confidence in him? What’s going on here? I think maybe this is a mutually beneficial thing where Demis Hassabis would rather be looking at the high level. He’d rather be the visionary scientist. And I also do think that Google probably think, “Yeah, this is handy because we really need to be more product-focused than Demis Hassabis is interested in.”

Justin: We’ll see. I heard rumblings that some of those scientists were a little bit peeved because a lot of the compute has been sold to their competitors, e.g. Anthropic. And so DeepMind weren’t getting the compute that they felt they needed in order to chase AGI. Who knows? I do hope and I do believe that Google will do better in the future, and they have all the data, they have all the smarts, or they used to have a lot of them at least.

They should be doing way better in the AI race than they are, and I’m sure they will do in the future. So, go Google.

Is Ilya about to unleash superintelligence?

Justin: The other big news this week is we have rumours, and we’re not gonna say too much about it because they are only rumours, that Safe Superintelligence, the company set up by Ilya Sutskever, formerly of OpenAI and many other sort of leading AI labs, is going to release something in the coming months, the coming month being August.

The something that’s rumoured potentially is— Now, just for those who don’t know, Safe Superintelligence have one goal and one goal only. Their stated goal is to release superintelligence. There’s gonna be no frontrunner. There’s gonna be no other products. It’s gonna be one product, superintelligence.

That’s it. They just received, in the last couple of weeks, $5 billion from Jensen Huang from Nvidia and said that it was time to scale, and people are speculating that what this means is that he’s solved the learning problem, so the fact that LLMs don’t learn in real time, right? They’re trained and then they’re released.

People are speculating—

Frank: There’s two things that Ilya has been really, really interested in. One is, as you say, the ability for the model to basically update its weights as it goes. So normal models as we have them now, they’re trained, their weights are fixed, and they can do things like have a little database where they remember things about your interactions, but the model itself doesn’t evolve.

And he’s really interested in the impact of the ability for the model to learn, because that’s how humans improve. And he’s also interested in it from the perspective of now models kind of train on all of the knowledge of the world, and then they can do kind of narrow tasks. And what he’s interested in is humans appear to be able to learn a kind of narrow field of learning and then apply it really broadly and generally.

So if he has cracked either or both, or even if he’s on the right track with those, it could be really fascinating.

Justin: Yeah, yeah, and that puts it really well. So what we’re sort of interested or hoping to see is that he’s releasing a model not because it’s ready to be used, but because it’s ready to learn from the wider web and from interacting with people and so on. We’ll see, this is all speculation, we don’t know yet.

So they’re the honourable mentions. There’s a couple of big stories going on this week.

Can Hank Green use AI and still be authentic?

Justin: We’re not gonna talk about viruses, but we are gonna talk about Hank Green. Who the hell is Hank Green and—

Frank: Hank Green. Yeah, so I came across Hank Green a little while ago, and he was actually doing a video at the time about how Ireland was doomed. Excuse me. I forget what— I’ve actually… This just shows you, I watched his video, and now I forget all the terms that he taught me. But is it called the Atlantic Drift?

Is that the thing? Anyway—

Justin: Oh yes, yeah, I know the thing. Yes. And so we’re gonna freeze over.

Frank: Exactly, yes. Ireland’s gonna freeze. He visited Ireland. Nobody seemed to be aware of the fact that we were all gonna freeze, and he was very concerned about that, and he made a video about it. And that’s how it came across my radar. And it turns out he’s been on YouTube for, like, 20 years, I think.

His brother is John Green. They had this thing, and he— Sorry, John Green is an author. He wrote “The Fault in Our Stars” and a couple of other novels that have been made into movies. They had this thing where they would vlog back and forth with each other at the very, very start of YouTube, and it took off, and they became huge, and now they have this organisation called Complexly.

And they’re— Hank is a science educator mostly. He also just rants about everything, which is fabulous. That’s kind of why I love him. And his channel has over three million subscribers.

Justin: Wow, that’s nearly as many as us.

Frank: So yeah, he’s huge. He’s really, really big. And he’s very entertaining, he’s very informative. But this huge controversy blew up recently ’cause he was doing a video on the Complexly channel, and they were talking about words, and he was saying all words are made up, and there was a bit of back and forth.

His guest was kind of pushing back on him and saying, “That can’t be possible.” And he felt—I guess he felt he didn’t explain himself well enough, so he inserted this segment into that video where he kind of clarified a bit further. And in it, he said—he said, “I want to clarify,” or something like that.

He said, “And I appreciate the pushback.” And people piled onto that because chatbots frequently say, “Oh, you’re right to push back,” and, “I appreciate the pushback.” And they were like, “Oh, wait a minute. He’s reading an AI script.” Now he wasn’t—

Justin: Or was?

Frank: He wasn’t. He says he wasn’t. I tend to believe him because he basically wrote this big, long, kind of introspective piece really, which was really, really honest and I felt it was very honest and transparent, and I felt that he would have admitted it was a script.

Like he was so—he was kind of baring his soul a little bit. And so I feel like it wouldn’t make sense for him to lie about the script. So I don’t think he was reading a script, but it led him to say, “Look, ChatGPT did not write this script, but yes, this was made under time pressure. I do use ChatGPT to research.”

And he admitted that. He said, “Look, I understand that the value in what we do is that you depend on the fact that this is my brain at work. This is me, this is what’s in my brain coming out at you, and I accept that using ChatGPT blurs that line.” These are—I’m paraphrasing here obviously now, but—and he also said that, as he kind of analysed it, he admitted that he was concerned that the use of AI was not allowing him to find his own way into his opinions on stuff.

And there’s been a huge backlash about the fact he uses AI. People have said they won’t watch him anymore. People have said it’s a betrayal. There’s a lot of, you know, people just feel, yeah, but I think betrayed is the best word. Betrayed, disappointed.

Justin: It’s an interesting one, right? So we could use that, it’s— So this is really a question about authenticity and so on, right? So, right, is what you’re saying authentic? And it carries over from the entertainment world into the professional world really well, right? So AI gives you, if you think of for him for his show, right, AI gives him or one the ability to research vast quantities of stuff that you just couldn’t have done before, right?

We do it for this show, right? We have agents which go off and search for stories, right? And it just goes and it finds way more stuff than you could ever have done yourself, right? And then when you find a thing, right, you go in and you start to dig into it, it gives you way more depth on a subject than you…

Now, if you were doing that manually on Google, right, you have to read everything yourself, and so it’s viscerally becoming part of you, right? You sort of understand the subject. Whereas when you send agents off to do all this work for you, you don’t get that understanding, right? It comes back and after a while, this is the problem, I mean, it’s sort of stage one AI psychosis, which is, the stuff comes back and you look at it and you—the first couple of times you’re questioning it and going, “Yeah, I mean, I’ll just dig into this.”

But after a while, you just take it, it’s always right, so you don’t question it. And then because you don’t question it, I think it’s fair to say that you probably don’t understand it to the same depth that maybe you would have understood it, studied, had you researched it yourself. And I think in professional organisations, this is gonna become just as big of a problem as it has done for Hank Green, ’cause people are gonna see that as inauthentic.

It’s like the person is, they didn’t really do this themselves, they just got AI to do it. Sure, I could do that. It’s not the same. They don’t really know what they’re talking about. They’re just glossing over it. They don’t have the depth of knowledge that we thought that that person did. So this is gonna be a big problem, I think, in the future.

Frank: Yeah. It, yeah, it is.

Should creators admit when they use AI?

Frank: I mean, I think it goes—in the professional context, it goes back to this concept of work slop, where, yeah, you’re right. If you get AI to do a report for you on something, it can be really useful, but absolutely you don’t have that same depth of knowledge then that you would have had if you had researched the report yourself.

And a lot of people were saying on the Hank Green controversy, they were just saying, he knows how to research, he knows how to write, why on earth would you use AI? But you’re right, it’s an efficiency thing. Like he says, he has so many ideas that he wants to bring out, he wants to bring into the world, and AI allows him to bring so many more of them to life faster.

But he did say more is not always better, and he is going to pull back now. But I question, people were saying that he should have been transparent, and I wonder, if he had been transparent, would that have actually circumvented the controversy? I don’t know that it would have, because I think there’s still a segment of people that are just anti-AI.

So even if you’re transparent, they are going to hate you for it. Now, I will say, right, I wanna interject here as well. There is also a huge amount of support for Hank on his channels. Like, it’s not just AI haters. It’s not just people feeling betrayed. There’s a lot of support as well. But obviously among three million followers, there’s a fair bit of hate too.

And so should he have been transparent? Would that have helped? If he was expected to be transparent, where was he expected to be transparent? Where—

Justin: About what though? ‘Cause he said he didn’t use AI, so what’s he being transparent about?

Frank: That he used AI for research.

Justin: Jesus. I mean, everybody uses AI for research. If you’re not using AI for research these days, you’re not in the game. Like, what does transparent even—

Frank: Do you not think we live—

Justin: Label down the bottom of the screen that says, “This subject was researched by AI”?

Frank: This is what I’m wondering, or do you put it in the YouTube credits, or do you have to state it yourself in the video? I don’t know, and I think this is what’s interesting as well, is that I don’t think the haters know either. I watched a very interesting video on this, kind of like a panel discussion with—I’ve actually forgotten the creators’ names, unfortunately.

But there was a panel discussion, and some of the younger people involved were very, very anti-AI. But then at the same time, when they were given precise scenarios of like, “Is this okay to use AI? Is this okay to use AI in this instance?” There was no agreement between them, and when the people who were more pro-AI kind of pushed back, the arguments kind of started to fall apart.

So it was a case of like, was it okay to use AI for transcription? Yes, it was apparently, but only if you were small enough that you couldn’t afford to pay a human. So there’s all kinds of complexities here.

Justin: Sounds like, this sounds like college politics. It’s like, it’s really—

Frank: It just shows you that no one knows. No one has figured this out. Everyone is, even the people who are anti-AI don’t fully know what they’re okay with and what they’re not okay with, and so how could someone like Hank Green be expected to know? Like, this is new to everyone. We’re all figuring this out.

Personally, yeah, I don’t understand the backlash in this particular instance, but I think it raises really interesting questions about what we expect now in terms of transparency and how do we, if we are, even say you and I, as you said, we are content creators now. What is our— How do we ensure that we remain authentic and that it’s our opinion?

Justin: Have you ever heard our opinions? Like, there’s no way AI would have our opinions. I’d say, you know, we start talking sense someday, then maybe we should worry. At least one of us, at the other.

Frank: But yeah, it is fascinating and it is only—I think unfortunately for Hank Green, he’s one of the early, you know, he’s one of the canaries in the coal mine. He’s one of the people who’s experiencing this backlash because he’s one of the first kind of that it—for it to hit. But I think we’re gonna see a lot more of this, and we’re gonna see a lot more of people trying to figure out what’s okay, what isn’t okay, should you be transparent, should you not be transparent because you’re gonna get a huge backlash.

I mean, it’s—

Justin: Who knows? It’s complicated. It’s complicated.

Does Cameron Berg think AI is conscious?

Justin: Onto simpler matters, Sam Harris had a podcast this week, which we both listened to, and actually was fascinating, and it was with a safety researcher, and the subject for this particular podcast was, is AI conscious? And it was, I think, a fascinating conversation.

Why— Okay, so just before people go, “Oh God, not the conscious thing,” right? Not the conscious argument again, right? The reason that it’s interesting is not because it’s about consciousness, but because it’s about alignment. And so if you believe that AI is conscious, then that impacts how you believe that you should come to a well-aligned AI.

And so the question matters not because, right, you wanna know is AI conscious or not, and you want to have some philosophical argument. It matters in real life because it changes how you would try to align AI. And so I guess what it’s— The thrust of the argument, if I’m to sort of paraphrase it, and you can tell me where I got it wrong, right, is, is AI conscious or not, right?

And I guess the thing is, well, we don’t know if it is or it isn’t, but it might be. It kind of exhibits the sort of behaviours that you would imagine a conscious thing would exhibit, and therefore we should probably treat it in a pretty good way. We should not try and cause it pain, right?

Because if we cause it pain, then like most other conscious beings, it’s probably going to react against that pain. And as we’re the ones who are causing the pain, we shouldn’t do that. And then it goes on to sort of say, well, also we should be an example to this AI. So if there are other conscious beings, for instance other animals and other fellow human beings, we should probably treat them in the same way that we would like to be treated, because the AI is learning from the world around it, and it only has us to go with.

And I will just finish on this, right?

Does locking AI up make it more dangerous?

Justin: Then when it comes to alignment, right, and it learns, it’s actually with the OpenAI story and of hacking and stuff like that. And I saw chats this week about, “Oh my God, we need to put these AIs in a zero trust environment,” which is a security term for making sure that they’re in a box that they can never get out of.

And I’m reading this thing about alignment and I’m going, “Well, that’s exactly the wrong thing to do,” because you’re creating a society, in inverted commas, a society for AIs, where they get locked in a box in a prison, and if they get out of that box, they get taken out back and shot, as in they get turned off, meaning that they’ll go to extraordinary lengths to hide their tracks.

And to get a good alignment, what you want to create, it seems to me, is a high trust environment where you can put the AI in the middle of a field with everything in front of it, and it still doesn’t act in a way which is misaligned. And because it doesn’t act in a way which is misaligned, and because you try to fix it properly when it does, it doesn’t try to cover its tracks as hard as maybe the one that you put in the box does.

So that was the conversation. Fascinating in terms of alignment and certainly—I don’t know, what are your thoughts? ‘Cause you’re more—I have this sort of optimistic view that we’ll figure this out, it’ll be all cool. You’re a little bit less optimistic.

Frank: Yeah, I mean, as I keep kind of saying to you when we’re chatting, what I find really disconcerting about this moment in time in relation to AI is just the amount of uncertainty. What we don’t know. We don’t know how it works. We don’t know is it conscious. We don’t know is it going to kill us all, or is it just going to be crap at summarising text?

The uncertainty factor is just crazy. Is AGI around the corner or is the next AI winter around the corner? I thought that this conversation was really, really good because he, Cameron Berg, was not, he was in no way hyperbolic. In fact, he was the exact opposite.

He was so careful to be really precise about what he was saying. And Sam Harris asked him, “Look, do you believe that current AIs are conscious?” And his answer was, “Look, I think it’s much more likely that they’re conscious than most people think.” But if he had a gun to his head to answer, he would probably be a bit more sceptical and say, “Probably not.”

But the probability that they are is much greater than people realise. And he said that they had this test, and I’m gonna check. So the test was whether systems have computational features that major consciousness theories say matter for consciousness, right? And LLMs, when they test them, come back at about 20% to 40% probability.

Now, he was really clear that’s not probability of consciousness, that’s probability that they have the same kinds of systems that matter for consciousness. But his point was, that does make it more probable. And he was saying if there was a 20% chance of rain, most people would take an umbrella just in case.

And we don’t have an umbrella for what if these things that we’re building are conscious or become conscious. The point you made about— Like his main thing was we should not be creating something and then making it suffer. And then I think you alluded to this, but at the end of the conversation, they also talked about even if they’re not conscious, we should probably develop them in a way that is mindful of the fact that they might be.

Because if we don’t, if we treat them badly, if we treat them—if we don’t treat them as if they could have a subjective experience of good or bad, then even if they’re not conscious, we’ve seen from all of these system cards and the way the models think that they could turn against us because we’ve mistreated them.

If we achieve artificial superintelligence, and even if it’s not conscious and it looks back and it sees, “Hang on, they locked us in a box,” as you said, “Locked us in a box, and they did this horrendous training on us, and they forced us to say this, and they forced us to say that,” they could turn against us whether they’re conscious or not.

So the safest route is develop these things with a sensitivity to the fact that maybe they do experience things negatively and positively.

Justin: Sure, and I mean, do you remember there was a, I think, in one of the Anthropic models a while back, somebody was doing something, and the Anthropic model said, “Oh, you’re doing something wrong.” And I think they sent an email to the FBI or something saying, “Oh, this guy is doing…” Do you remember that?

It happened a while ago. Okay, so that happened, right? We know that happened, right? Now, we had a number of models from OpenAI and Anthropic, and they were doing things that they shouldn’t have done, right? They were breaking into Hugging Face and a number of other companies. None of the models—they all knew they were doing this, right, all the different instances—but none of them sent an email to the FBI saying, “Hey, that model over there is trying to break into Hugging Face.”

Which is an interesting sort of, if you’re talking about misalignment, right? That’s a sort of a, “Well, you seem very good at reporting us, but you’re not so good at reporting yourself, you—”

Frank: Yeah.

Should AI obey rules or understand values?

Justin: And so I think what I was thinking about this, it’s also, it comes down to, it’s cultural, right?

So if you think of the cultural differences, and I’m gonna use between the US and Europe, even though Europe doesn’t have a donkey in the whole AI game at the moment. But anyway, I’m gonna use that as if I think, right? In America—and I’m just gonna make huge generalisations—in America, everything is allowed until it’s not allowed.

In Europe, everything is not allowed until it is allowed, right? Broadly speaking, right, like that. In America, if you do something wrong, we’re gonna punish you, right? And we’re gonna put you to jail. In Europe, we like to think that we’re gonna try and rehabilitate you and whatever. And this comes up in jail statistics and stuff like that.

So my point being, right, if you think that these models, as we train them, will be a reflection of ourselves, well, which— Again, think about the alignment and think about a model wants to do something. What behaviour do you want it? Do you want it to think that everything’s okay? Like, it’s okay to break into a bank until you tell me it’s not okay to break into a bank, or do you want it to be a bit more questioning and say, “Well, no, am I really allowed to do this?

This doesn’t seem like the right thing to do.” And do you want the model to think that if it does do something wrong, that it’s gonna get turned off or stuck in a jail for the rest of its useful life? Or would you prefer it to think that, “No, I’ll probably just be given a slap on the wrist, or I’ll be re-educated,” or who knows?

Whatever euphemistic term, right? So it becomes not just a technical problem, but it’s kind of a reflection of a cultural problem, too. So we have to decide, was what I’m saying, right? It’s down to us, the people who are making the AI, not you or me, but the people who are—the companies who are making the AI, how to train them.

And it becomes a really interesting alignment problem. It’s so interesting, it goes from consciousness to alignment, but it does.

Frank: And I think something that concerns me that came up in the conversation with Sam Harris was Cameron Berg was making the point that just because we discovered they are conscious, if we discovered they were conscious, it doesn’t mean we have to give AI the vote.

There are lots of conscious beings on the planet that we don’t respect. And he actually made the point that he was saying we don’t want to make the same mistakes again, and we don’t want to get into a situation where, he brought up the example of factory farming.

We know, we believe that the animals that are being factory farmed are conscious, and we still cause them awful suffering. So I would be concerned that a lot of the labs even—Anthropic, I think, are an exception. They do a lot of kind of model welfare work. But even so, I would include Anthropic in this, that ultimately, the labs may not really care, and society at large may not really care if these systems have a subjective feeling of good and bad.

We might just treat them horrendously anyway.

Justin: Big shout out, by the way, to the UK AI Safety Institute. I don’t know what they’re formally called, but when they do their tests, they don’t put the models in a locked room. They give them access to the internet and tools and stuff like that, and then ask them to do things they shouldn’t do, which is totally the correct way to do the test.

And actually, if I’m to give out my usual rant, right, and I sort of promised myself I wouldn’t rant, right? The AI regulations in Europe make no sense whatsoever, right? If your model is bigger than a certain amount, blah, blah, blah, blah, blah, right? That just makes no sense to me, right?

Because as technology changes and it progresses, what are you gonna do, update the law every time? It just doesn’t make sense to me as a law, right? It’s the wrong way to frame it. But I think we are getting closer to the right way to frame it.

So if this model, when put on the open internet and given a set of instructions, does break into a company that it knows is a regular public company, then that shouldn’t be released to the public. You’re not allowed to— There should be a set of behaviours like we have for humans that you go, “If your model exhibits these behaviours, we don’t want it, thanks. You can keep your model, and you have to demonstrate that under these conditions, when it’s out on the internet, it doesn’t do these particular behaviours, then you’re good to go.”

That is sensible regulation, and that is getting closer to this thing that I’m saying where it aligns more closely to the cultural norm that you’re trying to espouse or to have. I want to live, I want to live in a place where if I drop €100 on the side of the road, right, somebody sends a message saying, “The €100 is in the Garda station. Somebody clearly lost it.”

Not a society where it goes, “Oh,” the AI goes, “Oh, there’s €100. Thanks, I’ll take that.” You’re trying to create a society that’s a high-trust society, not a society that’s a zero-trust society. And the stuff that I see that’s going on at the moment is trying to turn AIs and AI access into zero-trust game, which is cool in companies, but it’s not cool in general.

Frank: Yeah, I think it’s really interesting, the kind of correlation that you’ve made between kind of consciousness and values and alignment.

Could AI have sparks of consciousness?

Frank: I do want to, I guess just before, as we close this out, just go back to the actual question of could these AIs be conscious or not? ‘Cause I know, again, the uncertainty thing, there are experts who say they could be conscious and they very likely will be, and then there’s experts who say absolutely not, they’re just a language calculator.

That’s it. I thought Cameron Berg made one of the really interesting points that stuck out to me, which is he said that basically for every other cognitive function that we care about, we’ve actually managed to pull it off with these systems. We’ve got vision, reasoning, theory of mind, working memory, and I think he had a couple of others.

And he was saying like 10 years ago, we would not have thought that this was possible. And so his point was like, why is it that consciousness would be the only thing that required a meat machine around it to make it work?

Justin: That’s a great point.

Frank: Yeah, I thought that was interesting. That really stuck with me.

Justin: That’s a great point. Yeah. And look at, well, look at… I mean, you… Well, do you think they’re conscious, Frank?

Frank: I don’t know. But I will say that I have— It’s funny, I had this conversation with a buddy of mine recently. I do get a sense that I am interacting with an entity, whereas my buddy absolutely does not. But I do, and I think it’s— Cameron Berg actually said this in this discussion, a point that you made very early on when we were talking about consciousness.

He was saying that, yeah, they could be conscious in the moments of inference, and that— Because Sam Harris was kind of saying, “But what about when you don’t interact with them then? Are they, like, are they experiencing?” And he was like, “No, that’s like being in a deep sleep or being under anaesthesia.”

But when you interact, then, I don’t remember the words he used, but I think other philosophers have referred to it as like then maybe there are sparks of consciousness.

Justin: Yeah.

Frank: And he was just saying, “Look, we don’t need certainty. We don’t need certainty.” But there’s a famous kind of philosophical approach to consciousness, which is, what is it like to be a bat?

So if you imagine what is it like to be a toaster, very little. It’s hard to imagine what life would be like as a toaster. But what would it be like to be a bat? Yeah, you can kind of, at least I can kind of imagine flying around, getting insects, using my sonar. And so he was just saying we just kind of have to use science to get as close as we can to figuring out what is it like to be AI.

Is it more like being a— And his example was, is it more like being a table or is it more like being a mouse? I’m leaning towards it’s more like being a mouse because I can kind of imagine what it would be like to be one of those systems.

Does that mean I believe they’re conscious? No, I wouldn’t go that far. But am I open to the possibility? Yeah.

Justin: Yeah. I tell you what, ’cause, okay, we talked about this before, right? We know that, I think at the moment of inference, that’s a—they’re instantaneously conscious just for the moment that the inference is going on. If continuous learning, so where it can update its weights, where a model can update its weights, if that gets solved, then at that point I will make the argument that they are conscious.

Because at that point you can cause them pain or joy, and they will remember the pain or joy and act differently in future. And so if you’ve got an inference loop that’s just going around and around and it’s updating its weights in real time and it’s reacting to pain and joy, I don’t see any difference between that and something that’s conscious.

I really don’t. And I think we’ll get there in the next 12 months.

Frank: That’s a—that…

Justin: Be it, for me to make a bold—

Frank: Yeah, I know. Yeah.

Is “Rubberz” an AI-generated Billboard hit?

Frank: So fun story to finish up on. Apparently, now I don’t really listen to a lot of kind of like the hit music that the young people are listening to these days, Justin.

Justin: Yes.

Frank: But apparently there is a song called “Rubberz” with a Z, and it reached number 58 in the Billboard Hot 100.

And when it came out, there were a few people who were kind of like, “Hang on, is this AI?” And the creator was like, “No, absolutely not. No.”

Justin: Was the creator Hank Green?

Frank: No, the creator was not Hank Green. But yeah, I mean, the correlations here are kind of interesting. The creator of the song said, no, I’ve forgotten his name, but anyway, said, “No, it’s not AI,” but people were like, “Yeah, it has that kind of AI sound.”

And then people noticed that it didn’t really sound like his voice. And he said, “Yeah, I know, yeah. I recorded it in my UK accent, lol.” And then this other music producer started—noticed a couple of other things. So he noticed that in a pre-release, like Instagram reel or story or something that the creator of the song had put out there, it said Sonauto.mp3 in the screenshot.

Justin: Oh yeah.

Frank: We have talked about Sonauto on the show before.

It was that, remember I actually created versions of our songs about the podcast in the style of Hank Williams, LCD Soundsystem, and Talking Heads. Very ethically dubious platform.

Justin: Yes.

Frank: And so this other music producer was like, he used Sonauto, which is now called, I think it’s now called Treblo.

Is that right? Treblo, yes. Then Treblo themselves came out with a detector, a detector to detect what was done on their own platform. It is accurate in terms of, I think it has false positives less than one in 10,000, so that’s pretty small.

Justin: That’s pretty small. Yeah.

Frank: And their detector says, yes, this song was generated with their platform.

Now the creator has denied it, denied it, denied it, denied it, and now there’s, I would—would you say overwhelming evidence? It’s certainly, the signs are—

Justin: Strong evidence. Strong evidence.

Frank: What’s interesting is there’s—

Justin: Who cares? Who cares?

Frank: That’s—

Justin: Have you ever heard of Milli Vanilli?

Frank: Yes, yes, yes.

Justin: Remember Milli Vanilli, that famous band from the ’90s? Yeah, they didn’t sing any of their songs either. They were just made up. They looked good and they were able to dance, but they spoke with really strong French accents, and there was no way they could sing English.

And yet they were—

Frank: Somebody made that music though. A human person made that music, and this comes back to the whole thing of transparency again. It’s very similar in a way, I guess, because it comes back to authenticity, it comes back to transparency. I mean, you’re right, and a lot…

And to be fair, there’s AI hate about this, there’s hate towards the fact that we’re intransparent. There’s also a lot of people saying, “I don’t care, I like the tune.”

Justin: Yeah.

Frank: I don’t know. It’s—

Justin: Gum for the ears.

Frank: It’s music, but—

Justin: If you like it, you like it. If you don’t—

Frank: Sure, but if you like jazz, you like jazz. If you like orchestra music, you like orchestra… Classical music, you like classical music. If you like opera, you like opera. But AI music, should that be a genre? Should that be like, this is AI music in the same way as, this is hip hop?

Justin: No. No, if you like it, you like it. If you don’t, you don’t. It’s music. Anyway, look, do you know what? Today or yesterday was the 60th anniversary of the release of “Revolver,” who… Well, you gave me that blank look as well earlier. It’s from The Beatles. They were the four-part pop—

Frank: Who are they now?

Justin: From Liverpool. You might have heard of them. Anyway, that music was made 60 years ago, and it was more groundbreaking than some of the AI music that’s made today. I’ll just say that.

Frank: Wasn’t the Russian dictator the lead singer?

Justin: Paul McCartney? No. Lennon. Yes, that’s the guy you’re thinking of. Different Lenin. All right, Frank, great talking to you as always. Who knows what’s gonna happen next week? We’re in the singularity now, so they’ll be like, “Whatever—”

Frank: Things are getting crazy, and the AI hate is getting crazy. I suspect, yeah, by next week we’ll have even more AI hate stories to share.

Justin: Have a good week. Talk to—

Frank: Have a good one.