Why the AI Consciousness Debate Hasn't Moved in Forty Years

In this episode I discuss why even the most well-known scientific minds keep talking past each other when it comes to the "Will AI Ever Be Conscious" debate. I argue that the answer isn't that the question is too hard., it's that the question is being asked in a vocabulary that prevents progress.

Become a Member at dekyon.io/seanmcclure/science-in-perspective

If you enjoyed this episode and want to go deeper, come find me over at dekyon.io. With a premium membership you get the full Science in Perspective experience: interactive visualizations of the key concepts, a study space to learn the fundamentals behind each episode, and an "Ask an Episode" feature where you can dialogue with an AI about the details and start connecting ideas across the whole show.

There's also notes, transcripts, deep-dive articles, and a community of folks thinking through this stuff alongside you. It's where the episode stops being something you just listened to and becomes something you truly understand. So head to dekyon.io, click on the science in perspective app, and become a member today.

Transcript

Today's episode is about one of the loudest scientific debates of the moment, whether artificial intelligence is or could ever be conscious. You've probably noticed that confident voices on both sides of this are everywhere, and that the debate hasn't actually moved in decades, even as the technology has obviously transformed beyond recognition. That stasis is the news, the fact that it's not moving. When a debate doesn't move, the issue usually isn't that the question is too hard, it's that the question is being asked in a way that prevents progress. So this episode is about what's gone wrong in the conversation, and why even our most credentialed scientists keep talking past each other, and what a shared vocabulary, one drawn from how nature, biology, and even economics already handle this kind of question, how that shared vocabulary would actually change the situation. I'll get into that in a bit. We're not gonna resolve whether AI is conscious in this episode, obviously, but we will make the question askable in a way the public discourse currently does not. So let's get into it. So- Look, why are people so confident when they say things like, "Well, AI isn't conscious," or, "AI could never be," right? Why does this particular dismissal feel so good to say? Well, one, there's a kind of status threat here, right? If AI can think, then thinking isn't what makes us special. People who have built their identity, career, or sense of self-worth around being a thinker have a personal stake in the answer. That's why you get such heat in the dismissals. They're not really about AI, they're about the human asking, "What am I then?" So this is the most psychologically real reason, but it's also the one that makes for the worst public arguments because nobody admits this is what they're doing, right? There's also this kind of category panic, right? We've spent our whole lives knowing what kinds of things think and what kinds of things do not, right? Rocks don't think, dogs sort of do, people definitely do. So AI breaks the sorting system. People hate when a sorting system breaks because a system was doing invisible work telling them how to treat things. So regardless what you believe, people are doing more emotional work than intellectual work when it comes to this debate. It's good, important that you see that, right? We're removing a floor that is uncomfortable here. But the real problem that I'm gonna indent- identify in this episode is that the reason AI consciousness debate has been so loud and so unsatisfying isn't actually that the question is too hard, even though that's how it often gets framed. Well, it's just a really hard question, right? And this is why scientists are debating. It's not that the question is too hard, it's that the question is being asked in the wrong vocabulary. Okay, so both sides, the dismissers and the believers, as per AI being conscious, they're arguing about minds as if minds were deterministic rule followers, right? They're not. Brains are not. LLMs are not, the large language models, right? Both are complex emergent systems, and there is a well-established science about what such systems do and how they differ from rule followers. We're gonna get into those differences to make sure that You understand and, and can see what this debate currently is, and more to the point, what it should be. So the disagreements I will show you in this episode are not the disagreements of a working debate. They're the disagreements of a debate whose structure has collapsed, I will argue. Let's get into some definitions, right? Before we walk into kind of the debate proper, you, uh, you should have kind of a toolkit of definitions to make sure that we agree, or at least that you understand how I'm using these words. There's five terms that the rest of this episode will really lean on, and, uh, I'll try to define them as cleanly as I can ima- uh, as I can manage. So none of these definitions themselves are gonna settle anything. They're just the minimum scaffolding you need to hear the arguments without getting lost in them, and to recognize when someone is using one of these words to mean something they haven't actually said out loud. So most of what's gone wrong in the publ- uh, in this public debate that we, we see now, um, about AI traces back to one of these terms typically being used loosely enough that two speakers thought they were arguing when they were actually kinda just defining things differently, right? Anyway, okay. So here are some words. Let's get some definitions down. Consciousness, right? Like, what is that? What's the definition? This is the disputed thing, right? And we should be honest about that. Consciousness is the word for whatever it is that makes there be something like you or me, right? When you taste coffee, when you see red, when you feel pain, there's this kind of experiential quality to those moments, often called qualia. Uh, consciousness, I guess we could say at a minimum refers to that. Of course, the trouble is we have no agreed scientific definition of it, no agreed test for it, and no agreed theory of how physical processes produce it, right? This is called the hard problem, and it, uh, quite frankly, has been hard for three centuries. So consciousness, you know what I just said there might kinda sound a little bit hand-wavy, but that's the point. We don't really have a solid definition of it 'cause we don't really know what it is in the way that we typically say know or understand, right? We can't give it that reduced, reductionist, clean, causal, mechanistic definition. We just can't. It's something different, and so that's why it becomes debated about what it even is, right? What about understanding and intelligence, right? It's related to consciousness, but it's not the same thing. Understanding refers to grasping meaning, knowing what something is about, what it implies, uh, how it connects to other things, right? Intelligence refers to solving problems, handling novelty, adapting to circumstances, finding paths to goals. Neither of these is the same as consciousness, right? A system could, in principle, uh, understand or be intelligent without being conscious, and the reverse is also imaginable. Then there's the word emergence. Super, super critical, right? It, it's actually a precise scientific concept. It's not a vague gesture. Emergence refers to properties or behaviors that appear at the level of a system as a whole, but cannot be found in any individual component. This is super, super critical. Temperature is emergent. A single molecule doesn't have one. A tornado is emergent. A single air molecule isn't doing what the tornado is doing. A market price is emergent. No individual transaction sets it. So emergence, even though this word tends to get kinda tossed around loosely, it's not mysticism and it's not magic. It's a regular feature of how complex systems work, and there's serious mathematics behind it. So let's be clear about that. Then there's level of description, okay? This is probably the most important thing I want you to understand. This is really the concept arguably that this whole episode hinges on. Any system can be described at multiple levels. So a gas can be described at the level of, uh, individual molecules or at the level of bulk properties like pressure and temperature, right? A brain can be described at the level of ion channels or neurons or networks or full-blown behavior. Each level has its own vocabulary and its own laws, and some properties only exist at certain levels. So asking where in a single molecule is the temperature is malformed. Doesn't make any sense. Temperature isn't a property that exists at that level, right? Asking the right questions about a system requires getting the level right. Asking the wrong questions at the wrong level produces confusion that can persist indefinitely, hence the problem here. So this will turn out to be really a, a, I guess you could say, a central diagnostic tool that we're gonna need throughout this episode for you to understand the situation. Okay. Finally, um, substrate, right? What something is made of and why people quietly think it matters. Substrate refers to the physical material a system is built from, right? So brains run on biological neurons. Computers run on silicon, right? The questions of whether substrate matters for consciousness or understanding, like whether you have to be made of the right stuff or whether being made of any stuff that does the right thing is enough, that's one of the oldest in philosophy of mind, and most people have an unexamined intuition that substrate does matter, that biology is somehow special, right? As you will see, right, this special intuition turns out to be doing a lot of unacknowledged work in the public arguments against AI consciousness. So we'll need to be aware of that. Okay, so we got top scientists who are disagreeing, and obviously that's normal. That's actually good. That's what you would expect. Let's go through what some of these big names are saying, and then I'll, I'll come back to this idea that actually it's not just scientists debating. There's something structurally off here. There's something malformed about the way the conversation is happening. People aren't speaking the same vocabulary. So we got Roger Penrose, one of the most decorated mathematical physicists alive, right? He says computers can never be conscious because computation can only follow rules. Sounds logical, right? Roger Penrose has actually argued for decades that no amount of computation, doesn't matter, no, no scaling of GPUs, no architectural cleverness, whatever you wanna say, uh, none of it's gonna rep- uh, produce consciousness or genuine understanding. His view is that something happens in the brain, it might be quantum mechanical, possibly something we haven't identified yet, something that computation cannot replicate in principle, right? So whatever you think of his specific quantum hypothesis, the core move is the one we actually keep seeing. Computation is rule following. Rule following can't produce awareness, therefore, computers can't be aware. It's a confident position from a brilliant mind, and as you'll see by the end of this episode, that it doesn't quite fit the thing it's arguing about. Stay with me. I'll, I'll tell you what I mean by that. But let's do some more names. John Searle, this is one of the most cited philosophers of the last fifty years, kinda says the same thing but in different words as Penrose, right? Y- you know, basically, symbol manipulation can't produce understanding, no matter how good the symbols get, right? So John Searle's famous thought experiment, right, imagines a man in a room shuffling Chinese symbols according to some rule book, and he produces responses that fool the native speakers without actually understanding a word. So go look up, you know, the Chinese room thought experiment, John Searle. You'll, you'll get more details on that. But Searle's claim is that this is what computers are doing always, no matter how sophisticated they get. There's convincing behavior, but there's no understanding. So forty years of philosophers have published responses. Searle has published responses to the responses, right? The argument refuses to die and refuses to resolve. Now, you as a listener need to notice something. A thought experiment from 1980 is still the central reference point in 2026. That's not really the sign of a healthy debate. Something's off there, right? There's-- nothing is moving, right? So we'll get back to that in a bit. Now let's do a few more names. Stephen Wolfram, this is gonna be the other side of the argument. A physicist, computer scientist, right? Probably with as deep a grasp of computation as anyone living. He says, "LLMs," again, that's the large language models, right, the technology underlying what we call AI today, "have basically put the final nail in the coffin of the idea that consciousness is something extra beyond physics." Right? So Wolfram is looking at the same evidence, draws the opposite conclusion. To him, LLMs have demonstrated that you don't need anything beyond ordinary physics and computation to produce the kinds of behaviors we associate with mind, right? Every time someone said, "But it can't do X," LLMs did X. We see this again and again and again. We see this today. They just had a proof come out in mathematics, right? Uh, related to Erdos problems and all this kind of stuff. People keep saying, "Well, that's not gonna happen. That's not gonna..." It keeps happening, right? The accumulating pattern for Wolfram in this specific case is that something extra, that something extra view of consciousness that humans supposedly have, it keeps losing predictions, and the computational view keeps making them. Right. So, so here is a thinker, Stephen Wolfram, of arguably comparable stature to Penrose, a very, very accomplished, and he's looking at the same field, and he's concluding nearly the opposite. Okay. Another one, Richard Dawkins. Again, the other side here. Richard Dawkins has spent his career thinking about what intelligence is and how it evolves, and he says that the line between evolved and engineered intelligence may be much thinner than our intuition suggests, right? I mean, Dawkins has noted that our intuition about the gap between evolved intelligence and engineered intelligence is weaker than we, we feel it to be, right? Evolution didn't design brains to be conscious, didn't design them all, at all, but consciousness emerged from a process that was just selecting for behaviors that worked. Okay? So if that's how it happened in us, then the question of whether it could happen in a different substrate, right, by a different optimization process is genuinely open. Of course, it is, right? Now, he doesn't claim, Dawkins, that AI is conscious. He's just saying our confidence that it can't be is poorly grounded. And, and of course, of course, right? Everything that's-- If you look at Penrose If you look at the Searle argument, it's all poor grounding because it actually misrepresents what computation is. Old school traditional computation is of course rules-based, but that is not the definition of computation. It's most definitely not the computation that's happening in artificial intelligence. In fact, artificial intelligence is the literal antithesis of rules-based programming. It's literally why it got created the way it did, 'cause rules-based programming couldn't solve the categorically hard problems, right? Okay. Now, so, so the people that we're lo- we're looking at there, I mean, these are not cranks. These are the top of the field, and they are not converging. They are not even close to converging. Again, there's... It's one thing if, you know, you got scientists and they're disagreeing, but you kinda get some consensus and convergence, and things are at least evolving. This is not even progressing for decades. When the best people in the field can't agree, something is wrong with the field, not the people. So credentials aren't ending this, okay, or even moving this. Forget ending this. It's not even moving it, right? Uh, and again, that's not because these are bad thinkers, it's because the debate is malformed in a way that more credentials can't fix. There was something structurally wrong with how this entire conversation is being conducted, and adding more famous names to either side is not going to fix it. So what will fix it? That's kind of what I'm here to put the opinion for and to s- and, and to talk about, right? Um, so like a malformed debate, it, it looks like a debate from the outside, right? But it, it doesn't function as one. You got both sides talking. Both sides are citing the evidence. Both sides have credentialed defenders. Looks all good. But the things they take for granted are so different that critique can-- uh, that, that a critique cannot actually land. So when Searle says no understanding, he means something that Wolfren doesn't e- Wolfram doesn't even think is a coherent category. And when Wolfram says LLMs put the final nail in, right, he means something that Searle doesn't think is the question even being asked. So they're speaking past each other, not out of bad faith, right, but out of structural incompatibility. They're using different vocabularies, different ontologies, different criteria for what would even count as evidence, right? So the exchange has the grammar of debate without the function of debate, okay? Let's, let's use an example in politics 'cause we can all relate to this one, right? Especially in the last ten years. Political polarization is, is kind of the everyday case of this. You've got two sides. Let's say they both watch the same press conference, and then they, they, they disagree not about what was said, but, uh, you know, about what kind of thing was said, right? That's the difference. So for, for example, one side hears policy and the other hears performance. One side hears reasonable concern, and the other hears coded language, right? They're, they're not weighing the same evidence differently. They're seeing different events. There is no neutral ground from which one side's critique can land on the other. That's the problem, right? Because critique requires shared premises, and there are none. Okay? So that's why so many political debates between the most polarized factions just don't move. There's no progress there. That's why polarization is always so bad. There's no progress. There's no movement. It, it's not because the participants are stupid. It's because the structure of the exchange has broken down beneath them, okay? So let's go back. You know, the AI consciousness debate, it has the same pathology, right? So I want in this episode to change what you think you're witnessing in the AI consciousness debate. Th- this is not about, well, oh, experts disagree about a hard question. No, that, that's not where we're at. It'd be great if it was. What you're actually witnessing is experts can no longer have a conversation because their vocabularies have diverged past the point of mutual critique, okay? In fact, C.S. Lewis made the same kind of point regarding morality in his book Mere Christianity, right, a while back. He, he was making it about moral disagreement, but the logical structure is universal. Disagreement is only a po- is, is only possible against a shared standard. If two people are arguing, they only can truly argue in the proper sense of that word because at some level they agree about what wrong and right is, or that there is a wrong and there is a right. Like at some level there has to be an agreement in order for a true disagreement to take place, right? Without one, without a shared standard, you know, what looks like argument is actually just, you know, I don't... two monologues happening in the same room kind of thing, right? So Let's get-- Let's, let's peel back the layers a bit on this. A formal system can be described at multiple levels, okay? And I really wanna be clear about this. Each level has its own vocabulary, its own primitive entities, its own laws, uh, its own properties. A property defined at one level is generally not definable at another level, not because we haven't figured out how, but because the property only makes sense given the entities at that level. This is something super critical to understand, and way too many people miss this. Take a tornado. Okay, yes, it has molecules, but the tornado is not the molecules. It's something different. That something solves a problem molecules themselves cannot solve, right? Levels of description are not arbitrary conveniences, 'cause this is a common pushback you might get, right? They're not just arbitrary conveniences, right? They are determined by the structure of the problems being solved. Okay, so a higher level exists wherever a group of lower level components produces a pattern that solves a problem the components cannot solve individually. So a tornado is the pattern in a body of air that resolves a pressure differential. A cell is the pattern in a collection of organelles that maintains itself against entropy. A market price is the pattern in a population of transactions that aggregates dispersed information about scarcity. Each of these is a real, causally efficacious higher level, right? Each exists because of the problem it solves, and each is invisible if you insist on looking only at the components, right? Not because the level isn't there, but because the level is defined by the problem, and individual components are not at the scale where the problem is being solved. Okay, so this is what's happening with consciousness, understanding, and intelligence, right? They are candidate solutions at the system level, so asking where in the components they are is just asking the wrong question by construction. Okay? Again, back to that tornado example 'cause I really wanna be clear about this. So a tornado is the specific organized pattern in the body of air that solves the problem of equalizing pressure differentials between different, uh, elevations or regions, right? So, okay, well, without the pressure differential, there's no tornado, right? The molecules go on bouncing around just doing other things, whatever they do. You add the differential, and the molecules organize, self-organize into a tornado. Not because any molecule wants to, obviously, but because the tornado is the form of motion that resolves the imbalance. The molecules are still molecules, but the tornado is real. It is the level at which the problem gets solved, okay? Again, levels are not just observer-imposed, right? They are real features of the dynamics. A sophisticated dismisser might say, right, "Oh, levels are just stories we tell," right? That, that the real physics is at the bottom. No. Uh, and, and, and this problem-solving frame that I'm putting forward here, it answers this directly. Tornadoes are not stories. They do work in the world. They lift houses. Ask anyone who's been affected by them. The pressure differential is real. The tornado is real. The resolution is real. Whether or not anyone is watching, whatever. The, the air organizes itself into the form that solves the problem. Levels are not a human convenience. They are where the dynamics live, okay? So back to consciousness and understanding. So whatever consciousness is, okay, 'cause we can debate about the right definition. Here's what we can say. It is the system-level pattern that solves problems of perception, integration, response, and self-regulation. That we do know, right? So whatever understanding it is, is, it is the system-level pattern that solves problems of meaning tracking, context handling, right, coherent response. These are real problems at the system scale, and the patterns that solve them are at the same scale. Asking where in a single neuron's firing the consciousness is or where in a single matrix multiplication an understanding is, right? I-is, is asking where in a single air molecule the tornado is, right? Doesn't make sense. The components are not at the scale where the problem is being solved. By construction, they cannot host the solution. Okay? These levels that I'm talking about, right, which, which appear in all phenomena, right? They are constituted by the problems they solve, and solutions cannot be found at scales smaller than the problem, okay? So the shared vocabulary we need in this is AI consciousness or can it ever be conscious in this debate is exactly this. It's not emergence in the loose hand-wavy sense everyone tends to use it. It's not complexity as a vibe, right? It's problem-solving at the appropriate scale. Okay. So we can make, you know, you're wrong meaningful again, okay? So under this vocabulary, the dismisser can no longer say, "Well, it's just symbol manipulation," because that move is now visibly a category error. Pointing at the components instead of asking where, uh, or whether the system-level pattern solves the relevant problem, right, that's an issue. Doesn't make sense for you to point there. You can s-- The believer can no longer say, "Well, look how impressive it is," as if impressiveness, you know, settle, settles anything. They have to specify which problems the system is actually solving and at which scale, and show that those are the problems whose solutions constitute understanding. That's a real argument, right? That's accessible, okay? So this makes the consciousness question empirically tractable in principle, actually, right? Whether a given system solves a given problem at a, at a, at a given scale is something we can actually investigate. We can specify the problem, look at the system's behavior, and ask whether the behavior constitutes a solution. I'm not saying this is easy, and there will be hard cases, but it's not malformed. It's like a genuine research program, right? So that's what a shared vocabulary buys you, right? Not consensus, not an end to disagreement. A debate where the disagreement can produce progress, right? So both sides now have some homework. All right. That's it for this episode. Thanks everyone for listening. And for those of you who are watching, if you haven't already, go ahead and check out scienceinperspective.com, science-in-perspective.com opens up an app. You can become a member, $9.99 a month, and you get access to all kinds of things, not just the transcripts, data visualizations of the concepts I talk about, um, uh, all kinds of breakdowns. There's a study space. You can spend time with these concepts, learn more about them, even, uh, ask, uh, ask an episode feature, which I just recently added. So you can ask AI. Uh, this AI is specifically aware about all the episodes that I, uh, have added to Science in Perspective, so you can get cross-episode connections, or you can just dig into one episode, ask whatever questions, all kinds of learning material on there so you can become better at understanding the science behind, uh, the different topics that I talk about. Okay, thanks again everyone for listening, and for those of you who are watching, until next time. Take care.