We Took Inert Sand and We Made It Think. Now What?

📖 Read: https://helioxpodcast.substack.com/publish/post/209700052

Radical Optionality

There is a particular kind of vertigo that comes from realizing the ground beneath you has started moving faster than you can track it. Not metaphorical ground — actual sand. Silicon, purified and structured and coaxed, against every intuition we have about dead matter, into something that can reason. Demis Hassabis calls this the miracle at the center of our age: humanity found a way to make sand think. It is a sentence you can turn over in your hands for a long time before it stops sounding impossible.

Here is the part nobody wants to sit with: the debate over whether advanced AI is coming has quietly ended. Not with a vote, not with a headline, but with the slow accumulation of evidence that made the argument moot. What remains is a much harder, much more interesting question — not if, but how do we behave now that the ground is moving. And this is where the story gets genuinely hopeful, if you're willing to follow it past the vertigo.

And maybe the most hopeful thing in the whole story: even in the best-case future, we're still the ones who get to decide what happens next.

Radical Optionality — Governing Transformative AI Under Uncertainty
and three other references

This is Heliox: Where Evidence Meets Empathy

Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter.  Breathe Easy, we go deep and lightly surface the big ideas.

Transcript

0:25

You know, usually when you stand on a beach and you look down at a handful of sand, there's this expectation of just like absolute inertness. Right. Just dead matter. Exactly. It's just ground up rocks, crushed shells, maybe some silica. You might pack it into a bucket, beaver sandcastle, wash the tide, wash it away. And that's the end of the story. I mean, it is the definition of mundane. It is. Yeah. It's foundational to the planet, obviously. Yeah. but it's completely passive. It just sits there until, you know, an external force acts upon it. But then you step into the world of modern computing and suddenly that exact same handful of sand becomes, honestly, the most complex dynamic substance in the known universe. It's wild to think about it that way. It really is. Because at its core, silicon is just highly purified sand. So we take this passive dirt, we run electricity through it, and we somehow like coax cognitive capabilities out of it. As Demis Asabas, who is a Nobel laureate and the CEO of Google DeepMind, as he put it recently, human beings have essentially found a way to make sand think. Make sand think. It's just, it's a miraculous reframing of the whole technological age. I mean, when you put it like that. We are taking the earth itself, structuring it at this microscopic level, and we're waking it up. And the speed at which this waking process is happening is, well, it's what brings us here today. Which is a perfect segue. So welcome to today's deep dive. We are looking at two... Very distinct roadmaps for surviving this exact miracle and honestly the absolute chaos it is currently bringing to our human system. Friction is just incredible right now. Oh, it's massive. And one of those roadmaps comes directly from the man who helped turn that sand into a mind. Demis Hassabis, via a July 2026 manifesto he published. It's titled, A Framework for Frontier AI and the Dawning of a New Age. A very ambitious title. Very. And we're layering that with a massive, incredibly detailed policy playbook from the Institute for Law and AI, which focuses on a survival strategy they call radical optionality. Right. And both of these texts, they are trying to grapple with a rapidly closing window of time. They aren't looking backwards at what we've built over the last decade. Yeah, they're looking forward. Exactly. They're looking at a fast approaching event horizon. And to be clear with you, the listener, the goal of this deep dive isn't to sit here and debate if Advanced Artificial General Intelligence or AGI is coming. No, that ship has sailed. Yeah. According to our sources, that debate is just over. The train has left the station. So, our mission today is to explore the incredible dilemma of how human governments and, well, societies, can possibly govern a technology that is evolving exponentially faster than our ability to write laws. Which is no small feat. No, it's terrifying. We're going to travel from the foothills of this new reality through the flawed ways we usually try to control technology, all the way into the nuts and bolts of how this radical optionality thing actually functions in practice. Because the gap between how fast AI is moving and how slow our institutions move, it isn't just like a bureaucratic annoyance anymore. Right. It is quite literally the defining challenge of the 21st century. I want to start with the sheer scale and timeline that Hassadis lays out because – Frankly, reading it gave me a bit of vertigo. Oh, I totally agree. He states that AGI, which we generally define as a system exhibiting all the cognitive capabilities of the human brain, is, quote, probably only a few short years away. he actually believes we are standing in the foothills of the singularity right now and notice the historical comparisons he reaches for to anchor that claim it's telling yeah he doesn't compare it to the internet no katabas doesn't compare the arrival of agi to the invention of the internet or the printing press or even the smartphone he states it is much more akin to the discovery of electricity wow or the taming of fire fire i mean he predicts the impact will be 10 times the magnitude of the Industrial Revolution happening at 10 times the speed. Which creates this terrifying and exhilarating duality for society to process, right? On one hand, you have the promise of immense, almost unfathomable abundance. I mean, Hassabis paints a picture of accelerated drug discovery, where we are simulating protein folding in seconds. Which they're already doing with AlphaFold, right? Exactly. And he's projecting that out to the development of novel clean energy sources like commercially viable fusion and the creation of advanced metamaterials. He's talking about crossing the threshold into post-scarcity. Where material resources just aren't the limiting factor for human progress anymore. Precisely. But right on the heels of that utopian vision, he brings the hammer down on the risks and and they are equally massive. Unprecedented risks, really. We are looking at catastrophic cybersecurity vulnerabilities, the democratization of biological and nuclear threats, and this overarching danger of increasingly agentic systems. Let's pause on that word for a second. Agentic. Yeah, that's a crucial distinction. Because we aren't talking about a chatbot that just passively answers your trivia questions, right? No, not at all. An agentic system is an AI that can be given a high-level goal. And it will autonomously go out into the digital world, break that goal down into steps, write its own code, interact with other software, and execute the task completely without human supervision. And when you have agentic systems that are recursively self-improving... the risks compound exponentially. Which brings us right to the core of the radical optionality paper, doesn't it? It does. It brings us to the pacing problem. The pacing problem. Okay, so this is the idea that technological progress is just lapping the legislative process, right? It's lapping it, lapping it again, and then changing the rules of the race before the government has even tied its shoes. Yeah, that's a great way to put it. The pacing problem is a known concept in technology law. We've seen it with social media and privacy. But the paper argues that AI takes it to an entirely new, unrecognizable level. The fundamental issue is that our laws are effectively obsolete before the ink is even dry. And the paper points out this massive psychological blind spot we all share when trying to comprehend this. which is that human beings are just notoriously biologically terrible at grasping exponential growth. Oh, we are awful at it. We naturally think in straight, linear lines. We truly do. We look at the past five years to predict the next five years. And the paper uses a couple of brilliant historical examples to illustrate how badly this fails. Like, look at the early days of the COVID-19 pandemic. Oh, man, man. In early 2020, you had highly trained, world-class epidemiologists looking at the initial case numbers and dismissing the virus as being, you know, less prevalent than the seasonal flu. Right, I remember that. They completely failed to intuitively grasp how an exponential infection rate would compound globally in a matter of weeks. Or the example they give about the International Energy Agency. Yes, the solar energy one. This one blew my mind. The paper notes that for over a decade, the IEA systematically underestimated the growth of solar power. Every single year, the IEA would publish a report predicting that the solar industry would just level off. And every single year, solar maintained roughly a 25% annual growth rate. The experts literally refused to believe the curve was going to keep going up that steeply. They kept trying to bend the projection back into a flat, linear reality. Now, apply that cognitive blind spot to AI. Okay. The exponential curve we are writing right now isn't just about faster silicon processors or feeding more data into the machine. It's about the fact that AI is now actively participating in its own creation. Wait, say more about that. Well, the paper highlights this stunning data point. Engineers at Frontier AI Labs are reporting that a significant majority of next-generation AI code is being written by current-generation AI systems. Oh, wow. So this is that recursive self-improvement thing we mentioned. I really want to unpack this because it feels like the engine driving the entire crisis here. It absolutely is. Because if an AI system is getting better at writing the code to make a smarter AI system, the timeline doesn't just move faster, it collapses inward. I was trying to picture the mechanics of this, and the best analogy I could come up with is like a factory. Imagine a factory that builds robots. Okay, I'm with you. But these robots are designed to be slightly faster and smarter than the generation before them. So a standard manufacturing upgrade cycle. Right. But the cycle doesn't take a year. The new smarter robots don't just get boxed up and shipped to a warehouse. They immediately turn around and redesign the factory itself to build even faster and smarter robots. And they do it instantly. Exactly. They finish that redesign by lunchtime. By dinner, you have a factory that is producing technology utterly incomprehensible to the human manager who turned on the lights that way. That captures the terrifying velocity of recursive self-improvement perfectly. It really explains why the timeline to superintelligence could theoretically shrink from, you know, a matter of decades down to a matter of months or even weeks. Weeks? That is insane. But mathematically possible. Once the system crosses a certain threshold of capability, human engineering becomes the bottleneck. So the AI just removes it. Exactly. The AI removes the human bottleneck. Which, I mean, forces me to push back on this whole premise. If we are looking at an event that is ten times the magnitude of the Industrial Revolution, and it could theoretically happen in a time frame measured in months, are we seriously expecting human institutions to manage this? It's a huge ask. I mean, it takes a local government a decade of environmental reviews and budget hearings just to fix a pothole or build a tiny bridge, and, How can a sprawling, slow bureaucracy possibly comprehend, let alone regulate, a recursive intelligence explosion? The authors of the radical optionality paper entirely agree with your skepticism. They acknowledge that traditional governance is fundamentally incompatible with the reality you just described. It's just too slow. Too slow, too rigid, and way too prone to political gridlock. Which is why they map out the three potential paths we can take as a society to regulate this space— and they argue forcefully that the two paths we instinctively lean towards are deeply structurally flawed. Okay, we need to take a quick break here, but when we come back, we'll walk through those three paths and figure out why the obvious answers might just be traps. Stick around. Welcome back to the Deep Dive. We're exploring the collision course between rapidly advancing artificial general intelligence and our extremely slow-moving legal system. That was a rough collision. Before the break, we established that human institutions are entirely outmatched by the speed of recursive self-improvement. The Institute for Law and AI's paper outlines three paths governments might take to deal with this. Let's walk through them. Sure. Path one is essentially the libertarian techno-optimist approach. They call it, let the market handle it. Right. And this is a highly prevalent view in Silicon Valley and among certain policymakers. The paper mentions figures like Adam Theer in the U.S. and Mario Draghi's competitiveness review in the EU who champion this sort of ideology. And the central argument is basically that overregulation suffocates innovation. Right, exactly. And their historical precedent usually points straight back to the nuclear era. It does, almost every time. The techno-optimists argue that our societal overreaction and heavy-handed regulation of the nuclear energy sector effectively killed a technology that could have provided the world with cheap, abundant, clean power for the last 50 years. We regulated it out of fear. Right. We regulated it out of fear, and as a result, we remained dependent on fossil fuels. So they argue that preemptively regulating AI out of a fear of science fiction scenarios will cost us the immense post-scarcity benefits Hassab has talked about. We will literally regulate away the cure for cancer. The logic makes sense on the surface. You don't want to strangle a golden goose in its crib. Of course not. But the paper points out a pretty massive flaw in applying the nuclear energy analogy to advanced AI. Yeah. The fatal flaw in Path 1 is misunderstanding the nature of the technology itself. Transformative AI is not just a commercial product like a better solar panel or a new pharmaceutical drug. It's a dual-use technology. Exactly. Yeah. It has severe national security implications. The paper draws the line very clearly. You might successfully argue for deregulating nuclear energy, but no rational person argues that the free market should dictate the private ownership of nuclear weapons. Because if an AI model can autonomously engineer a novel pathogen, or completely dismantle the cyber defenses of the U.S. electrical grid, you can't just cross your fingers and trust the invisible hand of the market to make sure it doesn't fall into the wrong hands. No, laissez-faire doesn't work when we are talking about digital weapons of mass destruction sitting in a corporate server bar. So if path one is out, the natural instinct for any government is to swing wildly in the opposite direction, right? Just lock it all down. Which brings us to path two, anticipatory governance and the precautionary principle. This is the opposite extreme. The precautionary principle suggests that we need strict, prescriptive, statutory regulations placed on the technology before it fully materializes. Okay, like what? Think of hard bands on certain types of AI research, capping the amount of computing power anyone can use, or implementing an FDA-style licensing regime where you are legally forbidden from building a model unless you can definitively prove beforehand that it is 100% safe. Which sounds incredibly comforting to the general public. Oh, it's very appealing politically. But the paper tears this approach apart as well, because predicting the specific path of an emerging technology is basically a fool's errand. Precisely. To govern anticipatorily, regulators have to accurately guess what the technology will do, how it will be used, and what form it will take. And history proves that governments are uniquely terrible at that kind of forecasting. The paper uses this incredibly telling example of the U.S. Audio Home Recording Act of 1992. Yes. It's a perfect example. This example is wild to me. So back in 1992, Congress was terrified that people were going to use cassette tapes and digital audio tapes to pirate music and destroy the recording industry. Right. Home taping is killing music. Yeah. So they spent years debating and ultimately passing this massive complex law to anticipatorily regulate the hardware of home taping equipment. They baked specific definitions of digital audio recording devices right into the statute. And almost the exact second the ink was dry on that law, the personal computer revolution took off. Oh, man, it's so funny in hindsight. CD-ROMs, MP3s, and peer-to-peer file sharing software like Napster completely bypassed the entire regulatory framework they had just spent years building. The law was instantly obsolete because they regulated the specific form that technology took in 1992 rather than the underlying capability. And if we do that with AI, we are cooked. Absolutely cooked. The paper points out that parts of the EU AI Act have some of these rigid, frozen definitions baked into them, which could become obsolete in months. But there is also a massive geopolitical reality here that makes Path 2 impossible, right? Yes, there is. we would merely be ensuring that the first AGI is built by an adversary who does not share democratic values. It is, from a national security perspective, geopolitical suicide. So we have a profound tension here. Path 1 says let it rip and ignores the existential risks of digital nukes. Path 2 says ban it all and assumes we can predict the unpredictable while simultaneously handing the future over to the people. It's a rock and a hard place. And this is where the authors introduce their middle way, path three, radical optionality. Radical optionality. I have to be honest, when I first read that term, I groaned. Too academic. Yeah, it sounds like a dressed up academic way of saying, let's kick the can down the road. It sounds like the authors are saying, we don't know what to do between path one and path two, so let's just keep our options open, muddle through, and do nothing for now. It is a very fair critique and one the authors anticipate. But the paper explicitly pushes back against that interpretation. Radical optionality is not about passive delay. It is not about muddling through. Okay, so what is it? It is a highly proactive, aggressive, and incredibly expensive strategy. The core idea is this. We consciously avoid passing heavy-handed, restrictive laws right now, but in exchange, we spend massive amounts of financial and political capital to rapidly build institutional capacity. I want to make sure I understand the mechanics of this. We aren't writing the strict laws yet, but we are building the machine that can write and enforce the laws the precise second we realize we need them. That is the essence of it, yes. You are aggressively investing in information channels, recruiting top-tier technical talent out of the private sector and into government, and establishing flexible legal authorities that can be activated at a moment's notice. So it's about readiness. Exactly. The premise is that you don't have to force a choice between safety and innovation today. provided you aggressively build the capacity to make a highly informed, rabid choice tomorrow, regardless of what unforeseen scenario unfolds. It's like a city realizing a massive fire is inevitable somewhere downtown, but they don't know which building is going to catch fire or what kind of chemical might be burning inside. That's a great way to think about it. Right. So instead of forcing every building to install the exact same rigid sprinkler system that might not even work on a chemical fire, the city spends a billion dollars building a state-of-the-art firehouse, hiring the best firefighters in the world, and laying down high-pressure water lines to every single block. Yes. They are preserving the option to fight any type of fire anywhere at any time. That analogy maps perfectly onto the specific, concrete proposals Demis Hassabis makes in his manifesto. If we move from the theory of radical optionality into the actual mechanics of the playbook, Hassabis advocates for the immediate creation of a U.S.-led standards body. And he specifically says it should be modeled after Feinera. Now, for anyone who doesn't spend their days in the weeds of the financial sector, FENRA is the Financial Industry Regulatory Authority. Right. It is a non-governmental self-regulatory organization that oversees broker-dealers in the United States. Congress authorized its creation, but it operates as a public-private partnership. Hassabis argues this new AI standards body needs to be federally overseen to have actual teeth, but heavily integrated with the private sector. The funding, he notes, would need to be substantial. Oh, substantial. We are talking billions. Yeah. And it would likely come primarily from the AI industry itself. much like how Wall Street funds fine a rock. I can see why that's necessary. I mean, how exactly is a non-governmental body supposed to enforce rules on a multi-trillion dollar tech giant? If you try to regulate Google or Microsoft on a standard GS-14 government bureaucrat salary, you will never attract the talent you need. The government simply cannot pay a machine learning engineer $5 million a year. No, they can't compete. But a massively funded public-private partnership might be able to compete for that talent. Furthermore, they need the literal compute resources, the massive server farms, to run large-scale penetration tests on these frontier models. Exactly. This standards body would be populated by world-class experts, and their primary responsibility would be establishing technical benchmarks to define what a frontier model even is. Right. If an AI lab builds a model that hits those thresholds, they become designated as a frontier lab. Initially, Hassabba suggests they would voluntarily share their models with the standards body for a rigorous safety review 30 days before release. Voluntarily at first? Yeah, but eventually this formalizes into a mandatory hurdle. You don't pass the evaluation. You don't get access to the U.S. market. But here's where the concept of radical optionality really flexes its muscle. The absolute necessity of flexible rules and definitions. Crucial. If you etch the definition of a frontier model into a stone tablet, you are doomed. And the paper gets an absolutely stunning case study of this from California. Oh, the SB 1047 story. Yes, the recently vetoed Senate Bill 1047. The story of SB 1047 is a master class in why statutory law fails in the face of exponential growth. The bill was attempting to capture only the largest, most potentially dangerous AI models. That makes sense. To do this, lawmakers defined a covered model using a financial and computational threshold. Specifically, it applied to any model that cost over $100 million to train and and used a specific astronomical amount of computing power. Which, if you look at the landscape in the summer of 2024 when the bill was gaining momentum, that seemed totally reasonable. It did. Only the massive tech giants, OpenAI, Google, Anthropic, could afford to throw $100 million at a single training run. It was a net designed to catch whales. But then, mere months later, in early 2025, the reality of the technology shifted overnight. The deep-seek moment. Yes. A Chinese startup released a model called DeepSeek R1. Utilizing new architectural techniques like mixture of experts and advanced knowledge distillation, this model achieved state-of-the-art reasoning capabilities, competing neck-and-neck with the most expensive models in the world. And they did it for how much? They executed the final training run for less than $6 million. It is the ultimate real-world manifestation of the pacing problem. A $6 million miracle completely bypassed a $100 million regulatory net before the law was even fully realized. If SB 147 had been signed into law, DeepSeek R1, despite being incredibly powerful and theoretically capable of dual-use risks... wouldn't have been covered at all. It would have swum right through the holes in the net. Which is exactly why the radical optionality paper argues forcefully against statutory definitions, definitions that are baked into laws passed by politicians in a legislature. Passing an amendment to change that $1 million threshold down to $5 million would take months, if not years, of political gridlock, committee hearings, and lobbying. It's the difference between a printed encyclopedia and Wikipedia. I like that analogy. Statutory law is a printed encyclopedia. The moment it comes off the printing press, it is instantly out of date. To update the definition of a frontier model, you have to publish a whole new edition through Congress. Which is impossible in this time frame. Right. But regulatory law, where an agile agency like this fine RA-style standards body sets the technical definitions based on the current science, is like Wikipedia. It can be updated the exact second. It's like Wikipedia. A $6 million model changes the game. That adaptability is the lifeblood of survival here. Hassabis himself argues that these safety evaluations and capability benchmarks shouldn't be reviewed annually. Annually is too slow. Way too slow. He says they should be updated quarterly to start. And if a benchmark becomes saturated, meaning the AI can easily pass it, that benchmark needs to be aggressively deprecated and replaced with a harder one. The regulatory body essentially has to act with the agility and paranoia of a nimble tech startup just to survive Moore's law. Okay, so we are building a massively funded, flexible standards body. We are treating legal definitions like over-the-air software updates. But let's take another short break. Good idea. Because when we come back, we have to talk about the data. None of this agile regulation works if the government is flying blind. They need to know what these labs are actually building behind closed doors. And that gets complicated fast. Okay. We are back with our deep dive into the governance of advanced AI. We've established that we need a flexible, highly technical regulatory body to manage this transition. But a regulatory body is useless without information. Completely useless. Which brings us to the flow of secrets. Gathering information is the operational core of radical optionality. But the paper makes a vital distinction that policymakers often confuse, the profound difference between transparency and reporting. Let's break that down. Transparency is about public disclosures, right? Yes. The paper cites things like California's SB 53 and New York's RAISE Act. These are laws that mandate AI companies publicly publish their safety and security protocols, their testing methodologies, and their governance structures. Which is undeniably useful. Transparency allows academics, civil society organizations, and independent researchers to scrutinize the guardrails that these companies claim to have in place. It builds public trust. But there's a limit. There is a hard limit to transparency. Companies cannot publicly disclose their most sensitive trade secrets or the specific architectural details of catastrophic vulnerabilities they've discovered without facing commercial ruin, or worse, giving a literal roadmap to bad actors. Right. If your AI discovers a way to bypass the security on global banking software, you don't publish a transparent blog post about it on Medium. No, you definitely don't. And that is where reporting comes in. Reporting is the confidential, highly secured disclosure of sensitive information directly to a government agency. And here the paper delves into some recent, somewhat fraught political history to illustrate how this mechanism has been attempted in the United States. It notes that the federal government previously collected this kind of information via the Bureau of Industry and Security, or the BIS. Let's look closely at the context the paper provides here because it's really important to understand the actual mechanics of the policy, regardless of the political theater surrounding it. Yeah, we have to separate the policy from the politics. So the Biden administration had implemented a reporting rule through the BIS. To do this legally, they utilized a piece of legislation called the Defense Production Act, or the DPA, to mandate these confidential reports from frontier labs. And the paper chronicles that subsequently the Trump administration revoked the broad Biden-era executive order on AI, and the proposed BIS rule was abandoned. Because there was pushback, right? Yes. Yes. Some conservatives and industry groups had heavily objected to using the DPA, which, for context, is a Korean War-era law designed for industrial mobilization during wartime. They objected to using that as the legal basis for peacetime AI reporting requirements. They viewed it as massive government overreach. But what I found really compelling about the Institute for Law and AI's analysis is that the authors impartially step back from the political tug of war. They don't take a side on the legal validity of the DP. No, they don't. It was essentially asking roughly five massive tech companies to send a highly secure email to a government agency once every few months. That's it. That's basically it. The email just says, here is how much compute we're acquiring. Here are the large models we are currently training. And here are the results of our red teaming safety tests. It does not stifle innovation to ask a multi-trillion dollar company for a quarterly status update. And those updates are utterly critical when we consider what the paper calls non-illegal threats. Yes, this concept is so important. The non-illegal threats. This part was wild to me. We have this mental model that if a tech company does something dangerous, they must be breaking a law. Right. There must be a statue. But with AI, the ground is completely uncharted. The paper brings up the incredibly revealing example of Anthropik's Claude Mythos preview. It's a fascinating and slightly terrifying case study. Claude Mythos was an experimental AI model that demonstrated incredible, state-of-the-art cyber defense capabilities. Okay. Sounds good so far. But in the process of proving its defensive skills, It successfully identified thousands of zero-day vulnerabilities in existing software, many of them critical. Just to make sure we are all on the same page, you know, a zero-day is a software flaw that hackers can exploit, which the software creator has zero days to fix because the flaw is entirely unknown to them. Exactly. If you find a zero-day in, say, water treatment plant software, you essentially have the keys to the kingdom. You do. Now, if Anthropic had simply released that model to the public as a product, They wouldn't technically be breaking any specific criminal law. Because it's just an AI model. Right. There is no law on the books explicitly banning the release of an AI that happens to be good at finding bugs. But releasing a system capable of pointing out thousands of critical zero-day vulnerabilities... to any hacker, terrorist group, or nation state who asks would be an absolute disaster for global cybersecurity. So without mandatory confidential reporting requirements, a private company might just stumble into building a digital superweapon, and the federal government would have absolutely no idea it existed until it was out in the wild causing havoc. That's the danger of not having reporting. But even if you have those reporting requirements on the books, how do you make sure the truth actually gets to the government? Ah, right. If a company's CEO decides to hide a terrifying safety test result to protect their stock price, how does the government find out? This is where the paper argues that radical optionality requires robust federal whistleblower protections. If a mid-level engineer at a frontier lab sees a catastrophic risk being willfully ignored by leadership, they need a legally protected secure channel to alert the authorities. Without fear of retaliations. Exactly. Without fear of being fired, blacklisted, or sued into oblivion for violating a nondisclosure agreement. The paper mentions that California's SB 53 has some limited whistleblower protections, but it argues forcefully for a federal standard. It cites the Bipartisan AI Whistleblower Protection Act introduced by Senator Chuck Grassley as a model. And it notes the EU is moving on this too. Right. It notes that the EU is bringing in some whistleblower protections starting in August 2026 under the AI Act. It's all about creating a nervous system for the government. Transparency requirements, mandatory confidential reporting, and protected whistleblowers all act as different sensory inputs. They allow the regulatory body to actually perceive the risks before they detonate. But this leads to a massive logistical and security nightmare in my head, and I have to push back on the paper solution here. Go for it. Let's say the whole radical optionality system works perfectly. A private AI lab accidentally discovers a model that can easily engineer a highly transmissible biological weapon. Okay, worst case scenario. Under this framework, they have to securely report this and hand over the algorithmic secrets, the model weights, to a government agency. Yes, the model weights being the massive matrices of numbers that constitute the actual trained brain of the AI. Right. But how on earth does a Silicon Valley tech company securely hand that over to the government without it leaking or being intercepted and stolen by a foreign intelligence service like China's MSS or a rogue state on the way? It's a huge logistical challenge. If the stakes are truly electricity or fire, securing these weights is the single most important task on the planet. And I look at the U.S. government's track record, the SolarWinds hack, the Office of Personnel Management data breach where millions of security clearances were stolen. And I think, how can Silicon Valley trust the Pentagon to secure their most valuable trade secrets? It is a totally valid concern and one the tech industry frequently raises. But the paper addresses this directly by pointing out a hard reality that Silicon Valley often overlooks. When it comes to keeping existential secrets against state-level adversaries, the United States government actually has decades of superior expertise compared to private tech labs. Which feels totally counterintuitive. We always picture tech companies as having the most advanced cybersecurity in the world and government agencies running on outdated mainframes. Tech companies have exceptional commercial cybersecurity. They are brilliant at stopping financially motivated ransomware gangs or low-level hackers. Right. But defending against a sustained multi-year state-level espionage effort by a hostile foreign intelligence agency is a completely different ballgame. The U.S. government has been administering the classification system for national security information for nearly a century, dating back to the actual Manhattan Project. That's a good point. They have the system of security clearances, the physical infrastructure of SCIFs, sensitive compartmented information facilities, and the intelligence-backed channels that a private company simply cannot replicate. So the paper is arguing that we can't leave the physical security of these server farms to private security guards. The Department of Defense and federal intelligence agencies need to step in to help set the physical and cybersecurity standards for the entire AI supply chain. Precisely. If a foreign power steals the model weights of an unreleased, highly dangerous AI, the game is entirely over. The core tenet of radical optionality requires that society maintains the option to not deploy a dangerous model. And you lose that option. You lose that option entirely if an adversary simply hacks your servers, takes the model, and deploys it themselves. So if we zoom out and look at all these pieces we've discussed, we have the Feinare style standards body with billion dollar funding. We have flexible Wikipedia style regulatory definitions that can adapt to a six million dollar breakthrough. We have the flow of secrets secured through mandatory reporting, protected whistleblowers and military grade classification systems. It's a robust container. It's an incredible blueprint for a container. But a container for what? What is the actual goal here? Which brings us to the final and, frankly, most profound segment of our deep dive, the endgame. The post-scarcity world. The post-scarcity world. This is where Demis Asabas' manifesto... and the meticulous policy architecture of the radical optionality paper truly converged. Because all of this intense regulatory scrambling, all of this national security anxiety, and all of these billions of dollars are ultimately in service of surviving the transition into an entirely new era of the human being. condition. The paper references a few major philosophical camps that are actively trying to define what this future should look like, and they are radically different. There's Leopold Ashenbrenner's concept of situational awareness. He essentially argues that because the national security risks are so severe, the only way forward is a locked down, heavily militarized, government run AGI Manhattan project. We nationalize the technology to ensure it doesn't leak to adverse Then you have Dario Amodei, the CEO of Anthropic, proposing a slightly different geopolitical structure, a coalition of democracies. Right. He envisions an entente where democratic nations strictly control the global AI supply chain, trading access to cutting edge AI compute for geopolitical alignment, effectively freezing out authoritarian states. And on the other end of the spectrum, you have thinkers like Vitalik Buterin pushing for DAC, or defensive acceleration. He argues that centralizing this much power in a Manhattan Project or a global coalition is inherently dangerous. And we should instead focus on open sourcing and accelerating defense-favoring technologies so that no single actor has a monopoly on power. What all these thinkers and Hassabis himself are recognizing is a singular, unavoidable truth. Which is? If we actually manage to mitigate the technical risks, if the radical auctionality playbook works and we successfully avoid the engineered bioweapons, the crippling cyber attacks, and the rogue superintelligence, we don't just go back to normal. We cross a threshold into a reality where the fundamental economic and philosophical structures of human life are completely upended. Hassabis asks some massive questions in his manifesto. What sorts of economic models work in a post-scarcity world? When an AGI can invent unlimited clean energy, discover a pharmaceutical cure for every ailment, and generate infinite material abundance through advanced robotics. The core concept of scarcity, which has driven human economics, trade, and conflict since the dawn of time, just evaporates. And that leads to the ultimate philosophical crisis. Right. If we don't have to work to survive, what values do we want to live by? What will meaning and purpose be? How might the human condition itself change when the struggle for survival is removed? It's a huge paradigm shift. And I have to point out a glaring contradiction in Hassabis' piece here. He writes this incredibly poignant line, quote, Resolving these questions obviously cannot and should not be left to technologists alone. It requires every part of society to come together to help define this new challenge. It's a beautiful democratic sentiment. It is, but he is literally one of the primary technologists driving the bus. He is leading a company racing to build this technology behind closed doors at breakneck speed. That's the paradox. If the pace is exponential, and AGI is only a few short years away, as he claims, How do we, the general public, the non-engineers, the artists, the teachers, actually get a seat at the table to decide the fate of humanity before it's a done deal? And that profound critique is the exact reason why the radical optionality framework is so desperately vital right now. If we don't build the institutional capacity today, if we don't establish the government standard bodies, the mandatory reporting requirements, and the secure testing facilities, the public will never get a seat at the table. Wow. If we default to Path 1, the decisions will be made entirely by a handful of unelected tech CEOs. If we default to Path 2 and ban it, the decisions will be made by authoritarian regimes. Or if things go catastrophically wrong, the decisions will be made by a closed security state enacting martial law to contain a crisis. So radical optionality is about preserving a choice. Radical optionality isn't just a technocratic plan to prevent a terminator scenario. It is a desperate attempt to preserve a democratic open framework so that humanity retains the agency to actually answer these massive philosophical questions together. It's about keeping the window of human agency open, even as the storm bears down on us. Exactly. We are building a shelter so that we have a place to sit down and debate the future. Okay, let's bring it all home and recap this incredible journey for you, the listener. We started in the exponential foothills of the singularity, looking at a technology that is taking inert sand and waking it up, an event that could be ten times the magnitude of the Industrial Revolution arriving in just a few short years. We examined the terrifying reality of the pacing problem, where human laws simply cannot keep up with an AI that writes its own code and recursively improves its own factory by lunchtime. We looked at the flawed extremes of our instinctual governance, the naive hope that the free market will safely regulate a dual-use superweapon, and the equally flawed hope that we can anticipatorily ban things before we even know what they are without handing the future to David. Which led us to the pragmatic, agile, and heavily funded toolkit of radical optionality, building FinRA-style public-private standards bodies, utilizing flexible Wikipedia-style regulatory definitions instead of rigid statutory laws. And securing the critical flow of secrets through mandatory reporting, whistleblowers, and government-grade classification. Exactly. And why should you care about all of this? Why does this dense alphabet soup of policy frameworks, DPA regulations, and compute thresholds matter to your daily life? Because the future isn't written yet. Right. Understanding these mechanisms isn't just homework for politicians and software engineers. Understanding radical optionality is how you, the general public, stop being a passive bystander. It is how you become an informed participant in what is arguably the most important transition in human history. If the public doesn't demand that our governments proactively build this capacity today, we entirely forfeit our voice tomorrow. And I want to leave you with a final, slightly provocative thought to ponder as you go about your day. We spent all of our time and we spent this entire deep dive worrying about the risks. Will AI destroy the power grid? Will it engineer a novel virus? Will it go rogue? The existential fears are loud. They are deafening. But what if the radical optionality plan works flawlessly? What if Hassabis' standards body functions perfectly? Yes. What if we navigate the cyber risks, the bio risks, and the geopolitical tensions without a single catastrophe? The best case scenario. Yeah. What if we actually achieve total post-scarcity, conquer every disease, and eliminate global poverty? We might conquer every existential threat only to face the most terrifying challenge of all. waking up every single day in a perfect world having to invent an entirely new reason for human beings to exist when the struggle to survive is gone we took inert sand and we made it think the real question is once the sand can do all our thinking producing and surviving for us what are we supposed to do it's a daunting question think about it we'll see you next time on the deep dive