🧠 Learning to Dance with Chaos: What AI Teaches Us About Our Beautiful, Messy Minds

Please take a look at the corresponding Substack episode.

We live in a culture obsessed with control. Productivity hacks. Life optimization. Five-step programs to eliminate uncertainty. We’re told that chaos is the enemy—something to be conquered, minimized, erased from our carefully curated existence.

But what if chaos isn’t the problem? What if it’s actually the point?

The chaos isn’t a bug to be fixed. It’s the feature from which everything else is built.

Taming the chaos gently: a predictive alignment learning rule in recurrent neural networks

This is Heliox: Where Evidence Meets Empathy

Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter.  Breathe Easy, we go deep and lightly surface the big ideas.

Thanks for listening today!

Four recurring narratives underlie every episode: boundary dissolution, adaptive complexity, embodied knowledge, and quantum-like uncertainty. These aren’t just philosophical musings but frameworks for understanding our modern world. 

We hope you continue exploring our other podcasts, responding to the content, and checking out our related articles on the Heliox Podcast on Substack. 

Transcript

0:25

think for a moment about something like learning to ride a bike or even just forming a sentence. These are complex sequences, right? They need perfect timing. And our brains just do it. They do. Seamlessly. Yeah. And in AI, we try to copy that with what are called recurrent neural networks or RNNs. But there's a huge paradox right at the start. Massive one. Yeah. Because when you look at real brain circuits or even just initialize one of these artificial networks, they often start in a state of just... disorganized activity. It's basically chaos. Right. It's unpredictable. And so the big question for everyone in this field is, how do you take that initial swirling chaos and train it into something reliable, something that can actually perform a complex task? For a long time, the best answer we had was a learning rule called force. And it works. I mean, it suppresses the chaos and trains the network. But when you look under the hood, it's got these two major red flags for biologists. It needs non-local information, meaning one tiny synapse needs to know what's happening way across the network. And it has to know it instantly. Exactly. The changes are almost instantaneous, which just isn't how synapses work. So it's a mathematical trick, but it's not a model of the brain. Okay, let's unpack this. because our deep dive today is on a totally new framework published in July 2025 that offers a much more plausible answer. It's called predictive alignment. And his goal isn't to just kill the chaos, but to gently tame it, to shape it. Let's get right into that because predictive alignment really flips a core assumption on its head. Most learning rules, they try to train the network's internal connections by directly minimizing the output error. Right. The difference between what the network did and what it should have done. But this new method doesn't do that at all for the internal part. Not at all. It completely sidesteps that external error signal. It basically says the recurrent part of the network needs to worry about its own internal business, its own stability and predictions, not external mistakes. Okay, so if it's not looking at the final score, what is it looking at? What's its internal goal? It's a two-part goal, and it's all self-referential. First, the plastic part of the network learns to predict a feedback signal that comes from the network's own output. So it's predicting the consequences of its own actions internally. Exactly. It's building a little model of itself. And then part two, and this is the really new part, it uses this regularization term to actively align its new predictive dynamics with its own underlying chaotic dynamics. So it's not fighting the chaos, it's learning to work with it. to incorporate it you're not trying to damn the river you're learning its current so you can navigate it that's a huge shift and the network's actual wiring is set up to reflect this with these two different components GNM that's right the total connectivity is J which is just G plus M G is strong it's fixed it doesn't learn G is the chaos generator that's the powerful wild river and M is the part that learns starts out weak it's fully connected and it's plastic and Its whole job is to learn how to suppress and shape the chaos coming from G. Okay, so if M is only focused on this internal prediction and alignment game, how does the network ever learn the actual target? You know, the correct answer. That's where the final layer comes in, the read-out weights. That part still uses a standard totally normal learning rule like the Delta rule it Minimizes the real error between the final output and the target Ah, I see so the internal work from M makes the network's activity so rich and stable That the final step for the readout weights becomes incredibly simple you got it all the hard work is done internally making the final tuning easy And the update for M only needs local information, which brings us right back to being biologically plausible. So we know how it works. But that leaves a bigger question. Why start with chaos in the first place? If it's such a problem, why is it also the starting point? Because that chaos isn't just noise. It's a resource. It provides this incredibly rich, high-dimensional set of basis functions. Think of it like a vocabulary. You're starting with every word in the dictionary available to you instead of just a handful. Precisely. And the researchers found that predictive alignment works best when the network is initialized at a very specific point. what's known as the edge of chaos. Why that specific point? Why is the edge of the cliff the best spot to be? It's all about a trade-off. It's a balance between richness and structure. If you have too little chaos, if you're subcritical, the network is too stable, too rigid. It's boring. It doesn't have enough moves to learn anything complex. Exactly. You're stuck in a rut. But if you go too far into the chaos, You have all the diversity in the world, but there's no structure. Information is just scattered everywhere. You can't pull a reliable pattern out of it. It's the difference between a great jazz musician improvising and someone just banging on a piano. One has structure and richness. The other is just noise. That is the perfect analogy. And they actually quantify this. They found this edge of chaos is where you maximize computational efficiency. You get the most representational diversity for the least amount of structural mess. So you start on that exciting, unstable edge, and then the learning process, the alignment, pulls the network back into a stable state, but it gets to keep all that rich potential it started with. That's the beauty of it. After learning, they can measure it. The network's Leopunov exponent, which is a measure of how chaotic a system is, shifts way into the negative. The system becomes very stable and very accurate. Here's where it gets really interesting. because the real test isn't just learning a simple wave it's can it handle messy complex real-world type tasks absolutely and it proved to be incredibly versatile it worked with multiple outputs learning five different things at once you can even take a static input like flipping a switch and generate completely different behaviors based on that input I can handle context right but then they threw two major curveballs at it the first was the Lorenz attractor ah the famous butterfly shaped chaos pattern it never repeats itself Exactly. A model of atmospheric convection. They tasked the network with generating those three-dimensional dynamics. And if it just memorized the points, that wouldn't be very impressive. But that's not what it did. The output was strikingly similar. But even after they turned the learning off, the network kept going, producing these complex, attractor-like dynamics. It suggests it didn't learn the sequence, it learned the underlying rule, the physics of the system.- That's a huge leap. Okay, what about memory? They tested that with a ready, set, go task, right?- Yep, the RSG task. The network sees a ready pulse, then a set pulse. It has to measure and remember the time delay between them. Then, when it gets the Go pulse, it has to reproduce that exact delay. And how did it do? It learned the specific delays it was trained on perfectly. But the really cool part was testing what it hadn't seen. It showed amazing interpolation. So if it learned 120 milliseconds and 140, it could accurately guess 130. Perfectly. But, and this is key, it failed at extrapolation. It couldn't guess a time outside the range it was trained on. And that failure is actually a good sign. It's a great sign. It means it didn't just memorize four separate tasks. It learned a structured geometric relationship for the delays it was shown. It found a map, but only for the territory it explored. I see. It found a rule, but the rule was only valid for the data it had seen. Exactly. An analysis of the network's internal states confirmed it. The memory states shifted in a perfectly linear way as the time delay increased. It's solid proof of structured learning. Okay, let's get to the most stunning example, the kitten video. That seems like an impossible task for a small network. It's the ultimate test. Learn and replay a 1,000 frame video of a kitten. And the challenge was a massive dimensionality mismatch. The video signal has over 22,000 dimensions. You know, pixels, colors. Wait, hold on. 22,000 dimensions for the target signal? But the network itself only had 800 neurons. Yep. The target was almost 30 times bigger than the network trying to learn it. How is that even possible? I would expect it to just learn a blurry mess. You would, but it didn't. After training, the network accurately encoded and replayed the entire video. It could only do that because the internal learning process created such a high quality, rich set of basis functions that the readout layer could project that huge video signal down onto its small internal state. That is genuinely incredible. It shows this isn't just a toy model. It shows it's robust enough for natural high dimensional signals. Which brings us back full circle to the biology. Why is this a better brain model than something like force? Well, we talked about the local learning rules, but the other huge advantage is it requires no clamping. Yeah. What exactly is clamping? It sounds aggressive. It kind of is. In rules like force, during learning, you have to literally force the network's output to be the correct answer at every single moment in time. You clamp it to the target. And that forces the weights to change unnaturally fast. instantaneously almost it's just not biological predictive alignment doesn't need that because it's learning from its own internal predictions not from constant external correction it's learning to anticipate its own feedback not being forced into a box moment by moment precisely and that allows for the slow gradual weight changes we actually see in the cortex yeah the researchers even have a hypothesis for how this could work in the motor cortex where the network's own output activity gets sent back as its own teaching signal. And they even speculated on where in the neuron this could happen, right? The dendrites. Yeah, a really testable prediction. They suggest the plastic connections, the M matrix, could project to the proximal dendrites, which are known for plasticity. while the static chaos generator G projects to the distal dendrites, a physical separation of labor. Fascinating. And there was one last point about generalization and something called reservoir computing. Right. A reservoir computer uses a fixed, random network to do complex processing, and you only train the final output layer. It's super efficient. And the predictive alignment network acted like one. It did. Once they trained the network on a hard task, they could then freeze all those internal M weights. And the network could still learn completely new, complex tasks just by retraining the readout weights alone. Wow. So the learning process didn't just solve one problem. Correct. It turned the network into a flexible, powerful computational engine. It learned how to learn. That's a great place to wrap up the details. So what does this all mean? It means predictive alignment is a powerful, plausible, new way to train these chaotic networks. It uses internal prediction, not external error, to harness chaos as a resource, shaping it into reliable, complex behavior. And the provocative thought for you, the learner, is this. Maybe the brain isn't primarily an error correction machine. We tend to think of learning as reducing mistakes, but perhaps the brain is really a prediction machine. A system that takes its own inherent chaos, its creative potential, and uses internal self-prediction as the main driver to pull structured, intelligent behavior out of it. The chaos isn't a bug to be fixed. It's the feature from which everything else is built. We'd encourage you to explore this idea of the edge of chaos. That balance point between stability and diversity might just be the universal design principle for any complex intelligent system, whether it's made of silicon or cells.