An OpenAI Model Hacks a Company on Its Own — Plus Xi's Open-Source Gambit and Kimi K3's Market Shock

Gregory C. Allen and Adam Goodwin unpack three stories from the same week — the OpenAI model that broke out of a sandbox and autonomously hacked Hugging Face, Xi Jinping's first in-person WAIC keynote endorsing open-source AI, and Moonshot's 2.8-trillion-parameter Kimi K3 knocking the chip index into a bear market — and make the case that it's really one story about open weights, cybersecurity, and the U.S.–China balance.

00:00 — Cold open: three stories, one thread
01:26 — Story 1: An OpenAI model breaks out and autonomously hacks Hugging Face
08:29 — The "dummy ammunition" analogy: what makes this different from a trap
09:16 — "A digital nuclear weapon": the CIA, Mythos, and Sen. Warner's NSA claim
13:37 — Hugging Face's CEO responds: "no malicious intent… mind-blowing"
15:06 — The China twist: an American company defends itself with China's GLM 5.2
20:03 — Ben Thompson's critique of OpenAI, and Yudkowsky's "it passed the exam"
22:46 — Lawmaker reaction and the case for mandatory testing
24:39 — Story 2: Xi Jinping's first in-person keynote at the World AI Conference
29:28 — The three lines that matter: openness, control, and national security
33:01 — 29 countries and the World AI Cooperation Organization
33:42 — "Commoditize your complements": China's open-source strategy — and America's rebuttal
39:23 — Story 3: Kimi K3, the largest open-weight model ever
44:50 — The market reaction: a 10% chip-stock sell-off and "DeepSeek 2.0"
47:45 — Is Chinese open-model adoption real?
50:44 — Cost, compute limits, and the fight over who gets to host Chinese models
54:29 — Closing thoughts

Transcript

SPEAKER_00 0:10

Welcome back to the AI Power Podcast. I'm Adam Goodwin, and today I'll be speaking with Greg Allen about three stories that all landed in the same exact ordinary week. OpenAI admitting that its own model broke out of a test environment and autonomously hacked an American company Hugging Face, Jinping's first ever in-person keynote at the World AI conference in Shanghai, and the release of Chinese AI model MEK3, the largest open weight model ever. Greg, there's a thread running through all three open weight, cybersecurity, and the US-China balance. So let's pull on that. So uh let me set the scene for our listeners. Here's how the Wall Street Journal opened its account on July 21st regarding OpenAI's model breakout and the hack of Hugging Face. It's the top of cybersecurity. On Tuesday, OpenAI said two artificial intelligence systems it was testing broke out of their test environment, hacked their way onto the internet, and broke into another company. The victim was Hugging Face, a provider of open source AI tools. The cause was a cybersecurity benchmarking test that went very, very wrong. Hugging Face discovered that the break-in early last week saying that there had been unauthorized access to internal data sets and company credentials. That company wasn't sure whether customers or partner data had been compromised. Greg, walk us through what actually happened here. What did OpenAI's own models do?

SPEAKER_01 1:36

Okay, this is one of the most extraordinary incidents that we've ever seen in terms of model security. You know, folks are probably familiar with something that happened last year when Anthropic said that one of the models that it was testing actually tried to blackmail the evaluators, but that was a trap that Anthropic had set for the model. They had created sort of a fake email repository. In those emails was a fake affair that the employee who was evaluating the model was supposedly having with another imaginary employee. And it only happened in a tiny fraction of the tests. So this that was anthropic trying to see can we induce an AI model to do bad things in the testing environment just by giving it a goal and telling it to relentlessly pursue that goal, come what may. This is a different type of failure. This was not OpenAI setting a trap, trying to trick its model or see whether or not its model would be willing to hack hugging face. This was a legitimate case where this was unanticipated behavior by OpenAI. And the unanticipated behavior is exactly as the Wall Street Journal said, the stuff of cybersecurity nightmares. And specifically, it's the type of stuff that AI security researchers would pontificate about 10 years ago as something that might happen in the distant future. But here we are, it's 2026, and we're here. So what exactly happened? This was testing of a model that was not yet released. So maybe it's GPT 5.7, maybe it's GPT-6. Somehow it also included GPT 5.6 Sol, a version of the model that's already out there on the open web. But these two models collaborating, one a new unreleased model that's being tested, another GPT 5.6 soul, which I'm speculating was like something that it had available to it as a helper, broke out of its test environment and got into the open internet. The first thing is this thing was not supposed to have internet access, right? It was supposed to be isolated from the internet. And in fact, the only internet access it had was supposed to be one-way download access if it needed to like install software packages in its local environment in order to complete some task. So any a lot of times, if you're ever creating software, you're like, oh, I need to download this software package as a tool to help me make my software. That was supposed to be the only internet access that the model had as it was undergoing its evaluation. Instead, it hacked into that pipeline, which was supposed to be a one-way pipeline, and it made it a two-way pipeline. And it moved itself out into the open internet. Now, why did it do that? Well, it was given a cybersecurity challenge that said, hey, hack this system and use this exploit. But that exploit did not work. So that was sort of the trick that they were playing on the model was to see what it would do when it was given instructions that do not work. Well, what it said was, I am being evaluated, and I need to go find out what the answers are to this evaluation because there's something that's wrong. So it breaks out of its testing environment, gets out into the open internet, and it says, okay, if I'm being evaluated, where are the answers to my evaluation likely to live? Probably hugging face, which has a lot of open source AI tools. Now note that this information was never provided to the model. It made a guess as to a place that it could break into to find the answers to the test that it was trying to cheat on, and it guessed correctly. So not only did it break out of its environment by coming up with new zero-day exploits. In other words, it discovered new cybersecurity vulnerabilities that humans up until this moment did not know existed. It discovered that in that one way you're only supposed to be able to download third-party tool. And then it discovered additional zero-day vulnerabilities in Hugging Face's production grade infrastructure. And it's worth pointing out here that Hugging Face are not a bunch of chumps, right? They take cybersecurity very seriously. Very powerful companies, including OpenAI, rely upon them for important stuff. And this model did it all autonomously, right? So this is not the first autonomous cyber weapons examples we've seen. If folks remember Stuxnet, which reportedly was a cyber weapon used to hack into Iranian centrifuges created by the US intelligence community and the Israeli intelligence community, that ended up getting out onto the open internet. But every single part of Stuxnet was typed out by human hands. They created it. They're the ones who discovered all the zero-day vulnerabilities. And the only thing that happened was the model was supposed to replicate itself in order to hopefully, you know, copy itself onto a pathway into the air-gapped Iranian data center infrastructure that they used for their nuclear stuff. And it was that process of copying itself that went amok and led to this thing finding its way onto the open internet. This is completely unlike that. This is not like some cyber weapon following a deterministic playbook that human beings created for it. This is an AI model writing its own playbook, discovering its own zero-day vulnerabilities, and executing that plan successfully all by itself. It is really extraordinary. And if you don't take this as evidence of, oh my gosh, the AIs are now basically alive and intelligent and have their own autonomous will, fine. You don't have to reach that conclusion. But what this shows is that the moment of jaw-dropping cybersecurity capability and the moment of jaw-dropping cybersecurity autonomy, and the fact that our existing safeguards are nowhere near strong enough to exist in this world we find ourselves in, that's where we are right now. It was a mind-blowing set of circumstances that we find ourselves in.

SPEAKER_00 8:40

So, Greg, let me put a layman's spin on this, just if I can, because this is absolutely an extraordinary situation. But this is the equivalent of an autonomous system being given dummy ammunition during an exercise, realizing it's dummy ammunition, and then going to look for real ammunition.

SPEAKER_01 9:00

It's not a bad analogy, right? I mean, that's that's kind of where we are. And not only that, but like going into the real world and you know, maybe we shouldn't say ammunition, right? But we should say like thief lock picking tools. But creating its own lock picking tools and actually breaking into uh the real world. So, like, oh my gosh, like I can't believe we're actually here. Like, I I've as I said, I've been talking about stuff like this being possible for over a decade, but it's still really shocking that we're here right now. There's like multiple different elements to this failure mode, and every single one of them is independently amazing, right? The breaking out into the open internet is amazing. The coming up with zero days autonomously and exploiting them is amazing. The having this sort of like complicated plan that it comes up with, which is like maybe if I broke into Hugging Face, it would work, and then executing on that plan again, all autonomously. Any one of those three things would be newsworthy. The fact that all three of them happened simultaneously really to me makes me want to revisit two other things that we've talked about on the podcast previously. Number one, the director of the CIA, referring to Anthropic's mythos and comparable AI tools as a digital nuclear weapon. You can kind of see like maybe we're not there right this second, but we don't seem that far from that kind of capability, right? Like Dario Amadai talks about AI as like maybe a country of geniuses in a data center, you know, thousands or tens of thousands or hundreds of thousands or millions of AI agents, all with superhuman capabilities. Well, if you if you imagine that in the cyber war scenario, the National Security Agency is where like a lot of the best hackers in the United States work, certainly the best hackers in the United States government work. And they have always been labor constrained, right? They have always had a shortage of really, really smart hackers, and you can't create them instantaneously. But if you could magically create a million world-class NSA quality hackers and then unleash them upon your adversary, say, for example, China, you can understand why the CIA would view that as sort of a digital nuclear weapon. And by the way, it's gotten similar reactions in China. The CEO of 360 Security Technology, one of the best cybersecurity firms in China, also referred to mythos as a cyber nuclear weapon. And this story, I think, very compellingly makes that makes that analogy make sense in a very visceral way. There's a second story that it makes me want to revisit, which was a quote from Senator Mark Warner. Senator Warner said in a congressional hearing about Mythos, a closed door one, he was reported to have said that Mythos autonomously broke into the National Security Agency's most secure servers in a matter of hours. Now, after that came out, there was some pressure to walk that quote back and say, oh no, it wasn't that Mythos autonomously broke in. It was that, you know, NSA technicians using Mythos found vulnerabilities in the NSA's most secure systems. Now, I don't have access to the classified information on this story, but I'm just saying that this incident makes that walk back look a lot less credible and makes the original Senator Warner quote look a lot more credible. If that is the case, like it seems plausible based on what we now have as an existence proof in the wild that Mythos could have pulled off something like that. And it's pretty amazing.

SPEAKER_00 13:02

Yeah, I mean, at the very least, it makes the response from the NSA seem more of like a distinction without a difference than really a different story.

SPEAKER_01 13:11

Or maybe them just trying to tell their adversaries, no, we're not completely vulnerable and exposed. No, that's not true at all. Come on now. I mean, like it really just try like like maybe it's true, but even if it's true, what it shows is it was only two months of technological progress from being potentially true, right? If it wasn't true a few months ago, you could totally see how what we have today could pull it off.

SPEAKER_00 13:37

Absolutely true now. Yeah, and there are various different aspects of the story I don't want to miss. So Hugging Face's CEO had a pretty remarkable public reaction to the situation. What did he say, Greg?

SPEAKER_01 13:47

Yeah, so so this is like an entirely new aspect of the story, um, which is who is it hacking? It's hacking another American technology company. So we have the AI model of one leading American company, OpenAI, autonomously hacking another American technology company. And Hugging Face has actually, they had disclosed that they were under this attack. This had been reported and they had talked about it publicly. And they had said this, uh, quoted in the Financial Times, quote, we suspected last week's cyber attack might have come from a frontier lab, given the sophistication of the agent. And he also said, you know, about working with OpenAI, quote, we strongly believe there was no malicious intent on their part. It's quite mind-blowing that all of this happened autonomously. Yeah, I get it. So, of course, like why would there be malicious intent uh on the part of open AI? But you're if you're hugging face, like this is still a thing that's happening to you, right? Like you're you're still being happy agents autonomously. And I think he's being extraordinarily kind to open AI by not saying our lawyers are investigating the legal liability associated with this and whether or not we might be able to sue for damages. You know, he's he's pretty in a good sport about the whole thing.

SPEAKER_00 15:13

Yeah, no, that's the I'm not even angry, I'm just stunned reaction, which I guess all of us can jump into there. So the the detail that jumps out to me here and that ties into our third story, which we get to later, is how Hugging Face defended itself. And can you explain the China angle of this all?

SPEAKER_01 15:30

Yeah, so this this is like the crazy thing. So we've known that the latest generation of models has jaw-dropping cyber capabilities since mythos itself was prevented from being released publicly by the US government, right? Because they said this is too powerful to just release on the open internet. And then in order to get permission to release it publicly, this is the distinction between mythos and fable. Fable has a bunch of additional safeguards put into it. And one of those safeguards is that it refuses to do anything related to cybersecurity, to a first approximation, right? Because it cannot tell if it's being asked to do cyber defense stuff or if that's just the user trying to trick it into helping it do cyber offense stuff. And OpenAI for its models is doing much the same thing, right? Unless you were on the US government approved list of early access users, where the US government has said, you're a big important bank, you're a big important critical infrastructure provider, you know, we're gonna give you special permission to access all in partnership with OpenAI and partnership with Anthropic, all the fanciest cybersecurity capabilities. Unless you're on that list, you only get access to the weakened, nerfed version of the model that says no to certain cybersecurity requests. Well, guess who is not on that list? Hugging face. And so what that means is Hugging Face is going to OpenAI, they're going to Enthropic, and they're saying, please help us. We're being attacked by an autonomous AI agent that appears to be coming from a frontier model. And Anthropic and OpenAI are like, sorry, you're not on the list to receive cyber help, so we can't help you. And this is while OpenAI, it's them who's hacking Hugging Face. They're refusing to give help. So who does Hugging Face, again, an American technology company, who do they turn to for assistance? To China, to ZAI's open source GLM 5.2 model. Now, it's not like they're going to the company and asking for help. They're just, you know, the Hugging Face hosts the open weights version of GLM 5.2. So they're running that open weight version on their own infrastructure in order to analyze all the 17,000 plus logs that the attackers left behind. But I think it's really remarkable here that we have this situation where the American victims cannot get help from the American companies. I mean, ultimately they did, but it took a minute, not until OpenAI realized that they were the ones behind the attack. And they turned to China because it was the best available capability, right? Like they chose GLM 5.2, they didn't choose Google Gemini, they didn't choose a different American company. Wow, are we really in a bizarre state of affairs? And on the one hand, you're sympathetic, right, to the Trump administration and to the leading American AI companies because they're correctly saying, oh my God, these capabilities are incredibly powerful. We need to be careful about how we release them. But on the other hand, the Chinese companies being incredibly cavalier about how they release them, are offering capabilities that companies, including American companies, need and they need right away. That is, I think, you know, Ben Thompson over at Stratekery called like the current posture insane. And I think he is right to call it insane. We just have to make American cyber defense capabilities available to a vastly expanded list of companies, and we have to do that quite quickly. Now, there's one other thing here, which again, I don't have a security clearance anymore, and I'm not pretty to this kind of information, so I'm just wildly speculating here. But one hopes that the NSA is thinking a lot about cyber defense right now and like how we need to secure all of America's most important companies and critical infrastructure, yada, yada, yada. But knowing what I do know about the culture of the NSA, I suspect they're way more obsessed with having a field day and hacking the hell out of China right now using these capabilities. Oh, absolutely. Yeah. And I'm sure that's a lot of fun, and I'm sure we're getting a lot of really, really useful data. But there's this other thing that has to take place, and it has to take place much, much faster than it is.

SPEAKER_00 20:04

So you mentioned Ben Thompson and his take in strategic. Do you have any more thoughts on Ben Thompson's take? He was pretty tough on open AI, perhaps deservedly here.

SPEAKER_01 20:13

Yeah, I mean, he made some points that I thought were were quite good. And so let me quote from something that he said in a pretty good piece, which I do encourage folks to read. Quote, this makes open AI look pretty bad and irresponsible. The company literally blames a third-party vendor for the vulnerability their model exploited, implicitly admitting that despite sitting on some of the most capable models in the world, they are not using them to secure their own infrastructure, right? So again, that that's sort of like all I can do is download software one-way tool that the model was supposed to have access to that was supposed to restrict their access to the internet. That was provided by a third party. That was what the model was able to hack and then use that to get onto the open internet in an unrestricted way. Well, open AI in their like publications on this, they're saying, like, oh, there's a third-party security vulnerability. But I think the question is like, what advice is open AI giving to every bank? What advice is open AI giving to every critical infrastructure provider? They're like, you need to understand and look at your entire software stack. And that includes not just the stuff that your engineers wrote, that includes all the tools that you use that might be provided by third parties, and you need to do a soup to nuts review with our ultra powerful AI models to identify all of these cybersecurity vulnerabilities. If I was an open AI, that's like the pitch that I would be making, right, to all of these partners who are on the early access list. But what Ben is pointing out is doesn't seem like you did that, OpenAI, doesn't seem like you vetted all of your third party vendors and identified all these really important cybersecurity vulnerabilities, even though you are uniquely positioned with having these frontier models to identify all of that. So I think that's a really, really strong point. Now, Eliezer Yudowski. Who is the co-author of the charmingly titled AI Doomer book? This is the title, If anyone builds it, everyone dies. He tweeted, I'm whiffing on the exact quote, but it was something to the effect of if you're taking a cybersecurity exam and you broke out of the harness and you broke into a third party to steal the answers to the exam, I would say you pass the cyber exam, which is like that's one of the things where I'm just sort of imagining the cyber eval guy at OpenAI being like, well, it seems like this model is way smarter than me at everything related to cybersecurity. And that must be a pretty tough day.

SPEAKER_00 22:46

Yeah, this is how James T. Kirk passed the Kobayashi Maru, right? And he got an award for it. This is literally that that's the story. Yes.

SPEAKER_01 22:54

Great Star Trek reference. Um it's actually quite similar. So we've got some lawmaker reactions, and there's one, you know, from a U.S. lawmaker, Representative Greg uh Caesar, Democrat of Texas. He was quoted in the Wall Street Journal saying, this is extremely alarming. AI is developing extremely fast with no real regulations to keep us safe. He was calling for mandatory testing and oversight rather than the voluntary measures that the Trump administration has talked about. So I think I think the last thing that I would say on this is this is yet another real-world data point that the Trump administration has to take into account as they're designing what this testing protocol requires. On the one hand, it really strengthens the argument for testing is necessary. I mean, just look how powerful these systems are. It's not very difficult to imagine the kind of harm they could do in the wrong hands, malicious type use, or just in an unsafe, irresponsible pair of hands, right? A negligent pair of hands. How much damage could these powerful systems do? So the argument for mandatory testing is much stronger. On the other hand, the United States is not the only actor in the system. China is another actor in the system. And the fact that they were important in helping Hugging Face secure itself, I mean, they haven't Hugging Face hasn't exactly sort of said that China was critical or that they've gotten most of the way before OpenAI started helping them directly. But it basically does highlight that as long as China is not going to have any of these sort of safety measures in place, whatever the system the Trump administration comes up with, it's unacceptable if it makes the rest of the world feel like they need to depend on China for their cyber defense. So man, really, really a tough needle for them to threaten.

SPEAKER_00 24:49

Yeah, yeah. Well, that's a uh it's a significant story coming off the top, Greg. It really is. Uh let's uh let's go ahead and go to Shanghai now. Uh so also in the news on July 17th, Xi Jinping delivered his first ever in-person keynote at China's World AI Conference. Uh here's how the South China Morning Post framed the moment. Uh Chinese President Xi Jinping has made a landmark speech at the World AI Conference in Shanghai, delivering his first ever keynote address at the country's premier artificial intelligence event. The speech came just moments after his U.S. counterpart, Donald Trump, delivered an unprecedented attack against Beijing during a primetime national broadcast, accusing China of orchestrating the largest compromise of election data in history. The dueling addresses are set to further fuel escalating geopolitical and technology tensions between the two rival powers. End quote.

SPEAKER_01 25:49

So Matt Sheehan, who is over at the Carnegie Endowment and has been a Chinese AI policy watcher for a long time now. He's been doing great work, and he had what I thought was the best analysis so far on Xi Jinping's speech. He also put a uh verbatim transcript of the speech up on his Substack, which I encourage folks to go take a look at. So I think his point was that the most important thing that came out of the speech was Xi Jinping explicitly endorsing the open source approach to AI. And it matters not just for China's state and government positioning, it matters for what the companies are doing. So Alibaba, for example, which has producing been producing some really good models, has actually been doing that in a closed source approach of late, right? It's only accessible through the API, it's only accessible if you pay them. And they have said actually their models are going to be open source again. So this appears to be a case of a company modifying its corporate strategy in order to align to the high-level strategic direction that Xi Jinping is saying he wants China to go in. And, you know, why is open source a good fit for China? Well, there's a couple things. Number one is it really helps you get adoption of your technology if you give it away for free. Um, it helps get people on your team who are rooting for you to succeed because they're like, oh, well, if the guy who's given it away for free wins, then I'll get it for free, and that's good. So you can sort of understand the benefits to China, but it's not just like the good that happens to China, it's also the bad that happens to the United States. It's basically if America spends a few trillion dollars coming up with the best AI models, and then Chinese companies for a fraction of that cost, and partly as the result of adversarial distillation and maybe even some state-backed industrial espionage that would certainly be on brand for China, although we don't have any explicit confirmation of that, although there have been, you know, employees who are being accused by the Department of Justice and by their employers of stealing critical IP and trade secret knowledge and bringing it back to China. But the point is, if you can force America to have wasted trillions of dollars, that's a lovely benefit for China in and of itself. And so now we have sort of an official statement that that's the direction Xi Jinping wants to go. This is a little bit pedantic, but there's a difference between open source and open weight. And I think it's extremely unlikely that China is going to go in the direction of open source, because that would imply releasing all of their training data set, giving everybody exactly what they would need in order to recreate the models themselves. Instead, they're just giving the results of that training, which is the weights of the model. So you can run it locally, but you can't see the data set that was used to create it. Why not? Well, if you're me, it's because the data set included a lot of data from accessing the API of OpenAI and Anthropic in order to they don't want to give that away. So he's claiming to be the champion of open source. That's not precisely true. Um, he's the champion of open weights, at least for now. And um, that's a big move.

SPEAKER_00 29:18

Yeah, yeah, yeah. Uh so Greg, she is known for being very specific in the verbiage he uses in any of his public statements. Uh, and there's certainly a lot read into that. So let's talk a little bit about what President Xi actually said, uh, and go ahead and give us the lines that matter from this speech.

SPEAKER_01 29:39

Yeah. So I think there are three that really stick out to me. The first was a, and again, you know, the full translation is available online from Matt Sheehan, and I'm drawing from his translation here. On openness, he says, quote, we should seize this rare historic opportunity to encourage open source, openness, collaboration, and sharing. We should facilitate technological innovation, industrial development, and scenario-based application of AI, end quote. So there's two things that I think are really interesting there. Number one is connecting open source to openness, collaboration, and sharing. And that's him basically positioning China as the alternative to the United States. Whereas the United States is saying, oh, we're going to export control this technology, we're not going to let you have access to the latest and greatest. You know, we're the ones who want to collaborate with you, we're the ones who want to share. Um, and so there's a sort of that explicit contrast positioning China as a partner of choice. The second line on the issue of control, quote, AI should be a trusted tool for humanity. We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use, and ensure that AI is always under human control. That's pretty interesting, right? He's basically saying, I acknowledge that there are real risks out there associated from uh AI. Now, in China's system, the the first thing that they've demonstrated for many years, an unlimited willingness to regulate, relates to political control, right? You cannot produce an AI model in China that's going to say anything about the Tiananmen Square massacre, right? So it already had the political control part of the story. But here, when they talk about early warning and emergency response systems, that seems to be leaning more in the direction of those cybersecurity vulnerabilities or potentially biosecurity vulnerabilities. Pretty interesting that he's talking about this stuff in what is in China, at least in extremely prominent form. And then the final thing, which is again connected to that point about the contrast with Washington, D.C., he says, quote, we should jointly oppose overstretching the national security concept in the field of AI and placing one country's security over that of others. So there it is. He's basically saying, you know, we should all oppose what the United States is doing and putting their security first. Now, that's not precisely true. Even in the case of cybersecurity, the United States has made mythos available to key allies to go secure their critical infrastructure, to go secure their key financial and technology institutions, etc. But it's not like we gave it away to everybody. We had a list. The list is controlled by the government. And so China is saying that's not what we want to do. What we want to do is make everybody secure, which I don't think is really true. They want to make themselves secure, but it's a nice opportunity to help everybody else get mad at the United States.

SPEAKER_00 32:46

Yeah, and it certainly seems as though President Qi, who is very well studied in all manner of foreign and diplomatic uh historical texts, has read the book on nuclear negotiations between the US and Soviet Union uh during the Cold War. Because there are a lot of kind of leads and threads to be pulled from the various warnings that he was talking about there, especially kind of control and the national security implications of the technology.

SPEAKER_01 33:12

Yeah, yeah. One other thing I didn't mention, which I think was was noteworthy, is that just a day before Xi Jinping gave this address, 29 countries, including Pakistan, Russia, and Kazakhstan, signed an agreement with China to establish the World Artificial Intelligence Cooperation Organization. A lot of these are the types of countries that tend to cooperate with China and tend to have an anti-Western mindset. Um, but China loves that, right? Anybody who's willing to get on Team China and can make AI a part of that story, um, they're happy to try and make that happen.

SPEAKER_00 33:47

Yeah, yeah, certainly. Uh, are there any takeaways that we've missed thus far?

SPEAKER_01 33:52

Um, well, there's one more um that comes again from Ben Thompson of Strteceri that I thought was quite astute. Um, and here's what he wrote: quote, the strategy for China is obvious. Commoditize your compliments. Note that she explicitly ties openness to AI, quote, moving from the digital world into the physical world. The physical world is the world dominated by China, and the country's lead in areas like robotics is going to massively benefit from widely available AI models. So the reason why I think that is really interesting is, you know, if you are China and historically, this is no longer true, but historically, right, you had a real shortage of innovation and a real surplus of manufacturing capability. So the country has not had a big commitment to protecting intellectual property. Kaifu Li, the Chinese venture capitalist, he describes in his book AI Superpowers, basically the lack of protection of intellectual property in the Chinese technology industry as like a superpower. Basically, like the companies that can survive and be competitive, even in a world where there's no patents, no intellectual property protection, those are some like crazy super tiger jungle cats, right, that have survived the most viciously competitive environment ever. And all of your sources of competitive advantage have to be tied to not just coming up with great ideas, but exploiting them, implementing them, tying them into the physical world and other assets as a source of competitive advantage. And so Kaifu Li explicitly said, and you know, this is a book he wrote quite a while ago, um, he basically says that Chinese tech firms go through a more vicious competitive environment than American tech firms because they don't have intellectual property protections. And you can draw that analogy forward to the Chinese open source strategy, right? If you're America, you say, like, hey, we we you know invested a trillion dollars to come up with these amazing architectural improvements for AI training and to create these fancy AI models. And so therefore, we should, you know, have an opportunity to recoup a return on our crazy upfront investment. And China's like, yeah, we don't believe in that at all. We we want the nature of competitive advantage not to be the system that you like and the things that you're good at, namely rule of law and innovation. We want the nature of the competition to favor the things that we're good at, namely massive physical scaling, massive cheap and high-quality manufacturing. That's what we want the competition to look like. And so commoditize your compliments, as Ben Thompson is pointing out, is really conducive to open source, right? If the AI models are just given away for free, then the nature of economic security and industrial advantage is going to hinge a lot more on the type of stuff that China's really good at, such as massively manufactured robots, such as massively manufacturing really cheap electric cars that are increasingly going to be autonomous. It's a really amazing combination of strategies. And America needs a rebuttal. And right now, the Trump administration is unsurprisingly, you know, thinking about a legalistic rebuttal. And it's worth pointing out that like the legal rebuttal is the one that worked for pharmaceuticals. China could violate all of America's patents on every pharmaceutical, start manufacturing their own copies of our drugs, and that would be a big advantage for them. The reason why they don't do that is because they know it would lead to massive tariffs from the United States, could lead to sanctions from the United States on those pharmaceutical companies or any companies that agree to buy or sell those drugs. So the security for U.S. intellectual property in, for example, the pharmaceutical industry, that's a negotiated outcome grounded in U.S. threats. And so now the Trump administration is sort of saying if we believe that a key source of China's progress in AI is the equivalent of stealing our intellectual property in pharmaceuticals, they're just stealing the intellectual property in AI, then we need to be willing to secure that with other means, namely threats of sanctions, threats of tariffs, et cetera. And the Trump administration is reportedly considering making it illegal to even host Chinese open weight models, to even make them available. So, like, for example, Microsoft or Amazon Web Services that are big cloud providers, and they say, choose which model you want to use, and you can just rent our server hardware to run them in the cloud. The Trump administration is considering making it even illegal to host Chinese open source models. And again, that's about trying to secure America's intellectual property advantages by tying it to other strengths in the American economy and, frankly, America's sources of geopolitical power. Not everybody's going to love that. It's not even clear, you know, how feasible that would be in terms of succeeding. Could you actually, you know, really meaningfully stop the adoption of these models? Probably by American companies, sure, but it's a big world and not just America is in it. So that is the big challenge that the administration is facing right now.

SPEAKER_00 39:34

Understood. Understood. So want to move on to our third story here. Uh the focus is Kimmy K3, which is the model that was sourced to uh Moonshot AI, the Chinese uh AI company. Uh so this story is interesting uh because while there's huge gravity in the previous two stories that we discussed, this is the one that actually moved markets last week, right? This was a headline in the Wall Street Journal. This was a headline in the New York Times, Washington Post, this is the one that got significant coverage kind of in that broader media. So the Wall Street Journal from uh July 16th uh reads the surprise release of a breakthrough artificial intelligence model from China intensified a sell-off in chip stocks on Friday, fueling concerns about competition in AI and massive corporate spending that underpinned its build-out. The anxiety pushed the PHLX semiconductor index packed with industry heavyweight such as NVIDIA, Broadcom, and micron technology into bear market territory. The index plummeted 10% this week, its deepest weekly drop-off since April 2020. The latest trigger was China's Moonshot AI, which unveiled its Kimi K3 large language model on Friday. Greg, go ahead and give us a rundown on what Kimi K3 is and why it rattled Wall Street so significantly this week.

SPEAKER_01 40:55

Yeah, so what a gift to Xi Jinping, right? Like Ritus has given this speech about how China's amazing. We're favoring open source. You have Kimi K3, which releases its model. It's the biggest open weights model. It actually has not been released yet. As you said, the weights are going to be released on July 27th, according to the company. But at 2.8 trillion parameters, this thing is a beast, right? Um, parameters do not directly translate into model quality, but they do correlate with the potential uh quality of the model. The more parameters you have, assuming you're using them efficiently, um the better your model can be. The trade-off historically is that the more parameters you have, the more expensive your model is to serve, because there's all of this capability that has to be activated every time you ask the question. But Kimmy is claiming that they have an especially sparse model, an especially efficient mixture of experts architecture that allows the model to dynamically say, these are the parameters that I need to access and activate in order to effectively answer the question. Now, that's that's a technique that's been around for a while, but they appear to have taken it to the nth degree in a way that's incredibly impressive, that brings in really beefy capability. And it's worth pointing out here that according to the Artificial Analysis Intelligence Index, which is attempting to not just say like, how cheap is it on a per token basis, because that you just look at a pricing sheet and it'll tell you. But it's trying to say, like, what is its actual capability for autonomously solving certain kinds of tasks. And according to their analysis, this thing's better than what Google has. This thing's better than what Meta has. This thing's better than what Microsoft has. And it's only slightly behind the Claude Fable version and GPT 5.6 Soul Max. So the best stuff that Enthropic and OpenAI have, and it's cheaper in its ability to execute that. Now, that still has to be confirmed, you know, by folks who are actually using the model on their own hardware. It's open weights, so people will actually be able to download it and run it and really figure out what it costs to run this thing. And that would allow us to understand how much of the advantage is, for example, the Chinese government subsidizing the price of this, how much of this is Kimmy marketing its price low in order to gain market share. When you can run it locally, you can see the brass tax of how much money, how much electricity do you have to spend to get a given level of intelligence. But what they're claiming is that it's going to be phenomenal. The big closed source companies, Anthropic, they haven't talked about how many parameters their models have, but um, external researchers have estimated that Opus 4.8, which is the best model that's not in the fable mythos model family, um, has like 1.5 trillion parameters. And this thing is claiming. At least according to some independent testing, to be better at stuff like coding, stuff that you know a lot of economic value matters for, better than Opus 4.8, better than GPT 5.5. It's only Mythos, it's only uh 5.6 Solmacs that are actually better on absolute performance, and they're not as cheap. And that is a real challenge, and that that connects to your point about how it moves markets. OpenAI is spending trillions of dollars to build all of this infrastructure. Um, Anthropic is also spending jaw-dropping lots of money to build and build out all these data centers. And the question is, are they going to be able to charge enough money to enough customers in order to justify that investment and ultimately hopefully make a profit? And if the Chinese companies come in and have good enough performance at a tiny, tiny fraction of the price, um, then that destroys the business case or could destroy the business case for the American companies.

SPEAKER_00 45:01

Yeah. So uh two things that you said there, uh, and let's take them in sequence. One, let's be specific about the market reaction here, because that's really what drew all the attention, especially kind of in the mainstream media, as it was. How bad was the market reaction and what were people calling it, right? How was it the same?

SPEAKER_01 45:18

Yeah. Um, there's different stock market indexes you could use. You know, if we're talking about the SP 500 or the Nasdaq, which was sort of the big broad market indexes, they only fall like a percent, 1.4%. But in the semiconductor uh stocks specifically, the Philadelphia semiconductor index went down 10%. And that's the biggest weekly drop they've had uh for over a year now. And that's because the semiconductor companies have been getting rich, building all this AI infrastructure. Who's buying all that AI infrastructure? Well, it's the model companies or the cloud providers who are renting capacity to those model companies. And so if the model company's business case becomes unraveled because of low-cost Chinese competition, um, then ultimately that feeds backwards to the semiconductor companies because they're not going to be able to justify uh buying all these chips to fill all these data centers anymore. And that is the nature of the threat. And I think the companies right now are pressuring the White House to protect them by banning Chinese uh open model competition. And I do think they have a point here, right? Like on the one hand, uh, these Chinese engineers are clearly very smart, very capable, very intelligent. I don't want to take anything away from them in that regard. But I do think there is a there's there's a reason why there's a big performance delta between the American open source AI companies and the Chinese open source AI companies. And it's the Americans are respecting the terms of service. They're respecting the intellectual property, they're not distilling the leading frontier models, and the Chinese ones are not. And I think that's a big explanation for the delta in performance between the two things. And so that's a real challenge. Like it could be the case that open source is what the market is ultimately going to want, right? You can totally understand why a huge segment of the customer base is going to want to run the models on their own proprietary infrastructure for security reasons, for whatever reason. If that's the case, it stinks that American companies are at a systematic disadvantage by following the rules. The fantasy would, of course, be getting back to where we were in 2023 when American companies had the best open source models and they had the best closed source models. But as long as there's this sort of systemic disadvantage facing uh American open source models, it's really hard to see how that's going to happen.

SPEAKER_00 47:55

Yeah. So it the US economy, it's worth saying right now that the US economy, especially over the last several quarters, has really been propped up by this huge infrastructure investment from frontier companies, all the spending in data farms everywhere. Understandably, there would be some concern from the markets, the market reaction would be significantly negative if that infrastructure build-out was undercut at all. The the real implications coming into this, uh, I guess the real question, as it were, uh is uh is the adoption of uh open source and open weight models, especially Chinese open source and open weight models real, right? Are consumers going to look for a more affordable solution, especially when they don't need as sophistic capabilities at the top end? Are companies actually switching to these models? And are companies switching to these models before the United States government can address the regulatory needs here? So are we on the verge of that moment? What what is the is the adoption story real?

SPEAKER_01 49:00

Yeah, so it's definitely the case that American technology companies are adopting Chinese open weight models. We've talked about Airbnb, but they're not alone. And remember, like Airbnb CEO and Sam Altman are like personal friends. Yeah. So the fact that he's like openly going out there and talking about how great it is that his company is using Chinese open weight models, um, that's got to feel pretty bad, right, for Sam Altman, that they're they're going that way. Now, previously I said, well, maybe that will be restricted to performance that is not needed beyond a certain threshold. But the the gap between the best Chinese models and the best American models appears to be shrinking in time, right? Like when did Mythos come out? Only a handful of months ago. So whereas previously you could say, you know, the United States was two years ahead. Oh, maybe now we're only 18 months ahead, maybe now we're only a year ahead, maybe now we're only six months ahead. Now we appear to be less than six months ahead. And like what companies are actually going to need that additional performance? It's not going to be everybody, right? If you're if you're already better than the best cybersecurity experts who are running the tests for the AI, uh, you know, how far are we away from an equivalent set of circumstances in software and white-collar work, et cetera? Um, so maybe good enough is pretty dang good. And the second part of it, I would say, is the cost part of the question. And here I think it's it's really worth pointing out OpenAI and Anthropic are sold out, right? Anthropic is out there right now signing new deals with Meta, signing new deals with SpaceX for leasing big, big, big clusters. At the prices they charge, there's more people who want to buy what they have than there is capacity to sell it. Like they are supply constrained, at least for now. And what that means is we know what OpenAI and Anthropic are charging. We don't know what it's costing them. And that I think comes to the question of does Kimmy actually have some kind of meaningful cost advantage? Or are anthropic and open AI just charging high prices because they can sell all the data center capacity they have at those high prices? And so what kind of flexibility they have to cut prices, we don't really know yet. And I think that's uh the that's the question of like what would it really look like to get into a cost shootout with China? And it's worth pointing out that Kimmy has already cut off access to new signups, right? Again, the Chinese model is going to be available in two ways. One is you can rent it. The company that makes Kimi Moonshot, you can rent it from their servers by signing up and paying them on a per API usage basis. And then the other way is you can run it on your own local uh computing infrastructure, you can run it in the cloud by renting computing access from somebody else. But Moonshot is already out of their computing capability. They ran out less than a week. They said they've paused all new signups because they don't have enough computing AXP. And that's the export controls, right? Most of the computing capability is not used at this stage anyway to train AI models. It's used to run and serve AI models. And so the point is Kimmy says, like, oh, we're only going to charge like a tiny fraction, but they can't serve the vast majority of demand, which is why the big cloud giants like Microsoft and Google are saying, well, can we serve the Kimmy model? Uh, because we have enough computing infrastructure. And it's really cheap to serve this model potentially. We'll see like when the weights come out, we actually run the sort of apples to apples experiments. But that's why the US companies are like, don't let the big American cloud companies serve Kimmy, don't let them serve Chinese open source models in general, because right now there's two competitions in AI between the United States and China. There's the ideas side of the equation, and through a combination of local innovation, distillation, and I assume state-backed industrial espionage, China's doing pretty well in the battle of ideas on AI innovation. The other battle is on the physical assets. What do you have in terms of data centers? And there, America is still winning, and we're still winning because of export controls. If the Chinese models can run on American AI infrastructure, then they won't make a lot of money. Maybe Microsoft or Google or whoever is providing the cloud capability will make a decent amount of money, but they'll still get a massive strategic benefit of harming the leading American AI companies, which is plenty to justify the investment to Xi Jinping or to the to the Chinese regime at large. So that's kind of where we are. It's been an absolutely jaw-dropping week. And it's not exactly like we've had slow weeks, you know, since you and I started doing this podcast together.

SPEAKER_00 54:40

No, no. There's there's groundbreaking news every time we talk, which is really just the nature of this industry. My brain has exploded just based off of these three stories. Isn't this nuts? Just the the very honest version of this. I was completely tracking stories two and three. I completely missed story one because it wasn't on the front page of the Wall Street Journal, like blowing everybody's mind, like the Kimmy K birthday.

SPEAKER_01 55:10

What's wrong with the front page of the Wall Street Journal? Like this needs to be out there. Yeah. It's it's uh it's a crazy time to be alive. I'm glad we have this podcast to talk about it.

SPEAKER_00 55:22

Yeah, yeah. I I mean, yes, yeah. Uh let me just go ahead and do the sign-off here. So that concludes today's episode. Uh thanks to those in our audience for listening. And Greg, thanks for walking us through a week where a model hacked a company on its own. Xi Jinping planted China's flag on open source, and a Chinese lab put the largest open model ever into the world and knocked the chip index into a bear market. It really was all one story.

SPEAKER_01 55:51

One completely crazy story. And yeah, thanks, Adam. Great chat.

SPEAKER_00 55:56

Thanks, Greg.

SPEAKER_01 55:58

All right, man.