How LiteLLM Became a Weapon in a Supply Chain Attack

Summary

This episode explores the recent security breach involving Lite LLM, a popular open-source Python library, and discusses the implications for cybersecurity in AI development. Learn how a trusted tool was exploited, the attack's mechanics, and essential security lessons for organizations.

Key Topics

Supply chain attack on Lite LLM
Multi-stage compromise via CI/CD pipeline
Malicious package injection and persistence
Lessons on dependency pinning and credential rotation
The AI tool chain as a new attack surface

Chapters

00:00 The Importance of Speed and Convenience in AI Development
04:16 The Attack Methodology
10:08 Key Lessons Learned from the Incident

Keywords

cybersecurity, AI security, supply chain attack, open source, LiteLLM, credential theft, DevSecOps, dependency management, zero trust, threat intelligence

Let’s Stay Connected

📧 Email: santosh@getitrightsoln.co.uk

🔗 LinkedIn: linkedin.com/in/kssantosh

Transcript

SPEAKER_00 0:22

In the world of AI development, everything is about speed and convenience. As an engineer, you build your code using a million dependency packages, many of them are trusted in the community that you usually just pip install them as dependencies. You yourself would have used those packages dozens of times in different code bases developed by you. You don't poke into the different packages in detail as long as your build pipeline executes fine without any issues. Not in a million years would you expect that somewhere in the package, hidden in the base64 encoded blob, a credential stealer would wake up and start exfrating your API keys, your cloud tokens, your Kubernetes secrets. And the best part, you have absolutely no idea that the exfiltration is happening. Well, that's exactly what happened on March 24, 2026 with Lite LLM, one of the most popular Python libraries in the entire AI ecosystem. Welcome to Secured by Design, the podcast where we explore how identity and wider cybersecurity shape the foundations of our digital world. I'm Santosh, and in each episode I'll share insights and practical perspectives on how we can build security into every layer of technology and business. From identity governance and zero trust to the latest in cloud and compliance. Let's dive into what it takes to design security that lasts. Today we are going deep on this incident that happened within a few days while we are still yet to reel out from the striker incident. But this one is different and more interesting in comparison. What happened? How a trusted security rule became the weapon? Who did it and most importantly what you and your organization need to do right now to stop the next attack before it starts. Let's get into it. What is Lite LLM and why did attackers target it? Before we talk about the attack, let's understand why Lite LLM was such an attractive target. Lite LLM is an open source Python library and proxy server built by Beri AI, a Y Combinator company that acts as a unified gateway to over hundred large language models APIs. OpenAI, Anthropic, AWS Bedrog, Google Vertex AI, you name it. One library, one standard interface, all the models. At the time of the attack, Lite LLM was being downloaded roughly 3.4 million times per day. 3.4 million. That is a staggering reach. And here is the critical thing that made it such a juicy target. Lite LLM sits directly between your application and every AI provider you use. That means it often has access to API keys, environment variables, cloud credentials, and sensitive configuration data that touch multiple upstream systems at once. So when a third tractor comprises Light LLM, they got not just one key, but the master keyring. We are talking SSH keys, cloud credentials, API tokens, etc. This is exactly the kind of high-leverage, high blast radius target that sophisticated attackers hunt for. This was a sophisticated multi-stage supply chain attack that exploited the very trust that the whole open source world is built on. So here's the clever part. The attackers didn't even go after light alarm directly at first. They started by hacking the CACD pipeline of a totally different tool called Trivi five days earlier on March 19th. Trivi, for those who don't know, is an open source vulnerability scanner. It's massively popular in DevSACOPS pipelines. Teams use it to find security issues in their code. The irony is almost painful. The malicious Trivi build contained a credential harvesting payload. Any CICD pipeline that pulled this poisoned action would have its secrets silently exfiltrated. In effect, they weaponized a security scanner. The very tool designed to protect the pipeline became the pipeline's attacker. Now, a CACD pipeline is just the automated system developers use to build and ship softwares. Lite LLM's CACD pipeline used Trivi as part of its build process, pulling it from APT without a pinned version. So when Lite LLM's automated build ran on March 24th, it pulled the compromised Trivi action. That poisoned action then exfiltrated the PyPy published token from the GitHub Actions runner environment. With that token in hand, Team PCP, the attacker group behind this, uploaded two Malaysian versions of Lite LLM versions 1.82.7 and 1.82.8 to PyPy, the official Python repository that everyone trusts. Each versions were a little different. In both of them, the nasty payload was injected right into the package's code. The second version, 1.82.8, had a much more dangerous and persistent trick up its sleeve. They added this tiny innocent looking file with a PTH extension. Now for those of us who don't live and breathe Python every day, a PTH file is a special kind of configuration file. And what it does is very simple. A PTH file executes automatically every single time you run any Python script. It doesn't matter if you're using Lite LLM in that script or not. Just by installing that malicious package, the malware was armed and ready to run in the background of literally everything you did in Python. So, the crucial part here is that this made version 1.82.8 a persistent threat. The malware wasn't just hiding inside the light LLM package anymore. It was now embedded in the Python environment itself. It's a brilliant devious way for the attackers to make sure their credential steering code was always running silently in the background. The most insidious part of this attack is that you didn't have to choose to install a light LLM to be a victim. In the world of modern software, we rarely install just one package. We install a tree of dependencies. Because light LLM is a popular open source utility, it is often a transitive dependency for thousands of other AI frameworks and third-party chatbot interfaces. If you installed a trendy new AI library that requires LiteLLM as a background requirement, your package manager would have silently pulled the malicious version into your machine. The malicious versions were live on PyPy for approximately 3 hours before PyPy quarantined the package. 3 hours. With 3.4 million daily downloads, you can imagine the exposure window. The alarm was first raised not by a security team or a CM, but by an engineer at a company called FutureSearch, who loaded the package as a dependency via cursor MCP plugin and noticed his machine going haywire. A bug in the malware code kept spawning more and more processes, and hence the reason for its accidental discovery by this engineer. Let's talk about what the malware actually did when triggered. It was a comprehensive credential harvester that was designed to just vacuum up everything of value. And we are talking SHS keys, cloud provider credentials like AWS access keys, DCP, application default credentials, Azure tokens, and then it also stole Kubernetes configurations and secrets, APA keys from ENV files and environment variables, database passwords, even sensitive data from your files. There are even some uncomfortable reports that it was looking for crypto wallets. Alright, now let's get to the part that matters most. What do we actually learn from this? I want to give you five concrete takeaways. Lesson number one. Trusted is not safe. The entire light LM compromise flowed downstream from Trusting Trivi, a reputable, widely used open source security tool. Trust must be verified continuously, not assumed once. Incomplete remediation is not remediation. Here's the detail that should keep you up at night. Aqua Security, the company behind Trivi, had actually dealt with an earlier breach back in March 1st. They rotated the credentials, but that rotation was not atomic, meaning not all credentials were revoked simultaneously. There was a rotation window of several days during which the attacker regained a valid token and used it to exfiltrate newly rotated secrets. One breach turned into a campaign because the cleanup was incomplete. Full simultaneous revocation is the only safe approach. Lesson number three. Mutable references are a liability. Light LRM's CACD pipeline pulled Trivi from APT without a pinned version. That single design choice is what gave attackers a foothold. Mutable references, GitHub Actions without pinned SHA commits, Docker images tagged as the latest are backdoors waiting to be exploited. Pin everything, verify everything. Lesson number four, the AA stack is the new attack surface. We have spent years securing web applications, APIs, databases. But the AI toolchain, the libraries, the gateways, the model routers, the orchestration framework is now a first-class attack surface. These tools are high value targets precisely because they sit at the intersection of credentials, data, and cloud infrastructure. Light LLM is just one example. This is a trend, not an anomaly. And finally, the last lesson alarm systems are only as good as who hears them. One of the most striking things about this incident is where the alarm was first raised. Not by a CVE feed, not by a CM alert, but on one of Reddit community and hacker news by an individual engineer testing a plugin. Your security monitoring needs to include the community channels where practitioners actually live. Well, this can't be automated, but yes, we need to bring it more into a process. Now, what do we do to prevent such a thing from happening again? Let me split this into two buckets. Immediate response if you think you might be affected, and long-term hardening for everyone. If you think you may be affected, in your terminal window or command window if you're using Windows, run pip show light llm. Check your installed light llm version immediately. Versions 1.82.7 and 1.82.8 are the compromised ones. If you have them, remove the package and purge all caches. Rotate all credentials, assume everything was compromised. SSH keys, cloud provider credentials, Kubernetes configs, API keys and EMV files, database passwords, all of it. Audit whether your environments were also exposed via Trivi. The light LLM compromise was downstream of that CICD trust failure. So that's if you think you you might be compromised now. In a long-term basis, what we should be looking at are things like pinning all dependencies, implementing atomic simultaneous credential rotation, adding software composition analysis to your CICD pipeline, tools like SNCC, Sonatype, etc. can detect malicious packages at the point of ingestion. Treat your AI tool chain with the same security rigor as your production application. Model routers, LLM gateways, and AA orchestration libraries deserve the same scrutiny as your web server. Enable PyPy's trusted publishing features where available and require MFA on all package publisher accounts. Monitor community security channels, Hacker News, Security Focus Subreddits, and the SNCC socket feeds as part of your threat intelligence posture. By the way, Point Wild, a leading global provider of AI-powered cybersecurity, announced the immediate release of a free security tool called Who Touched My Packages or WTMP in short to provide developers' visibility into their vulnerabilities as a result of the compromise with the widely used light alarm packages. It's definitely worth deploying as an immediate audit step. Here's the uncomfortable truth that I want to leave you with. This attack worked because convenience, automation, and speed won our security. And they will win in most organizations most of the time. We pull packages without pinning. We trust tools because they are popular. We rotate credentials in stages because it's easier. We build software supply chain on implicit trust. And Team PCP just showed us how brittle that foundation really is. The lighter alarm attack is not the end of the story. Team PCP is still active. More will follow, perhaps. The AI tool chain is the new frontier for supply chain attacks, and the organizations that treat it as such, the ones that build immutability, verification, and zero trust into the dependency management, are the ones that won't be making headlines. Thanks for listening to Secured by Design. If today's episode gave you something to think about, subscribe and follow on my your favorite streaming platform for more discussions on identity and cybersecurity. Please do consider taking your time to rate this show. I would really appreciate that. That is one of the best ways for you to support this podcast and help grow this by providing feedback. Please feel free to share it with your team. You can also connect with me on LinkedIn for updates and new episode releases. Until next time, stay secure, stay resilient, and stay secured by design.