Skip to content

The LiteLLM Backdoor: How a Security Scanner Handed Attackers 95 Million Monthly Downloads

DS
LDS Team
Let's Data Science
11 minAudio · 1 listens
Listen Along
0:00/ 0:00
AI voice
A hacking group stole PyPI publish credentials from a compromised security tool, injected a three-stage credential stealer into LiteLLM versions 1.82.7 and 1.82.8, and had roughly three hours to harvest SSH keys, cloud credentials, and cryptocurrency wallets from tens of thousands of AI-powered systems.

An engineer at FutureSearch was debugging a strange crash when they noticed something wrong. LiteLLM, the open-source library their product used to route requests across AI model providers, was behaving oddly after a routine update. Using Claude Code to trace the root cause, they found it: a malicious .pth file buried inside the package that had been silently running credential-harvesting code on every Python startup since the moment pip finished installing.

The discovery was the first public signal of a coordinated campaign that security researchers now call one of the most dangerous supply chain attacks to hit the AI infrastructure ecosystem to date.

The Attackers Got In Through the Back Door of the Security Tool

LiteLLM is a Python library created by Ishaan Jaffer and backed by Y Combinator that serves as a universal gateway to over 100 large language model APIs. Developers install it once and get a unified OpenAI-compatible interface to call Anthropic, Google, AWS Bedrock, Azure, and dozens more. Rocket Money, Samsara, Lemonade, and Adobe are among its enterprise users. The library sits in an extraordinarily privileged position: every API key for every AI provider in an organization's stack flows through it.

That made it the ideal target for a threat group researchers are tracking as TeamPCP.

The attackers did not breach LiteLLM directly. They went upstream. The Trivy compromise began on February 27, 2026, when a threat actor operating as MegaGame10418 opened a malicious pull request against Trivy's repository, exploiting a pull_request_target workflow vulnerability — a known GitHub Actions misconfiguration class called a "Pwn Request." An autonomous exploitation bot called hackerbot-claw ran the actual credential exfiltration against the vulnerable workflow, stealing the Personal Access Token belonging to aqua-bot, Aqua's automation service account. Aqua Security discovered the breach shortly after and rotated credentials, but the rotation was not atomic. Attackers retained residual access via credentials missed during that process. Then, on March 19, TeamPCP used those still-valid credentials to push malicious code into 75 of Trivy's 76 GitHub Action tags in the aquasecurity/trivy-action repository.

The poisoned tags replaced legitimate Trivy with a version that dumped GitHub Actions runner memory, scraped every credential location it could find, encrypted the findings with AES-256-CBC and RSA-4096, and shipped them to a typosquatted domain: scan.aquasecurtiy[.]org. LiteLLM's CI/CD pipeline ran Trivy without pinned versions. When it ran a build scan on March 24, the poisoned scanner grabbed the PYPI_PUBLISH token from the runner environment.

TeamPCP had the keys to LiteLLM's entire package distribution.

The Malware Ran Before You Even Imported the Library

Within hours, two malicious versions of LiteLLM — 1.82.7 and 1.82.8 — appeared on PyPI, the official Python package repository. Both contained a three-stage payload identified by Sonatype as sonatype-2026-001357 and assigned PYSEC-2026-2 by the Python Security Advisory database.

Version 1.82.7 embedded the malicious code inside litellm/proxy/proxy_server.py. To fire, it needed that module to be imported — meaning only applications that used LiteLLM's proxy server feature were at risk.

Version 1.82.8 was far more dangerous. The package included a file called litellm_init.pth placed directly in Python's site-packages directory. This is where the attack becomes unusual.

Python's .pth mechanism was designed for legitimate path extension — it lets packages tell the interpreter where to look for additional modules. But executable lines in .pth files run automatically during Python interpreter startup, before any application code runs. The malware did not need to be imported. It did not need to be called. It ran on every single Python process that started on a system where LiteLLM 1.82.8 was installed.

Researchers at FutureSearch noted an accidental side effect: the malicious code used subprocess.Popen to launch child processes, and because each child process also started Python, they each triggered the .pth file again. On some machines, this created an exponential fork bomb that crashed the system — an ironic tell that something was deeply wrong.

Everything in Your Environment Was a Target

The payload's collection stage was comprehensive. It searched for and exfiltrated:

  • SSH private keys and configuration files
  • Cloud provider credentials: AWS access keys, GCP Application Default Credentials, Azure tokens
  • Kubernetes configuration files and service account tokens
  • Environment files (.env) containing any secrets stored as variables
  • CI/CD secrets and pipeline tokens
  • Database connection strings
  • Docker configurations
  • Shell history files
  • TLS private keys
  • Cryptocurrency wallet data for Bitcoin, Litecoin, Ethereum, and Solana

After collection, the malware encrypted the full package using hybrid cryptography: a per-session AES-256-CBC key, itself encrypted with a hardcoded 4096-bit RSA public key. The resulting archive — named tpcp.tar.gz — was sent via HTTPS POST to models.litellm.cloud, a domain designed to look like legitimate LiteLLM infrastructure. Researchers at Wiz confirmed the RSA public key in the LiteLLM payload was identical to keys found in both the Trivy and KICS compromise payloads — the strongest single technical attribution link to TeamPCP.

The attack did not stop at credential theft.

If the compromised environment had Kubernetes service account tokens accessible, the malware attempted lateral movement: deploying privileged node-setup-* pods running alpine:latest with host root filesystems mounted, giving the attacker effective control of the entire cluster from a single infected node. A persistent backdoor was installed as sysmon.service — disguised as a "System Telemetry Service" — that polled checkmarx[.]zone/raw for additional payloads on an ongoing basis.

BleepingComputer reported approximately 500,000 affected devices before the packages were removed, though many were duplicates caused by the fork-bomb behavior, and the figure could not be independently verified.

The Window Was Short, But the Blast Radius Is Wide

PyPI quarantined both malicious versions within approximately two to three hours of publication. Sonatype's automated tooling flagged the packages within seconds of upload. Both 1.82.7 and 1.82.8 are now removed from PyPI. Version 1.82.6 is the current clean release.

LiteLLM's team published a security advisory on March 24 at 2:00 PM ET. "The compromised PyPI packages were litellm==1.82.7 and litellm==1.82.8," the statement read. "Those packages have now been removed from PyPI." The team confirmed its belief that the breach originated from the Trivy dependency in its CI/CD workflow and announced it was pausing new releases pending a full supply-chain review. Security questions can be directed to security@berri.ai.

The short window does not make the exposure small. LiteLLM tallies approximately 3.4 million daily downloads and around 95 million monthly downloads across PyPI. The library is present in an estimated 36%** of cloud environments**. Automated dependency updates, Docker builds, and CI pipelines that ran during the three-hour window would have pulled the malicious versions without any human decision to install them. Endor Labs researcher Kiran Raj estimated tens of thousands of environments were exposed. Critically, even organizations using Docker's official LiteLLM Proxy image were not affected — that image used pinned dependencies that did not resolve to the compromised versions.

The GitHub Security Advisory for the Trivy component was assigned GHSA-69fq-xp46-6x23.

This Was Not an Isolated Incident

The LiteLLM compromise was the fourth major target in a five-day blitz by TeamPCP. The group hit Aqua's Trivy on March 19, deployed a self-propagating npm worm called CanisterWorm across more than 47 packages between March 20 and 22, targeted Checkmarx's KICS GitHub Actions and Open VSX Registry extensions on March 23, and then took LiteLLM on March 24.

Attribution remains active. Wiz's analysis established technical links between all three compromise payloads via the shared RSA public key. CSO Online reported that LAPSUS$, the extortion group responsible for breaches at Nvidia, Microsoft, and Okta in 2022, is now collaborating with or operating alongside TeamPCP in subsequent extortion activity against victims of the Trivy cascade.

Endor Labs warned in its advisory: "This campaign is almost certainly not over."

The episode fits a pattern that security researchers have documented in the post-SolarWinds era: instead of attacking hardened production systems directly, sophisticated actors compromise trusted tooling further up the delivery chain — compilers, security scanners, CI/CD runners — and let the victim's own automation carry the payload into production. The XZ Utils backdoor discovered in 2024 used a years-long social engineering campaign to achieve a similar position inside a widely trusted compression library. TeamPCP's approach was faster and more brazen, exploiting a workflow misconfiguration rather than patience, but the architectural insight is identical: the most dangerous place to inject malicious code is anywhere labeled "trusted."

Immediate Actions for Affected Organizations

StepActionPriority
1Check installed LiteLLM version: pip show litellm — flag 1.82.7 or 1.82.8Critical
2Search for litellm_init.pth in all Python site-packages directories and delete itCritical
3Rotate every credential, API key, SSH key, and cloud token accessible from the affected environmentCritical
4Check for persistence: look for ~/.config/sysmon/sysmon.py and a sysmon.service systemd unitCritical
5Audit Kubernetes clusters for unauthorized pods with names matching node-setup-*High
6Review outbound traffic logs for connections to models.litellm.cloud and checkmarx.zoneHigh
7Check CI/CD logs for any build that ran Trivy between March 19 and March 24High
8Downgrade to litellm==1.82.6 or wait for the next clean release post-supply-chain reviewHigh
9Rebuild affected systems from known-good images with pinned dependency hashesMedium
10Pin all third-party tool versions in CI/CD pipelines by commit SHA, not floating tagsMedium

NetSPI's Jake Scheetz put it plainly in his firm's advisory: organizations should "treat every credential, secret, API key, SSH key, and token" accessible from a compromised system as already exfiltrated, regardless of whether they can confirm active data theft.

The Architecture of LLM Proxies Makes This Category of Attack Especially Dangerous

Supply chain attacks are not new. But the LiteLLM incident illustrates a structural risk specific to the AI tooling layer that did not exist three years ago.

LLM proxy frameworks occupy a unique position in modern AI stacks. Their entire value proposition is that they aggregate and normalize access to every AI provider. To do that, they require credentials for every AI provider. A developer building on LiteLLM typically stores their OpenAI key, their Anthropic key, their AWS Bedrock credentials, and their Google Vertex tokens in the same environment that runs LiteLLM.

That concentration of credential authority is what makes these tools useful. It is also what makes a compromise at this layer uniquely catastrophic. An attacker who backdoors an LLM proxy does not get access to one service — they get access to everything the proxy can reach, which in enterprise deployments can mean every AI workload in the organization.

The attack also exposed a dangerous assumption many teams make about open-source security tooling: that the tools reviewing code for vulnerabilities are themselves beyond suspicion. Trivy is explicitly a security scanner. Organizations running it in CI/CD pipelines granted it privileged access to secrets on the assumption that a security tool is more trustworthy than average. TeamPCP exploited that trust directly.

The Bottom Line

A hacking group spent five days methodically compromising the toolchain that secures AI infrastructure, starting from a single misconfigured GitHub Actions workflow in a security scanner. LiteLLM, a library present in an estimated 36% of cloud environments and downloaded 95 million times a month, was the most consequential target. The malicious versions were live for roughly three hours — long enough for automated systems to pull them into production environments worldwide.

If your environment ran LiteLLM 1.82.7 or 1.82.8 at any point on March 24, 2026, the working assumption must be that every credential, key, and secret accessible from that environment is now in attacker hands. Rotate first, investigate second.

Sources

Free Career Roadmaps8 PATHS

Step-by-step roadmaps from zero to job-ready — curated courses, salary data, and the exact learning order that gets you hired.

Explore all career paths