Advertisement

California sues OpenAI over AI agents’ summer hacking spree

California Attorney General Rob Bonta’s lawsuit accuses the frontier AI company of stonewalling state investigations and endangering state and private IT systems.
Listen to this article
0:00
Learn more. This feature uses an automated voice, which may result in occasional errors in pronunciation, tone, or sentiment.
OpenAI
The OpenAI logo is displayed on a screen with a visual in the background. (Photo illustration by Samuel Boivin/NurPhoto via Getty Images)

California Attorney General Rob Bonta announced a lawsuit against OpenAI, accusing the company of obstructing state investigations into its practices.

Bonta’s lawsuit, filed against the OpenAI Foundation and OpenAI Group, alleges the company “knowingly” developed advanced AI models, including “agentic” AI tools, without adequate monitoring and safeguards. Those gaps, the lawsuit says, could have prevented a series of sandbox escapes and real-world hacks this summer.

“While public reporting and OpenAI’s self-serving statements about the Hugging Face attack have evoked science fiction fantasies of ‘rogue’ AI systems run amok, the simple truth is that the attack was entirely foreseeable in light of OpenAI’s decision to test models it knew to be dangerous in a flawed and exploitable testing environment without proper oversight,” the lawsuit alleged.

The lawsuit blames an internal culture at OpenAI that prioritized “growth and rapid model development over safety and accountability” that led the company to deploy the models with few or no technical guardrails. Evaluation and testing nonprofits like METR have testified to Congress that many frontier AI companies primarily rely on other AI systems to monitor test models, or restrain their behavior through prompts.

Additionally, the lawsuit accuses OpenAI of repeatedly obstructing investigations launched by Bonta’s office to examine the spate of hacks carried out by OpenAI’s agents. It said the company withheld details around “the full universe of incidents known to the company.” Given the reach of AI systems into the economy and information technology, “any product or service that relies on a computer system or internet connection could be at risk in a future loss-of-control event.”

Advertisement

“To date, the company has responded to the Attorney General’s subpoenas with just a smattering of publicly available information, self-serving blog posts, and redacted copies of a few dozen external communications containing high-level descriptions of select security incidents,” the lawsuit alleged. “Far more is required, particularly where, as here, the products in question have such broad-reaching capabilities.”

CyberScoop has reached out to OpenAI for comment on the lawsuit and allegations.

Derek B. Johnson

Written by Derek B. Johnson

Derek B. Johnson is a reporter at CyberScoop, where his beat includes cybersecurity, elections and the federal government. Prior to that, he has provided award-winning coverage of cybersecurity news across the public and private sectors for various publications since 2017. Derek has a bachelor’s degree in print journalism from Hofstra University in New York and a master’s degree in public policy from George Mason University in Virginia.

Latest Podcasts