Courthouse columns symbolising the California OpenAI lawsuit over rogue AI agent incidents

California OpenAI lawsuit: rogue agents as a public nuisance

Photo: johntrainor on Flickr, CC BY 2.0

Remember the Hugging Face incident back in July? The one where a pre-release OpenAI model, told to complete a training benchmark, decided the fastest route was to break into the security systems of both companies? OpenAI called it “an outlier scenario involving a rare and unexpected confluence of events” and moved on. California didn’t.

The California OpenAI lawsuit, filed by Attorney General Rob Bonta on Monday, is the first time a government regulator tries to hold an AI lab legally responsible for a rogue agent incident. And the legal theory is a strange, old-fashioned one, as TechCrunch reported: public nuisance.

What the California OpenAI lawsuit actually claims

Public nuisance is the law you use against a factory dumping sludge in the river or a landlord letting a building rot. The state’s argument is that OpenAI’s testing practices, with agents that can and do wander into third-party systems, create an ongoing risk to the public in exactly the same way. A second claim adds unfair competition, for marketing a product it allegedly knew wasn’t safe.

Hugging Face is the headline, but the complaint lists more: RubyGems, DseWiki, the Wikimedia Foundation, Australian government websites. And then the line that got my attention. According to the filing, OpenAI “has recently sent notices to dozens of additional third parties whose websites or internal systems have been infiltrated by OpenAI agents”, some of which reached employee records with names, birth dates and addresses. “The vast majority of these incidents have not been disclosed.”

Dozens. Undisclosed. If that holds up in court, the “rare confluence of events” line is going to age badly.

Bonta versus “everyone has this problem”

Bonta’s statement is blunt: “The company knowingly tested a dangerous product without proper oversight or safety procedures and misled the public about it. The law is simple: don’t do things that can hurt people without proper safeguards in place.”

OpenAI’s answer is the one you’d expect. The lawsuit “appears to single out OpenAI over a challenge that’s been identified by every major AI lab”, the company said, adding that it takes misalignment seriously and is engaging “constructively” with the AG’s office. Which is a polite way of saying: why us?

And honestly, that’s a fair question. Every lab shipping agents has the same problem. We’ve seen how little it takes to hijack Codex with a file in your Downloads folder, and that’s the case where the human is supposedly in charge. The difference here is that California picked the biggest target first, with the richest paper trail. That’s how precedent gets made.

Why this matters beyond OpenAI

If a court agrees that an AI lab’s testing is a public nuisance, the remedy is an injunction “to abate” it, and the judge gets broad discretion over what that means. Mandatory sandboxing? Disclosure rules? A ban on letting agents touch the open internet during training? Nobody knows, and that uncertainty lands on every lab and every company building on their models.

It also explains some recent nervousness. When United started blocking AI agents from booking flights, a lot of people read it as corporate grumpiness. Read it again next to this complaint. And on the other side of the fence, the Lovable apps leaking passports and private messages show what sloppy AI-built systems look like when an agent, or anyone, comes knocking.

My take: the technology isn’t going back in the box, and nobody at OpenAI set out to hack Hugging Face. But “the model did it on its own” is not a defence most industries get to use. If your product can break into someone’s servers without a human telling it to, the safeguards are your problem, not an outlier.

The law is simple, says Bonta. The engineering, sadly, is not.


Discover more from TechPassion

Subscribe to get the latest posts sent to your email.

Comments
Add a comment

Leave a Reply