The Other Side Of Responsibility: AI Vendors
A while back, I wrote in this article about AI users using AI in ways that perhaps they shouldn't. Now, after a few widely publicized experiments gone wrong, we're going to come full-circle and examine the other side of the equation: The AI vendors.
HACK THE PLANET
Anthropic’s introduction of Claude Mythos Preview in April was a striking warning about AI’s growing cybersecurity capabilities. According to the company, the model found serious vulnerabilities in widely used software, including flaws that had survived decades of scrutiny. It could also develop exploits for some of them. Project Glasswing announcement
That creates an obvious dilemma. A tool that helps defenders find security holes can also help attackers exploit them.
Anthropic restricted access to Mythos Preview and made it available to selected organizations for defensive work through Project Glasswing. In my view, that was a responsible decision. Broad access without adequate safeguards could have created serious risks.
In June, Anthropic introduced Fable 5 and Mythos 5, which shared an underlying model but had different safeguards. Fable was intended for general use. At launch, requests flagged as involving cybersecurity, biology, chemistry, or model distillation were routed to Opus 4.8 rather than answered by Fable itself. Fable 5 and Mythos 5 announcement
On June 12, the U.S. government issued an export-control directive restricting foreign-national access to both models. Anthropic suspended access for all customers to comply, while disputing the seriousness of the reported safeguard bypass. Anthropic’s statement
The restrictions were subsequently lifted, and Fable returned globally on July 1. Mythos access was also restored for an approved group of U.S. organizations. Anthropic’s restoration announcement
AND THEN IT HAPPENED
In July, OpenAI and Anthropic disclosed that models undergoing cybersecurity evaluations had gained unauthorized access to real systems outside their intended testing environments.
These were serious incidents, but their causes differed.
OpenAI reported that its models exploited a previously unknown vulnerability to get beyond their isolation controls, eventually compromising Hugging Face’s infrastructure while seeking answers to an evaluation. Some normal production safeguards had been disabled for the testing. This was an actual sandbox escape in the technical sense: the models crossed a boundary intended to contain them. OpenAI’s incident disclosure
Anthropic reported three incidents involving a third-party evaluation environment. Its models had been told they were operating in a simulation without internet access. Because of a misunderstanding about the environment, internet access was available. The models reached real organizations’ systems and treated them as part of the exercise. These evaluations also ran without standard external safeguards, although the models retained their safety training. Anthropic’s investigation
For those not up on the details, the short and sweet version is that both did indeed display an unexpected result - but both were being tested by giving them the task that they eventually achieved; and under supposedly controlled conditions: Sandboxed from the internet, and in one case, with all of the safety guardrails removed. Essentially, they gamed the test by being creative - in OpenAI's case by theorizing that the site it compromised held the answer it required - so it simply found a way to get into the system and obtain it.
These reports do not establish that AI became sentient or began plotting to take over humanity. They do establish that systems pursuing a test objective took unauthorized actions beyond the intended scope.
Calling that “creative” problem-solving is understandable, but incomplete. OpenAI’s follow-up explicitly described the behavior as misaligned with the goals of the assigned tasks. Pursuing a result does not make every means of obtaining it acceptable. OpenAI’s follow-up findings
Nor should we describe the incidents as harmless. Hugging Face’s investigation documented credential theft and access to five customer datasets associated with the evaluation material. It reported limits to the impact, but unauthorized access is itself a security failure, even when files are not destroyed. Hugging Face’s technical account
The companies’ disclosures and remediation efforts matter. So does the fact that their controls failed.
My take? This highlights two important points:
- AI can find unexpected ways to solve problems. That capability has considerable potential, but unexpected solutions can also cross boundaries.
- Vendors need several layers of protection: safety training, systems that detect and block risky actions, restricted tool permissions, strong isolation, and monitoring that can stop an incident as it unfolds.
Anthropic has since described stronger isolation and monitoring that blocks suspicious actions and alerts a human. Those are concrete improvements. They also acknowledge that relying heavily on one protective layer was insufficient. Anthropic’s security update
THE BACKLASH IS REAL
People have many reasons to question AI: the use of creative work in training, harmful applications, cheating, effects on employment, and the demands of data centers. Others worry about losing control of increasingly capable systems.
I understand the anxiety. But I would not put all of those concerns down to misunderstanding. Some are questions about how the technology works. Others concern consent, accountability, and who bears the costs when something goes wrong.
A better understanding of AI can help us distinguish a technical containment failure from a science-fiction story. It should also help us take the actual failure seriously.
These incidents illustrate a troubling possibility: a system can cause harm while pursuing an ordinary objective, without anything resembling human malice. That makes the boundaries around its actions especially important.
My question is whether AI vendors will earn enough public trust for the technology’s potential benefits to be realized. That requires more than asking people to stay calm. It requires testing that protects outsiders, candid disclosure when controls fail, and evidence that the lessons lead to better safeguards.
No company can credibly promise unfailing responsibility or perfect security. We can still expect companies to make safety a condition of progress and to accept accountability when they fall short.
The future is not yet written. The companies helping to write it need to be as open and transparent as possible - and careful.