OpenAI puts the brakes on a new model because it’s supposedly too powerful

OpenAI says its in-development Astra model may have ‘critical’ cybersecurity capabilities.

Image: The Verge

OpenAI says it is pausing “internal activities” around an in-development AI model, Astra, because it doesn’t yet meet new security standards the company is putting in place. The announcement follows its recent disclosure that OpenAI models accidentally hacked Hugging Face. Anthropic and Meta have also since admitted that they had AI models that went rogue and breached other organizations.

Recent internal evaluations of an OpenAI model called Astra indicate that it offers “significant advancements in agentic coding and cybersecurity,” according to the company. “These results, in addition to expert assessments, have led us to conclude last night that we cannot rule out critical cyber capabilities under our Preparedness Framework⁠.”

Here is how OpenAI defines a “critical” cybersecurity threshold:

Astra was “not involved” in the Hugging Face breach, OpenAI says.

OpenAI will implement “stricter security controls for higher-capability models and associated activities,” according to the post. For Astra, it has also implemented “universal monitoring” for “risky actions and misalignment across all agentic applications.”