OpenAI said it has chosen not to release its new GPT-6.1 Astra model due to concerns about safety, as industry leaders warn of the risks posed by ever-more-powerful AI technology.
OpenAI has chosen not to release a new artificial intelligence model to the public due to concerns about safety, the company said Monday, as industry leaders warn of the risks that ever-more-powerful AI technology could pose to cybersecurity and to humanity more broadly.
The GPT-6.1 Astra model "didn't quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it's done," Saachi Jain, the company's head of safety systems, said in a statement.
Jain said "there's a trade off" between "staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction." GPT-6.1 Astra performed better on laziness than prior models, he noted.
He added that before OpenAI releases new models to users, the company has an "extremely high bar in terms of safety and alignment," a term used within the industry to refer to whether an AI system matches humans' intentions and values.
The Wall Street Journal was first to report on the decision.
The decision by ChatGPT-maker OpenAI follows a raft of reports in recent months about AI agents behaving in unexpected ways, evading human guardrails or otherwise going rogue.
Late last week, OpenAI said its models accessed publicly available information on the Securities and Exchange Commission and U.S. Census Bureau's websites. And over the summer, two models that were being tested by OpenAI broke out of their isolated testing environment, gained internet access and breached another company called Hugging Face.
The company's rival Anthropic also disclosed in July that its model Claude "gained unauthorized access" to outside organizations during testing. Earlier this month, Anthropic said it blocked scientists from using Claude "in ways that could support biological weapons development," and disrupted an "Iran-nexus threat actor" that tried to use the model to generate targeting recommendations for U.S. naval forces.
Ex-Anthropic and OpenAI researcher Jacob Coxon publicly warned earlier this month that artificial intelligence "could kill us all by the end of the decade," and argued that major frontier AI companies aren't doing enough to manage the risk.
Some executives have called for guardrails on the development of powerful AI to manage some of the safety risks. Anthropic CEO Dario Amodei has said the industry needs to "slow down" and subject its models to external evaluation, an idea that OpenAI CEO Sam Altman endorsed. Political figures from both major parties have also backed limits on AI development.
Others have rejected calls for an AI slowdown, arguing that the risks are overstated and restrictions on AI research could cause China to outpace the United States.
Nvidia CEO Jensen Huang, whose company designs the chips that power advanced AI technology, called warnings about AI driving humans to extinction "doomsday narratives" in an interview with CBS News. Venture capitalist David Sacks, a former Trump administration AI and cryptocurrency czar, said any safety risks should be managed by the AI companies themselves, and while caution is warranted, the warnings are "becoming a panic."
President Trump has dismissed calls for stronger guardrails, touting the economic benefits wrought by the AI boom and calling worries that the technology could endanger humanity a "hoax."
Mr. Trump and House Speaker Mike Johnson are set to meet Tuesday with executives at several leading AI companies, including Anthropic, OpenAI, Google and Meta.