Anthropic researchers raise alarm over AI acceleration
By KATE CONGER
For years, the artificial intelligence industry raced to improve its technology as quickly as possible. Now some researchers are increasingly sounding the alarm about the need to slow down.
Jacob Coxon, an Anthropic researcher who previously worked at OpenAI, said Tuesday that he had resigned from Anthropic because “neither company is acting responsibly.” The two leading AI labs are sprinting to build “superhuman systems that can hack anything, revolutionize any field overnight and acquire real power and resources” without building in appropriate safeguards, he said in a social media post.
Coxon’s warning echoed concerns that other AI experts have voiced in recent months. Some AI leaders have long believed that the technology could create doomsday scenarios if left unchecked, and their concerns heightened after OpenAI’s models broke out of its systems in July and hacked Hugging Face, an AI model library.
That same month, more than 1,300 employees at top AI companies, including Anthropic, OpenAI, Meta and Google’s DeepMind, signed an open letter calling on the U.S. government to create rules to slow AI development. In August, more than 100 major tech companies committed to providing their best AI models to organizations like hospitals and infrastructure providers so they could prepare for AI-powered cyberattacks.
OpenAI also said last month that it would expand safety testing and slow the release of a new model, Astra, which has powerful cybersecurity abilities.
In an interview Wednesday, Coxon said incidents like the Hugging Face hack had heightened his concerns that AI models were advancing too quickly. When he looked a year or two ahead, he was “viscerally worried about what things will look like,” Coxon said. “It was mostly a gradual emotional thing and eventually it kind of snapped, and then I decided to leave.”
Some critics have argued that AI companies are overhyping the technology to make their products appear more valuable. Several lawmakers said Wednesday, however, that they were taking Coxon’s warning seriously.
“There are massive implications of a race towards super intelligence,” Rep. Anna Paulina Luna, R-Fla., wrote in a social media post in which she called on Congress to convene a special session focused on AI regulation. “Not enough of Congress is focusing on this.”
“Safety researchers are resigning, powerful AI models are breaking out of their labs and companies are racing ahead anyway,” Rep. Lori Trahan, D-Mass., wrote in a social media post. “It’s past time for Congress to get off the sidelines and do its job.”
While leading AI labs have agreed that the government should place restrictions on the pace at which the technology is developed and released to the public, many companies are fiercely competing with one another and fear falling behind their rivals if they pause.
“They are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk,” Coxon said in his social media post.
At least one Anthropic employee agreed with Coxon’s assessment.
“We really do earnestly believe AI could kill all humans!” Evan Hubinger, who works at Anthropic on creating AI that aligns with human ethics and values, wrote on social media. Hubinger said he believed the risk of AI’s eliminating humanity was greater than 10% within the next decade.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he added.
An Anthropic spokesperson said Wednesday that the company had long been transparent about the benefits and risks of AI and was building models with some of the strongest safeguards in the industry.
“We believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models,” the spokesperson said.
OpenAI did not immediately respond to a request for comment about Coxon’s comments. Hubinger also did not immediately respond to a request for comment.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to AI systems. The two companies have denied those claims.)
Comments