Researchers at artificial intelligence company Anthropic have warned that AI could pose an existential threat to humanity within the next decade, with one of them resigning in protest.
The latest warnings were made in social media posts on Tuesday by a former Anthropic researcher, who said he quit after becoming concerned that Anthropic and his previous employer, OpenAI, were either ignoring the risks posed by advanced AI or failing to address them adequately.
“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” wrote Jacob Coxon.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
The post drew responses from at least two other Anthropic employees, who supported Coxon’s warnings about the potential dangers of AI.
Evan Hubinger, a lead in Anthropic’s alignment division, which focuses on ensuring the company’s AI models operate in accordance with human goals, said Coxon was “correct”.
Hubinger added that the AI industry was falling behind in efforts to address the technology’s potentially catastrophic consequences.
“We really do earnestly believe AI could kill all humans!” Hubinger wrote. “I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Hubinger’s remarks represented a notable endorsement of Coxon’s bleak predictions from an employee who remains at Anthropic.
A second response came from Samuel Marks, Anthropic’s scalable oversight lead, who published a lengthy analysis while emphasising that the views expressed were his own and did not represent those of the company.
“AI developers believe their technology could cause human extinction (or similarly bad outcomes),” wrote Marks. “This could happen in the next few years. In general, the more senior the employee, the more concerned they are.”
The researchers’ warnings about the possibility of human extinction come amid more specific concerns raised by OpenAI chief executive Sam Altman over the growing cybersecurity capabilities of artificial intelligence.
