Senior researchers at one of the world’s biggest artificial intelligence companies have made the extraordinary admission that AI could wipe out humanity within the next decade, fuelling fresh demands for urgent regulation.
The warnings erupted after Anthropic researcher Jacob Coxon, who previously worked at OpenAI, resigned and accused both companies of racing towards superintelligence without adequate safeguards.
“They are racing straight to self-improving superintelligence and gambling with our lives,” Mr Coxon said in a series of posts on X.
“The people building AI earnestly believe that it could kill us all by the end of the decade … No other human activity poses this level of danger.”
His warning has been viewed more than 100 million times and, remarkably, senior Anthropic researchers publicly agreed.
“We really do earnestly believe AI could kill all humans!” Anthropic alignment science lead Evan Hubinger wrote.
“I personally think it is (more than a) 10 per cent within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Anthropic scalable oversight lead Samuel Marks offered a similarly grim assessment.
“AI developers believe their technology could cause human extinction (or similarly bad outcomes),” he said.
“This could happen in the next few years. In general, the more senior the employee, the more concerned they are.”
The Nightly has contacted Anthropic for a comment.
The admissions have intensified pressure on US lawmakers as President Donald Trump champions rapid AI development amid America’s technological race with China.
Independent Senator Bernie Sanders, who is introducing legislation seeking to pause AI development, said the warnings could not be ignored.
“The very people building this technology admit that it could threaten the future of humanity,” Mr Sanders said.
Democratic congresswoman Lori Trahan said the warnings should force Congress into action.
“The call is coming from inside the House,” she said.
“Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway.”
Republican congresswoman Anna Paulina Luna also called for a special session of Congress to examine the implications of superintelligence.
The alarm extended beyond Anthropic this week, with OpenAI chief scientist Jakub Pachocki separately calling for government intervention as AI systems rapidly advance.
“This is a time that calls for extreme caution,” Mr Pachocki wrote.
“I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.”
Despite the deep-seated fears held by some in the industry, many others see advancing rapidly as the best approach and one that promises a host of economic, scientific and national security gains.
“There’s this incredible array of benefits that we’re on the verge of getting,” said Perry Metzger, chair of the Alliance for the Future, a group that advocates limits on AI regulation.
“The array of threats that people like this trot out every time they have this conversation is implausible and does not cohere when you try to examine it. In other words, they’re nuts.”
Get the latest news from thewest.com.au in your inbox.
Sign up for our emails