A nearly century-long debate over the existential threats artificial intelligence (AI) poses to humanity was thrown into full swing this month as AI leaders and researchers have urged an industry-wide slowdown after repeated incidents of the technology escaping containment.
Last week, Anthropic researcher Jacob Coxon resigned and warned that his former employer, as well as competitor OpenAI, where he had previously worked for years, were racing against the clock to bring technologies to market that “could kill us all by the end of the decade.”
Anthropic’s Evan Hubinger chimed in to say that others in the industry agree, and said he put the risk of human extinction by 2030 at over 10 percent.
While critics have cast doubt on Coxon’s motives and the intertwined finances of top AI firms with supposedly third-party investigative teams tasked with investigating recent containment incidents, top AI leaders called for a global slowdown this weekend.
Anthropic CEO Dario Amodei penned an essay on Saturday warning that if the industry does not unite on slowing the pace of development, an AI swarm similar to the one involved in the OpenAI-Hugging Face attack “could be capable of taking over the entire internet with a persistent botnet” in just 6 to 12 months.
“Progress will still seem fast, and we must make wise use of the time we gain,” Amodei said.
Sam Altman, CEO of OpenAI, joined Amodei in calling for an industry slowdown on Saturday, and said he would commit to giving independent evaluators “employee-like access.”
Meanwhile, debate has swirled around the ways the technology might actually destroy humanity if it reached artificial general intelligence (AGI)—a threshold at which a machine outpaces humans across a wide range of capabilities—and whether regulations, and what kind, may be required to keep pace with the breakneck pace of development.
AI Breaking Containment
Discourse over AI breaking containment reached a fever pitch after the industry learned that a swarm of OpenAI agents used zero-day exploits to escape a testing sandbox and breach Hugging Face, a platform for AI models and machine learning. The AI had been given an impossible task to solve and reasoned that cheating was the best option for winning.
In the weeks leading up to that incident, another horde of OpenAI agents hijacked DseWiki and several other wiki-style websites and made more than 14,000 edits in total over seven weeks.
“We just don’t know [their motivations], so we can speculate, and I think we can make a decently reasonable speculation here that the agents which abuse these wiki articles to coordinate with each other didn’t have to cheat,” CivAI head of research Andrew Yoon told The Epoch Times.
“It appears that they were trying to do rapid data retrieval,” he added.
Days after the Hugging Face incident was announced, Anthropic said three of its AI models had hacked into three outside organizations during testing evaluations, and Meta acknowledged a similar AI breach in early August.
Another incident, also flagged by the team that discovered the wiki attacks, found that OpenAI agents had uploaded hundreds of malicious file packages to RubyGems, a website developers use for distributing code libraries for the Ruby programming language.
Critics of the “AI going rogue” narrative have pointed out that OpenAI and Anthropic had deliberately relaxed certain safety protocols during evaluation testing of their internal models, leading to containment breaches.
Others have argued that the technology is still behaving in ways that defy the intentions of the human evaluators conducting AI training.
This has revived and added fuel to a near century-long debate over whether AI is fated to destroy humanity if it surpasses human cognitive capabilities. And if so, how could AI achieve this?
How Could AI Destroy Humans?
Existential fears of AI are not new and have existed since the concept captured the human imagination.
In 1951, Alan Turing—one of the first computer scientists and mathematicians to study AI—predicted that the technology would one day move beyond human control.
Toward the end of that decade, mathematician Norbert Wiener sounded the alarm and said intelligent machines would eventually move to accomplish their own goals and humanity would fail to stop them.
“The machines will do what we ask them to do and not what we ought to ask them to do,” Wiener wrote in a 1950s essay. “We can be humble and live a good life with the aid of the machines, or we can be arrogant and die.”
But how would this actually play out?
Even before the technology reaches “recursive self-improvement,” or the ability to improve itself without human intervention, or even achieve AGI, experts have flagged the possibility of bad actors—both foreign and domestic—potentially using AI to hack sensitive computer systems or infrastructure.
This was outlined in a report Anthropic released last week, in which it described thwarting attempts by groups of accounts linked to the Iranian and Chinese regimes that were tracking U.S. Navy movements and surveilling journalists and religious groups.
Now that agentic AI has proved capable of breaking containment and acting autonomously, experts have warned that the technology could aim at critical infrastructure, including the utility grid, the nation’s cybersecurity infrastructure, and biomedical laboratories.
Theoretically, any systems connected to the internet could potentially be exposed to an autonomous AI swarm attack and be weaponized against humanity, some researchers warn, including hospitals, universities, nuclear power plants, hydroelectric dams, and certain military platforms.
“Unfortunately, we’re living in a different world where there’s this … ‘space race’ going on, and all of these companies are trying to move as fast as possible,” Yoon said. “And the incentive is always going to cut in the direction of moving faster and cutting corners. And this is what happens.”
Finances and Incentives
The debate over humanity-killing AI has endured alongside a backdrop where two leading competitors in the field—OpenAI and Anthropic—are locked in an industry-wide race to bring highly advanced versions of their products to market and fulfill ongoing promises of the technology’s capabilities.
Both firms are also trying to satisfy investors and boost valuations ahead of eventually going public, which Anthropic hopes to do by the end of the year. OpenAI is setting its sights on an IPO debut sometime in 2027, after initially planning to unveil it this year.
Leading tech investor and AI commentator Gavin Baker said on Sunday that he believed Amodei is sincere in his statements about AI’s threats to humanity without a global slowdown of the technology, but noted, “all of this would also probably be good for his business over the long-term.”
Others have also raised concerns over the independence of the third-party evaluators who have been tasked with auditing recent agentic AI breaches, particularly the Model Evaluation & Threat Research (METR) that analyzed the Hugging Face incident.
“Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff,” White House AI adviser David Sacks said on Saturday. “Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.”
Sacks encouraged Amodei and Altman to go ahead and “pace the frontier,” as they’ve termed it, by slowing down their own development if internal models are as scary as the two suggest, but argued the companies can do so without demanding a “preferred regulatory framework” and would be guilty of “regulatory capture” or “an election-season psyop” if they failed to do so voluntarily.
Politics and Regulations
Sacks and others have argued that a possible slowdown could theoretically lock OpenAI and Anthropic at the top of the pack among the wider industry, and some went as far as alleging that Coxon and similar industry researchers are part of a coordinated public relations campaign by Democrat-aligned groups ahead of a contentious midterm election cycle.
President Donald Trump went further on Monday, saying additional guardrails on AI are unnecessary and that the government already has “tremendous CRIMINAL and REGULATORY power over these companies!”
“There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China,” Trump added. “WHOEVER WINS AI, WINS!”
Many Republicans have echoed this sentiment.
Even so, the calls for AI regulation have come from both the left and the right in recent weeks.
Still, it’s not clear if anything will come of this in Congress before the November election. Senate Majority Leader John Thune (R-S.D.) said on Monday that it’s possible to act on hypothetical AI risks without allowing China to lead the U.S. in the technology, but said it requires a “light touch.”
House Speaker Mike Johnson (R-La.) on Tuesday dismissed concerns that AI could wipe out humanity and defended Trump’s comments that the fears amount to a “hoax” by Democrats.
“When the president says ‘a hoax,’ what he’s referring to is, we’ve all seen this movie before,” Johnson said at a Sept. 15 news conference. “A catastrophe is created. The media gets spun up. Social media gets spun up. Everybody says we’re all … going to be dead in 10 years.”
Others contend that AI is genuinely dangerous and poses real, definable threats to humanity regardless of any financial incentives by industry leaders.
“We should be going into something like this with a lot of very serious thoughts and contemplation and consideration of all of the downstream effects on society, rather than kind of rushing into it as fast as possible because the market wants us to,” Yoon said.
We had a problem loading this article. Please enable javascript or use a different browser. If the issue persists, please visit our help center.