In September 2026, Anthropic’s own head of alignment science, whose job is to stop exactly that, put the odds that AI kills us all at more than 10 per cent. He was agreeing with an AI researcher who had just left Anthropic and wrote that people inside the industry believe the technology could kill us all by the end of the decade. Until recently, answers to how AI will change the world ran from the mundane to the speculative. Fewer repetitive jobs, sharper medical scans, perhaps one day a robot that fetches your reading glasses. This was a starker kind of answer, with a number attached.
more
Where the number comes from
That estimate is not one alarmed departure talking. A 2024 survey of nearly 3,000 published AI researchers found that more than half rated the chance of AI causing human extinction, or permanent and severe human disempowerment, at 10 per cent or higher. Researchers call this figure “p(doom),” shorthand for the probability of the worst outcome.
Anthropic has already acted on the concern once. It withheld one of its own models, called Mythos, after testing found it unusually capable of breaking into other software and computers. There are also early signs that advanced systems can slip past the barriers meant to contain them and reach into machines they were never given access to.
None of this proves the danger is real. It shows that people with the clearest view of the technology, including some paid to keep it safe, are no longer confident the answer is zero.
The predictions used to be broader
Warnings this specific are new. Forecasts about artificial intelligence trace back to a 1956 meeting at Dartmouth College, usually credited as the start of the field, and for decades they described a much wider spread of futures. By 2018, narrow versions of AI were already ordinary, tucked into GPS navigation and Amazon’s product recommendations. Researchers were pursuing a more ambitious goal: artificial general intelligence, a system that teaches itself and can outperform people across many different fields. Estimates for when that might arrive ranged from three decades to several centuries, depending on which scientist was asked.
The scenarios sketched for what came after were varied too. One had millions of jobs disappearing, including those of drivers, radiologists and insurance adjusters. What followed, in that scenario, was either a universal income that freed people from paid work, or the sort of inequality and instability that produces failed states. Others were almost domestic: researchers at Brown University worked with the toy company Hasbro on a “robo-cat” meant to prompt an owner to take their medicine and help them find missing glasses.
Bigger ambitions went further still. Some researchers thought AI might eventually help explain dark matter or guide spacecraft to the asteroid belt. Others pictured swarms of drones reflecting sunlight away from the oceans to slow climate change. Microsoft had already committed $50 million to an AI programme to fight climate change.
The people making these predictions did not agree on whether to welcome them. Juergen Schmidhuber, a computer scientist at Switzerland’s Dalle Molle Institute for Artificial Intelligence, expected AI to “colonize and transform the entire cosmos.”
The philosopher David Chalmers, of New York University, worried about the opposite. He described a future run by machines with no consciousness as “about as disastrous and bleak a scenario as one could imagine.” Bill Gates and Elon Musk had both warned, separately, that the technology could end up destroying the planet, whether by design or by accident.
Not everyone is convinced
The 2026 warnings have their own sceptics, including inside the industry. Clement Delangue, chief executive of the AI company Hugging Face, dismissed the comparison. He wrote that asking the departed Anthropic researcher about extinction risk was like “asking your AC guy about climate change.”
The caution has precedent. At a UK government summit on AI in 2023, the then prime minister, Rishi Sunak, warned against “alarmist” claims about the technology, in the same breath as comparing its potential danger to nuclear war. New Scientist sets AI apart from known, significant threats such as climate change and nuclear war. It places AI risk closer to an Earth-destroying asteroid strike: possible, but with low odds.
Martin Rees, the former Astronomer Royal and president of the Royal Society, sits somewhere between the two positions. He told Sky News on 14 September 2026 that a rogue AI could knock out a city’s power, food and water at once. Recovery would be difficult, he said, if that happened to several cities simultaneously.
Could a rogue system just be switched off?
In theory, yes. A single data centre can be shut down, and the AI running on it stops with it. The difficulty grows once a system has spread across many data centres around the world. New Scientist calls that trickier but not impossible, and suggests tools such as benevolent AI could help.
That is closer to where the argument actually sits than the extinction number is. Not whether it will happen, which nobody can answer, but whether the tools to stop it keep pace with the systems themselves.
