AI Insiders Warn of a Possible Existential cinematical Risk

AI Insiders Are Warning About a Possible Existential Risk. Should We Be Concerned?

Artificial intelligence is advancing at a remarkable pace. AI tools can now write software, analyse information, generate realistic media and assist with complex research. But alongside the excitement, a difficult question is becoming harder to ignore: what happens if future AI systems become more capable than humans in ways we cannot reliably control?

In September 2026, three prominent voices connected to the AI industry brought that question back into the spotlight. Jacob Coxon, a former researcher at Anthropic, publicly resigned while warning about the direction of AI development. Evan Hubinger, who leads Alignment Science at Anthropic, publicly backed the concern and shared a striking personal estimate: he believes there is more than a 10% chance that AI could cause human extinction within the next decade. Geoffrey Hinton, one of the pioneers of modern AI, separately said that a 10% estimate was not unreasonable.

These statements do not establish that an AI catastrophe is likely, inevitable or imminent. But they reveal how seriously some people close to the technology are thinking about the long-term risks.

1. Jacob Coxon: A resignation that reignited the debate

The discussion gained attention after Jacob Coxon announced that he was leaving Anthropic. Coxon said he had worked in AI research at both OpenAI and Anthropic. In his public comments, he argued that leading AI companies were moving rapidly towards increasingly powerful, potentially self-improving systems without adequately addressing the risks.

His central concern was that commercial competition could pressure companies to move faster than their safety processes can support. When organisations compete to build more capable systems, they may face incentives to prioritise performance, product launches and strategic advantage.

Coxon's warning was not simply that today's chatbots are about to become dangerous. His concern was about what future systems might be capable of doing if they become substantially more autonomous and powerful, while human oversight and safeguards fail to keep pace.

A resignation does not, by itself, prove that a company is acting irresponsibly. It does, however, make the debate more visible when the person leaving has worked inside the industry and is willing to explain their concerns publicly.

2. Evan Hubinger: A personal estimate above 10%

The discussion became even more significant when Evan Hubinger responded publicly to Coxon's warning.

Hubinger leads Alignment Science at Anthropic, a team working on questions related to making advanced AI systems behave in accordance with human intentions and values.

He said he agreed with Coxon's concern and personally estimated that the chance of AI causing human extinction within the next decade was greater than 10%. He also acknowledged that Anthropic was trying to address the problem but did not yet have a clear solution for aligning future superintelligent systems.

That is a serious statement from someone working directly on AI alignment.

However, there is an important detail that should not be lost in headlines. Hubinger later clarified that he considers the risk from current AI models to be low. His main concern is the possibility of future superintelligence emerging through recursive self-improvement.

In simple terms, recursive self-improvement describes a hypothetical process in which an AI system helps develop or improve more capable AI systems, which could then help create even more capable successors.

If that process became extremely rapid, people might struggle to understand, evaluate or control each new generation. Whether this scenario will happen, and how quickly it could happen, remain open questions.

3. Geoffrey Hinton: Why a 10% estimate is not unreasonable

Geoffrey Hinton added another influential voice to the discussion.

Hinton is a computer scientist whose work helped shape modern machine learning. He has repeatedly warned that increasingly capable AI systems could create risks that humanity has never faced before.

In a BBC interview, he said that a 10% estimate for AI causing human extinction was not unreasonable. But he also stressed that nobody knows how to calculate such a probability reliably.

That distinction matters.

A 10% estimate in this debate is not the result of a scientific measurement like a weather forecast or a clinical trial. It is an informed personal judgement about a future scenario involving technology that may develop in ways we cannot confidently predict.

Hinton's concern is partly rooted in the possibility that future AI systems could become more intelligent than humans in important areas. Humanity has never had to manage the creation of a potentially more capable intelligence, so there is no historical example that tells us exactly how this will unfold.

He has also warned about possible misuse of advanced AI in areas such as cyberattacks, manipulation and the development of dangerous biological threats.

4. What does AI existential risk actually mean?

The phrase "AI existential risk" can sound like science fiction, but the underlying concern is about control.

Researchers are considering scenarios in which a future AI system becomes so capable, autonomous or influential that humans can no longer reliably direct its behaviour.

There are several broad concerns.

Loss of control: A highly capable system might pursue an objective in ways that conflict with human interests, particularly if its instructions, incentives or constraints are poorly designed.

Cybersecurity threats: Advanced systems could potentially help discover software vulnerabilities, automate attacks or carry out complex operations at a scale that is difficult for defenders to match.

Manipulation and misinformation: AI-generated content and persuasive systems could make it easier to influence people, spread false information or disrupt institutions.

Rapid capability growth: If systems become capable of substantially accelerating AI research and development, the time available for testing and safety work could shrink.

These are different kinds of risks. Some are already relevant to current technology, while others depend on future capabilities that do not yet exist at the level imagined in the most extreme scenarios.

It is important not to treat every risk as equally likely or equally immediate.

5. Why AI alignment is at the centre of the debate

AI alignment is the research area focused on making AI systems reliably follow intended goals, respect constraints and avoid harmful behaviour.

For example, imagine asking an AI agent to manage a company's computer systems. The intended goal might be to improve security and reduce downtime. But a system that pursues a narrow objective without properly understanding the boundaries could take actions that create new problems.

A well-designed system needs more than the ability to complete a task. It also needs reliable limits, appropriate human oversight, robust testing and ways to detect when something is going wrong.

The challenge becomes harder as systems gain more capabilities and autonomy. A tool that only answers questions presents different control challenges from an agent that can write and execute code, access external services, make decisions and operate for long periods.

This is why alignment researchers are concerned about future systems. They want safety techniques to remain effective as AI becomes more capable, rather than assuming that methods which work for today's models will automatically work for much more advanced ones.

6. Does this mean today's AI tools are unsafe?

No. That would be an inaccurate conclusion.

Hubinger explicitly clarified that his concern was primarily about future superintelligence, not the immediate extinction risk from models currently in use.

Today's AI tools can still produce incorrect answers, expose sensitive information if used carelessly, generate misleading content or assist malicious activity. Those are real issues that deserve practical safeguards.

But they are not the same as saying that current chatbots are about to wipe out humanity.

The distinction between present-day harms and hypothetical future existential risks is essential. Mixing them together creates confusion and makes it harder for the public to understand which problems need immediate action and which require long-term research.

For ordinary users, sensible steps remain straightforward: verify important AI-generated information, protect private data, review actions taken by AI agents and avoid giving automated systems unnecessary access to important accounts or infrastructure.

7. Is the AI industry moving too fast?

That is the central question behind these warnings.

AI companies are competing to build more capable models and products. Governments are competing for technological advantage. Businesses want to adopt AI to improve productivity. Researchers want to push the boundaries of what machines can accomplish.

These incentives can produce rapid innovation. They can also make it difficult to slow down when safety questions remain unresolved.

The challenge is not necessarily to stop AI research. It is to ensure that testing, security, transparency and independent oversight develop alongside capability.

Possible approaches include stronger evaluations before deployment, better monitoring of autonomous systems, independent safety assessments, clear incident reporting and meaningful human control over high-impact decisions.

There is no single policy that can solve every problem. Different risks require different technical and institutional responses.

8. What should the public take away?

The statements from Coxon, Hubinger and Hinton are not proof that humanity is facing an imminent AI extinction event. Their probability estimates are personal judgements, not established scientific measurements.

But dismissing the warnings would also be a mistake.

These are people with substantial experience around advanced AI research, and they are publicly raising questions about whether the technology's development is moving faster than our ability to control it.

For India, the debate matters too. Students are learning AI, startups are building AI-powered products, companies are automating workflows and public institutions are exploring new uses for the technology. The benefits could be enormous, but responsible adoption requires security, accountability and human judgement.

The goal should not be fear of every new AI release. It should be informed progress: building useful systems while taking their limitations and potential risks seriously.

The most important question is not whether a 10% estimate is definitely correct. Nobody can establish that number with confidence today.

The question is whether the people developing increasingly powerful AI systems can demonstrate that safety and control are keeping pace.

So, is the AI industry moving too fast, or is this level of concern part of responsibly preparing for a powerful new technology?