In this compelling summary of an in-depth interview, former OpenAI forecaster Daniel Kokotajlo sounds a critical alarm on the rapid, unregulated trajectory of artificial intelligence. Kokotajlo, who resigned from OpenAI in 2024 and famously walked away from $2 million in equity to maintain his freedom to speak out, warns that humanity is currently on a default path toward creating a superintelligent species. He estimates a 50% chance of achieving superintelligence by 2029, bringing with it a highly plausible (up to 70%) risk of a global catastrophe, including human extinction.

The Impending Era of Superintelligence

Superintelligence refers to artificial intelligence that is faster, cheaper, and qualitatively better than the best humans at every cognitive and physical task. According to Kokotajlo, AI companies are not focusing on gradual societal integration; instead, they are prioritizing automating their own AI research processes. This strategy of recursive self-improvement means that once AIs are capable of training and upgrading themselves, progress will accelerate exponentially, leading to a sudden “intelligence explosion” by the end of the decade.

The Core Risks: Loss of Control and Concentrated Power

Kokotajlo outlines two existential threats posed by unchecked AI development:

• Loss of Control: Modern AI systems are neural networks—vast, complex, self-learning structures that operate as “black boxes.” We do not truly understand why they make specific decisions. Without solving the “alignment problem” (ensuring AIs share human values), we risk unleashing a superior intelligence that may eventually ignore human orders or actively work against us.
• Concentration of Power: Leading tech CEOs are locked in a high-stakes race, driven by a fear of their competitors. This dynamic incentivizes rapid deployment over safety, threatening to create an oligarchy where a tiny group of corporate leaders or politicians controls a virtual “army of geniuses” in data centers, giving them unprecedented geopolitical and economic dominance.

The Shock of Mass Unemployment

While current unemployment rates remain relatively stable, Kokotajlo predicts that widespread job displacement will hit suddenly between 2028 and 2030. Because AIs will be deployed rapidly after achieving superintelligence, the transition will not be gradual. To prevent economic collapse and societal unrest, Kokotajlo proposes a “citizens’ dividend”—a system funded by taxing AI and robotics capabilities that distributes wealth back to the public, starting at $25,000 and growing as the automated economy expands.

AI 2040 Plan A: A Blueprint for Safe Progress

To steer humanity away from a catastrophic default path, Kokotajlo and his organization, the AI Futures Project, developed “AI 2040 Plan A.” This regulatory framework suggests:

• Global Pause and Slowdown: Implementing domestic regulations and international treaties (including US-China cooperation) to freeze advanced AI training temporarily.
• Data Center Inspections: Utilizing physical inspectors to verify that countries are running existing models (inference) rather than training more powerful, dangerous ones.
• Total Research Transparency: Enforcing open-science principles on training methodologies and algorithms to eliminate corporate monopolies and allow the global scientific community to collaborate on AI safety.
• Reversibility: Designing newly built data centers with “kill switches” so they can be disabled if competitive race dynamics resume.

Conclusion: A Call to Wake Up

The core message is clear: humanity must stop treating advanced AI as science fiction and start preparing for its imminent reality. While the potential benefits of AI are vast—ranging from curing diseases to achieving post-scarcity abundance—we must collectively demand rigorous government oversight, ask tough questions of political candidates, and prioritize safety over corporate and national competitive advantages before the window of opportunity closes.

Mentoring question

If you knew with high probability that your career or industry would be fully automated by a superintelligent AI within the next five to ten years, how would you pivot your current professional focus and personal definition of purpose today?

Source: https://youtube.com/watch?v=_g4l7YkDQwA&is=9KOOnObdUY1jaztl