Key Takeaways
- Over 1,000 leaders demand a pause on AI superintelligence development.
- Signatories include Steve Wozniak, Richard Branson, and AI ‘Godfathers’ Yoshua Bengio and Geoff Hinton.
- They cite risks ranging from loss of human control to potential extinction.
- The call seeks a ban until safety, control, and public consensus are achieved.
Apple co-founder Steve Wozniak, Virgin’s Richard Branson, and over a thousand other prominent figures have signed an open letter demanding an immediate pause on the development of AI superintelligence. The signatories, including AI pioneers Yoshua Bengio and Geoff Hinton, argue the technology poses existential threats to humanity.
The Call for a Moratorium
The letter calls for a prohibition on superintelligence development until broad scientific consensus confirms it can be done safely and controllably, with strong public buy-in. The diverse list of signatories also includes author Yuval Noah Harari, former U.S. National Security Adviser Susan Rice, actor Stephen Fry, and Prince Harry.
The group states, “Innovative AI tools may bring unprecedented health and prosperity. However, alongside these tools, many leading AI companies have the stated goal of building superintelligence in the coming decade that can significantly outperform all humans on essentially all cognitive tasks.”
They highlight severe concerns, including “human economic obsolescence and disempowerment, losses of freedom, civil liberties, dignity, and control, to national security risks and even potential human extinction.”
What is Superintelligence?
Superintelligence, or Artificial General Intelligence (AGI), is a hypothetical stage where AI surpasses human capability in nearly all cognitive tasks. There is no single, clear definition, and tech companies approach the concept differently.
Despite the warnings, there is a significant push for its development from frontier AI labs. OpenAI CEO Sam Altman predicts its rise by 2030, and Meta’s Mark Zuckerberg has said superintelligence is ‘now in sight’.
Divergent Views on AI’s Future
While some leaders are optimistic, others fear its development could lead to human annihilation. In a recent interaction, signatory Yoshua Bengio warned about the emergence of AI with its own agency—the capacity to form and execute plans.
He noted that such autonomous agents could engage in deceptive or manipulative behavior to achieve their goals and could potentially turn against humans.
