'AI Will End Humanity'
In the space of one week, the people building artificial intelligence told the world it could end all of humanity. This month, Evan Hubinger, lead of Alignment Science at Anthropic, put in his own odds, “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” He also made a distinction: the risk from today’s models is low.
TL;DR
- AI researchers have voiced serious concerns about the possibility of AI causing human extinction within the next decade.
- Concerns are framed as 'P(doom)' (probability of doom), which has become a popular concept in the tech industry and a political tool.
- Critics argue that these claims are unscientific, unfalsifiable, and may serve to create a 'crisis cartel' among major AI companies.
- Proposed solutions, such as industry slowdowns and government regulation, are scrutinized for potentially benefiting established companies by creating barriers to entry.
- Voluntarily slowing down AI development by Western companies could cede a significant advantage to state-sponsored actors in adversarial nations.
- The article suggests that the real risks of AI might be in its inscrutable self-improvement and humans' inability to detect or correct flaws, rather than an AI power grab.
- Historical parallels are drawn to past fears of new technologies, emphasizing adaptation over outright rejection.