Forget 'superintelligence': error-prone AI nearly sparked world war three this month
The risks of AI aren’t what we think they are, as a recent security incident between China and the United States reveals
TL;DR
- A CNN report detailed how a US military intelligence report, generated by a chatbot, falsely claimed a Chinese ship was transporting nuclear weapons components, leading to plans for interception.
- This incident, which nearly escalated into war, was caused by an error-prone chatbot, not a superintelligent AI, and received less public and political attention than hypothetical AI extinction scenarios.
- The article criticizes the public and policymakers' focus on futuristic AI threats while overlooking the present dangers of relying on current large language models (LLMs) in critical applications like warfare.
- LLMs are described as 'stochastic parrots' that regurgitate patterns from training data, and their output should not be mistaken for true intelligence or infallibility.
- The authors advocate for regulating 'AI' systems as error-prone products, emphasizing the need for human accountability and thorough evaluation in high-stakes scenarios, rather than believing marketing claims of superintelligence.