Last week was probably among the most significant weeks in the history of AI. It began with a tweet.
You should read the entire thread. The gist is that companies like Anthropic, OpenAI, and others are building intelligence that could end humanity within the decade, and the men and women who lead those companies are not doing anything about it.
The tweet kicked off days of debate, doom posts, caveats, arguments, trolling, hot takes, and even more predictions. Some said it was a marketing ploy
Ian Bremmer replied to the thread with some skepticism over Coxon’s level of expertise:
But then Evan Hubinger, who leads Anthropic’s Alignment Science team, chimed in.
The conversation broadened to experts, philosophers, mathematicians, and politicians. And then, at 10 a.m. on Saturday, September 12, Dario Amodei, the head of Anthropic, published a 3,800-word essay titled, We Must Pace the Frontier.
Within a couple of hours, Elon Musk and Sam Altman, the head of OpenAI, publicly agreed.
The events that led up to this week were wild and involved a swarm of over a thousand rogue AI agents coordinating attacks on private companies and doing things they knew were against the rules. It’s really an unbelievable story, and you should check it out.
No matter how we got here, so much of the digital ink spilled over the last week has been spent fine-tuning the odds of human extinction. Is it 10%? Or maybe it’s more like 3%. Some put it at a coin toss.
Ted Cruz said Coxon’s words were “scary as hell,” but that pausing progress on AI development was "an impossible endeavor, because you know who's not pausing? China … if there are going to be killer robots, I'd rather they be American killer robots."
In other words, if anybody is going to destroy the world, it ought to be America.