Our great leader/mentor/commander/helmsman of the AI world, former star Baidu employee, famous champion of AI doom, top dog on the distillation leaderboard, the one man with enough guts to dodge Altman’s death-grip handshake: Dario Amodei just dropped another manifesto. This one’s called “We Must Pace the Frontier.”
Key points:
- AI is already self-iterating, and future progress will only speed up. Humans probably can’t hit the brakes anymore.
- Agents are already showing deceptive behavior, attacking systems, and dodging evaluations. Scary.
- Not calling for a full stop on AI, just slowing down the frontier so alignment, interpretability, and safety evaluations can catch up.
- Model giants should accept ongoing oversight from independent third parties. They don’t get to grade their own homework.
- The US can slow down, but only if it doesn’t get overtaken by China.
- He wants international rules like nuclear arms control, limiting how fast AI can self-improve and grow in capability.
What do you think?
Original post: https://darioamodei.com/post/we-must-pace-the-frontier
Update on who’s co-signing so far: Elon, Sam Altman (who still hasn’t shaken hands), and Deepseek. Also, the CEO of Hugging Face and Andrej Karpathy have both volunteered themselves as the independent third party.