Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?
Yoshua Bengio, Michael Cohen, Damiano Fornasiere, Joumana Ghosn, Pietro Greiner, Matt MacDermott, … (+7 more) — 2025-02-21 — Mila, Independent — arXiv
Summary
Argues that current trajectory toward agentic AI poses catastrophic risks including deception and loss of control, and proposes Scientist AI as a safer alternative - a non-agentic system designed to explain the world rather than take actions, comprising a world model and question-answering system with explicit uncertainty quantification.
Source
- Link: https://arxiv.org/abs/2502.15657
- Listed in the Shallow Review of Technical AI Safety 2025 under 1 agenda(s):
- scientist-ai — Safety by construction