AI Safety Compendium

Ben Garfinkel

2 min read

Ben Garfinkel

Ben Garfinkel is a research fellow at the future-of-humanity-institute at Oxford University and one of the most prominent constructive critics of classic AI risk arguments within the ai-safety community. His work is notable for calling the community to higher epistemic standards while still affirming that AI risk is an important priority.

Constructive AI Risk Skepticism

Garfinkel’s most distinctive contribution is his systematic critique of traditional AI existential-risk arguments. He identifies several weaknesses in the canonical case for AI doom:

  • Fuzzy concepts — Many risk arguments rely on terms like “optimization power” or “general intelligence” that lack rigorous definitions, making it difficult to evaluate whether the arguments using them are actually valid.
  • Toy experiments — Risk arguments are sometimes supported by thought experiments or simplified scenarios that may not scale to real-world AI systems.
  • Assumptions of sudden capability jumps — Classic scenarios often assume a rapid “hard takeoff” from below-human to far-above-human capability. Garfinkel argues the evidence for such discontinuities is weaker than often presented, and more gradual transitions change the risk calculus significantly.
  • Need for clearer evidence — The community should develop arguments based on concrete evidence and testable predictions rather than abstract reasoning.

Crucially, Garfinkel does not argue that the risks are low — rather, that the community should hold itself to higher standards. He distinguishes between “AI risk is important” (which he affirms) and “the specific classic arguments for AI risk are strong” (which he scrutinizes).

Broadening the Risk Landscape

Beyond alignment, Garfinkel highlights political and military risks from AI that receive less attention in the safety community: military applications raising war risks, accidental use of force by autonomous weapons, and great power conflict if one side achieves a decisive AI capability advantage. He argues that ai-governance and policy may address these risks more effectively than technical alignment research alone.

Practical Implications

Garfinkel advocates for diversifying the research portfolio beyond alignment, strengthening arguments with empirical evidence, actively seeking out critiques to avoid groupthink, and investing in governance alongside technical work.

Graph View

Graph view

The interactive citation graph is desktop-only. Visit this page on a larger screen to explore how concepts, agendas, papers, and organisations link together.

Backlinks