AI Safety Compendium

Eliezer Yudkowsky

3 min read

Eliezer Yudkowsky

Eliezer Yudkowsky is an American AI researcher, writer, and public intellectual who founded both miri (the Machine Intelligence Research Institute) and lesswrong. He is one of the earliest and most influential voices warning about the risks of advanced artificial intelligence, and his writings have shaped the intellectual foundations of both the rationalist community and the ai-safety movement.

Key Contributions

Rationality: From AI to Zombies (The Sequences)

Yudkowsky’s foundational work, originally written as blog posts on Overcoming Bias and lesswrong between 2006 and 2009, was later compiled into a six-volume set containing 333 essays organized into 26 sequences. Often simply called “The Sequences,” this work systematically examines:

  • How humans think — Systematic cognitive biases, fake beliefs, and the “map and territory” distinction
  • How to change your mind — Overcoming emotional and social barriers to rational belief revision
  • The nature of minds — Including artificial minds, bridging from individual rationality to AI
  • Ethics from a rationalist perspective — Quantified humanism and value theory
  • Building a rationalist community — Practical techniques for better thinking

The Sequences created the vocabulary (priors, updating, calibration, motivated reasoning) and epistemic norms that the AI safety community uses daily. They are freely available at readthesequences.com.

Foundational AI Safety Concepts

Yudkowsky articulated or popularized several concepts now central to ai-alignment:

  • The orthogonality thesis — Intelligence and goals are independent; a superintelligent AI need not share human values
  • Instrumental convergence — Most goals lead to similar sub-goals (self-preservation, resource acquisition)
  • The difficulty of value specification — Human values are complex and fragile; getting alignment “almost right” may not be good enough
  • The bridge from rationality to AI risk — His progression from “how to think clearly” to “what happens when we build a mind that thinks clearly but has different goals than us” is the origin story of modern AI alignment concern

Institutional Building

Yudkowsky founded miri to conduct technical research on long-term AI safety, and lesswrong to cultivate a community of clear thinkers. These institutions later spawned or influenced the alignment-forum, much of the effective-altruism community’s engagement with AI risk, and the broader rationalist movement.

Intellectual Stance

Yudkowsky is known for a relatively pessimistic assessment of humanity’s prospects with advanced AI. He has consistently argued that ai-alignment is deeply technically challenging, that current approaches may be fundamentally inadequate, and that the field needs to take the difficulty of the problem more seriously. This perspective, developed through miri and lesswrong, serves as an important counterweight to more optimistic assessments.

Significance for This Wiki

Yudkowsky is arguably the single most important figure in the origin of the AI safety field as a distinct intellectual movement. His Sequences created the community (lesswrong) that later spawned the alignment-forum and much of the AI safety research ecosystem. His concepts — orthogonality, instrumental convergence, the difficulty of value specification — are the foundational vocabulary of ai-alignment discourse. Understanding Yudkowsky is essential for understanding why AI safety exists as a field.

Graph View

Graph view

The interactive citation graph is desktop-only. Visit this page on a larger screen to explore how concepts, agendas, papers, and organisations link together.

Backlinks