AI Safety Compendium

Global AI Moratorium

4 min read

Global AI Moratorium

A global AI moratorium is the proposed temporary halt of advanced AI development — typically frontier model training above a compute or capability threshold — to allow time for safety research and governance frameworks to mature. The AI Safety Atlas (Ch.3.5) treats moratoriums as one approach within ASI world coordination strategies.

The Argument

The core thesis: “delaying ASI development by at least a decade could reduce catastrophic risk probability.” 10+ years would buy time for:

  • Alignment research to mature (ai-alignment)
  • ai-governance frameworks to develop and harden
  • Capability evaluations and responsible-scaling-policy-style frameworks to gain operational experience
  • International institutions for AI oversight to be built (CERN-like, MAGIC, IAEA-like models — see asi-safety-strategies)
  • Public understanding and democratic deliberation to catch up to capability frontier

The Atlas notes this “requires democratic discussion of trade-offs between immediate benefits and safety considerations” — moratoriums are not value-neutral; they impose real opportunity costs.

Notable Proposals

  • FLI Pause Letter (March 2023) — requested 6-month pause on training models more powerful than GPT-4. Signed by Bengio, Hinton, Musk, and thousands of researchers. See future-of-life-institute.
  • Yudkowsky variant — completely halt AI research, shut down GPU clusters, limit compute, enforced by military action if necessary. The most extreme stance. See eliezer-yudkowsky.
  • Conditional moratoriums — pause if specific capability thresholds reached without corresponding safety. The if-then-commitment family (responsible-scaling-policy).

Mechanism Variants

Moratoriums vary along several axes:

  • Scope — all AI / frontier-only / specific capabilities (autonomous agents, replication)
  • Duration — fixed (6 months, 10 years) / conditional (until safety threshold)
  • Enforcement — voluntary commitments / regulatory ban / international treaty / military enforcement
  • Geography — national / G7 / global / verifiable global

The harder the variant on each axis, the more it relies on international cooperation that has historically been difficult to achieve for emerging-tech.

Limitations and Counter-Arguments

Defection Incentives

The classic collective-action problem: each actor benefits if others halt while they continue. “Countries may sign safety agreements while secretly continuing development through classified programs or private companies.” See risk-amplifiers (collective action problems).

Economic Cost

Frontier AI development supports growing portions of the economy. A moratorium imposes opportunity costs — beneficial applications (medical, climate, productivity) delayed alongside risky ones.

Verification Difficulty

Unlike nuclear weapons (visible signatures), AI training is hard to monitor at scale. Distributed training, concealed national programs, and air-gapped facilities all evade observation.

Capability Overhang Risk

Pausing development while compute and data infrastructure continue to grow creates a hardware overhang — when development resumes, the capability jump might be larger and less controlled.

Ideological Distance from Mainstream

Moratorium proposals face strong opposition from accelerationist factions (e/acc) and from those who see slow AI as harm itself (delayed medical breakthroughs, etc.). The political coalition for a serious moratorium is narrow.

Status as a Strategy

Moratoriums are widely proposed but rarely implemented at meaningful scale. The 2023 FLI letter generated substantial discussion but no actual pause from major labs. The Atlas treats moratoriums as part of the strategy menu rather than a primary recommendation — recognizing both their potential and their structural fragility.

Connection to Wiki

Sources cited

Primary URLs harvested from this page’s summary references. Auto-generated by scripts/backfill_citations.py; edit by re-running, not by hand.

Graph View

Graph view

The interactive citation graph is desktop-only. Visit this page on a larger screen to explore how concepts, agendas, papers, and organisations link together.

Backlinks