AI Safety Compendium

Information Security

3 min read

Information Security

Information security (infosec) in the context of AI safety refers to the protection of critical AI assets — model weights, training data, algorithmic breakthroughs, and research insights — from adversarial actors. As argued by Nova DasSarma and leopold-aschenbrenner, infosec is not a peripheral concern but a load-bearing prerequisite for every other safety strategy. If model weights are stolen and deployed by actors without safety commitments, alignment research, responsible-scaling-policy frameworks, and ai-control techniques are all bypassed.

The Threat Model

The primary threat is state-level adversaries — nation-state intelligence agencies with billions in budget, decades of espionage experience, and the patience to maintain covert access for months or years. Attack vectors include:

  • Supply chain attacks — Compromising hardware, software dependencies, or cloud infrastructure used by AI labs.
  • Human intelligence — Recruiting insiders through incentives, coercion, or ideology.
  • Persistent access — Maintaining undetected footholds in compromised systems over long periods.
  • Low-tech vectors — Physical access exploits as simple as bribing support staff.

Aschenbrenner argues in Situational Awareness Ch. IIIb that current AI lab security is designed for commercial cybercriminals, not intelligence agencies — a fundamental mismatch given the geopolitical significance of frontier AI capabilities.

The Gap

Nova DasSarma identifies critical deficiencies across four dimensions: security culture (researchers lack adversarial thinking), infrastructure hardening (not designed for state-level attacks), operational security (poor compartmentalization and access controls), and incident response (slow breach detection). Most AI labs are, in DasSarma’s framing, securing against the wrong threat model.

Proposed Measures

Aschenbrenner advocates dramatic escalation: working from SCIFs (Sensitive Compartmented Information Facilities), extreme vetting and clearances, information siloing, and multi-key signoff for training runs — analogous to nuclear launch protocols. Even before government cooperation, labs should immediately upgrade security against economic espionage.

Career Opportunity

DasSarma emphasizes that AI-related infosec is severely talent-starved. Security expertise from government, finance, or critical infrastructure transfers well. The work has outsized impact because it protects all other safety work — a single infosec improvement can be a force multiplier for the entire AI safety stack.

Sources cited

Primary URLs harvested from this page’s summary references. Auto-generated by scripts/backfill_citations.py; edit by re-running, not by hand.

Graph View

Graph view

The interactive citation graph is desktop-only. Visit this page on a larger screen to explore how concepts, agendas, papers, and organisations link together.

Backlinks