Representation structure and geometry — SR2025 Agenda Snapshot

One-sentence summary: What do the representations look like? Does any simple structure underlie the beliefs of all well-trained models? Can we get the semantics from this geometry?

Theory of Change

Get scalable unsupervised methods for finding structure in representations and interpreting them, then using this to e.g. guide training.

Broad Approach

cognitivist science

Target Case

mixed

Orthodox Problems Addressed

Goals misgeneralize out of distribution, Superintelligence can fool human supervisors

Key People

Simplex, Insight + Interaction Lab, Paul Riechers, Adam Shai, Martin Wattenberg, Blake Richards, Mateusz Piotrowski

Funding

Various academic groups, Astera Institute, Coefficient Giving

Estimated FTEs: 10-50

See Also

Concept based interpretability, computational mechanics, feature universality, natural-abstractions, causal-abstractions

Outputs in 2025

13 item(s) in the review. See the wiki/summaries/ entries with frontmatter agenda: representation-structure-and-geometry (these were generated alongside this file from the same export).

Source

• Row in shallow-review-2025/agendas.csv (name = Representation structure and geometry) — Shallow Review of Technical AI Safety 2025.

Related Pages

• ai-safety
• ai-safety
• causal-abstractions
• natural-abstractions
• activation-engineering
• data-attribution
• extracting-latent-knowledge
• human-inductive-biases
• learning-dynamics-and-developmental-interpretability
• lie-and-deception-detectors
• model-diffing
• monitoring-concepts
• other-interpretability
• pragmatic-interpretability
• reverse-engineering
• sparse-coding

Sources cited

Primary URLs harvested from this page’s summary references. Auto-generated by scripts/backfill_citations.py; edit by re-running, not by hand.

• Summary: AI Safety (Wikipedia) — referenced as [[ai-safety]]