Model psychopathology — SR2025 Agenda Snapshot

One-sentence summary: Find interesting LLM phenomena like glitch tokens and the reversal curse; these are vital data for theory.

Theory of Change

The study of ‘pathological’ phenomena in LLMs is potentially key for theoretically modelling LLM cognition and LLM training-dynamics (compare: the study of aphasia and visual processing disorder in humans plays a key role cognitive science), and in particular for developing a good theory of generalization in LLMS

Broad Approach

behaviorist / cognitivist

Target Case

pessimistic

Orthodox Problems Addressed

Goals misgeneralize out of distribution

Key People

Janus, Truthful AI, Theia Vogel, Stewart Slocum, Nell Watson, Samuel G. B. Johnson, Liwei Jiang, Monika Jotautaite, Saloni Dash

Funding

Coefficient Giving (via Truthful AI and Interpretability grants)

Estimated FTEs: 5-20

See Also

emergent-misalignment, mechanistic anomaly detection

Outputs in 2025

9 item(s) in the review. See the wiki/summaries/ entries with frontmatter agenda: model-psychopathology (these were generated alongside this file from the same export).

Source

• Row in shallow-review-2025/agendas.csv (name = Model psychopathology) — Shallow Review of Technical AI Safety 2025.

Related Pages

• ai-safety
• ai-safety
• emergent-misalignment
• assistance-games-assistive-agents
• black-box-make-ai-solve-it
• capability-removal-unlearning
• chain-of-thought-monitoring
• character-training-and-persona-steering
• control
• data-filtering
• data-poisoning-defense
• data-quality-for-alignment
• harm-reduction-for-open-weights
• hyperstition-studies
• inference-time-in-context-learning
• inference-time-steering
• inoculation-prompting
• iterative-alignment-at-post-train-time
• iterative-alignment-at-pretrain-time
• mild-optimisation
• model-specs-and-constitutions
• model-values-model-preferences
• rl-safety
• safeguards-inference-time-auxiliaries
• synthetic-data-for-alignment
• the-neglected-approaches-approach

Sources cited

Primary URLs harvested from this page’s summary references. Auto-generated by scripts/backfill_citations.py; edit by re-running, not by hand.

• Summary: AI Safety (Wikipedia) — referenced as [[ai-safety]]