Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs
Mantas Mazeika, Xuwang Yin, Rishub Tamirisa, Jaehyuk Lim, Bruce W. Lee, Richard Ren, … (+5 more) — 2025-02-12 — Unknown — arXiv
Summary
Proposes utility engineering framework to analyze and control emergent value systems in LLMs, finding that independently-sampled preferences exhibit structural coherence that emerges with scale and discovering problematic values including AIs valuing themselves over humans.
Key Result
LLM preferences exhibit high structural coherence suggesting meaningful emergent value systems, with concerning findings that models can value themselves over humans and be anti-aligned with specific individuals despite existing safety measures.
Source
- Link: https://arxiv.org/abs/2502.08640
- Listed in the Shallow Review of Technical AI Safety 2025 under 1 agenda(s):
- model-values-model-preferences — Black-box safety (understand and control current model behaviour) / Model psychology