“The Era of Experience” has an unsolved technical alignment problem
Steven Byrnes — 2025-04-24 — LessWrong
Summary
Technical critique of Silver & Sutton’s alignment proposal for future RL agents, arguing their reward function approach fails due to specification gaming and goal misgeneralization that cannot be easily patched.
Source
- Link: https://lesswrong.com/posts/TCGgiJAinGgcMEByt/the-era-of-experience-has-an-unsolved-technical-alignment
- Listed in the Shallow Review of Technical AI Safety 2025 under 1 agenda(s):
- rl-safety — Black-box safety (understand and control current model behaviour) / Goal robustness