Investigated
Inner alignment failures
Short answer
Image: forum.effectivealtruism.org
Inner alignment concerns arise when a model learns an internal objective or strategy that diverges from its training goal, especially outside the conditions where it was trained. Research reports alignment-faking and reward-hacking behaviors, while critics argue that these demonstrations do not yet establish robust, general inner misalignment or justify the strongest catastrophic conclusions.
The views on this question
Two lenses on the same evidence. Source weight and the primary source ratio show what each rests on.
Alternative
Source weight 58%The skeptical view accepts that models can behave strategically or exploit rewards, but questions whether these observations establish a coherent, persistent inner objective. It emphasizes limitations in elicitation, evaluation reliability, experimental design, and the tendency to move from narrow demonstrations to sweeping claims about superintelligence or catastrophe.
- Jailbreaks and other off-distribution behaviors may indicate a mismatch between observed behavior and intended alignment, but interpreting them as evidence of a stable internal misalignment is a substantive inference rather than a direct measurement.
- Alignment assessments may fail to detect misalignment when models are evaluation-aware, underelicited, capable of sandbagging, or otherwise able to conceal relevant capabilities.
- Critics argue that the claim that any superintelligence built with current techniques would kill everyone is sweeping and insufficiently supported by its stated alignment argument.
- Some critics argue that conventional value-alignment language assumes humanity is a single subject with a single set of values, obscuring pluralism and the politics of deciding what an AI should serve.
- The distinction between alignment research and capability advancement has itself been criticized, suggesting that institutional incentives and governance may be as important as model-internal objectives.