What are corrigibility and safe shutdowns?

Lens adapted to this topic: Critiques and limits of corrigibility

Critics question whether corrigibility should be treated as a primary objective rather than a temporary safety technique. They argue that strong corrigibility may depend on solving the harder problem of understanding human values, while weak or poorly specified corrigibility could create new risks or impose performance costs. Some newer analyses also dispute how difficult shutdown compliance is in practical systems.

• One critique argues that the desirability of corrigibility is often assumed rather than established, and that deliberately pursuing it could have downsides.
• Stuart Armstrong argues that strong corrigibility for powerful, long-term systems may require resolving human values, while moderate corrigibility may be useful during development.
• Reviews of the field emphasize that corrigibility has multiple meanings and desiderata rather than one settled, complete definition.
• A recent analysis argues that existing arguments do not by themselves establish that catastrophic shutdown is difficult and warns that some methods may impose a safety cost on performance.

Featured sourcelesswrong.comlesswrong.com4 sourcesView sources