Directing AI systems toward human-intended goals, values, and ethical principles.
Dario Amodei
We discuss five practical research problems in designing AI systems that behave safely: avoiding negative side effects, avoiding reward hacking, scalable oversight, safe exploratio...
Dylan Hadfield-Menell, Stuart Russell
To build systems aligned with human values, machines must act when human objectives are initially unknown. We formalize this cooperative setting as a two-player game with incomplet...
Nate Soares
A corrigible artificial agent is one that tolerates or assists in its own modification, correction, or shutdown by its human operators. This paper investigates theoretical hurdles ...
Benja Fallenstein
We define reflective oracles, which provide a foundation for game-theoretic analysis of rational agents that can simulate and reason about other agents. Using reflective oracles, w...
You are about to share this research paper context with your Ari companion. No personal browsing history or private memory is accessed.