0 citations
Model evaluation for extreme risks
arXiv (Cornell University)2023
Citations Over Time
Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, L. Lawrence Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack A. Clark, Yoshua Bengio, Paul Christiano, Allan Dafoe
Abstract
Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities. Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills. We explain why model evaluation is critical for addressing extreme risks. Developers must be able to identify dangerous capabilities (through "dangerous capability evaluations") and the propensity of models to apply their capabilities for harm (through "alignment evaluations"). These evaluations will become critical for keeping policymakers and other stakeholders informed, and for making responsible decisions about model training, deployment, and security.
Related Papers
- → Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts(2021)49 cited
- The Soviet Strategic Offensive in Manchuria, 1945: 'August Storm'(2003)
- → Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts(2021)
- Correlation of offensive and defensive operations(2002)
- → Just Say No: Analyzing the Stance of Neural Dialogue Generation in\n Offensive Contexts(2021)