Discovering agents
Citations Over TimeTop 11% of 2023 papers
Abstract
Causal models of agents have been used to analyse the safety aspects of machine learning systems. But identifying agents is non-trivial – often the causal model is just assumed by the modeller without much justification – and modelling failures can lead to mistakes in the safety analysis. This paper proposes the first formal causal definition of agents – roughly that agents are systems that would adapt their policy if their actions influenced the world in a different way. From this we derive the first causal discovery algorithm for discovering the presence of agents from empirical data, given a set of variables and under certain assumptions. We also provide algorithms for translating between causal models and game-theoretic influence diagrams. We demonstrate our approach by resolving some previous confusions caused by incorrect causal modelling of agents.
Related Papers
- → Causality in the Sciences(2011)198 cited
- → [Causal graph model and its application in nutritional epidemiologic research].(2021)
- → Causal Path Analysis of Teaching Competency based on the Structural Causal Model: Focusing on the Comparison across South Korea, England, and Finland using TALIS 2018 data(2022)
- → Causality(2021)