Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization

Abstract

Simulation-based falsification is a practical testing method to increase confidence that the system will meet safety requirements. Because full-fidelity simulations can be computationally demanding, we investigate the use of simulators with different levels of fidelity. As a first step, we express the overall safety specification in terms of environmental parameters and structure this safety specification as an optimization problem. We propose a multi-fidelity falsification framework using Bayesian optimization, which is able to determine at which level of fidelity we should conduct a safety evaluation in addition to finding possible instances from the environment that cause the system to fail. This method allows us to automatically switch between inexpensive, inaccurate information from a low-fidelity simulator and expensive, accurate information from a high-fidelity simulator in a cost-effective way. Our experiments on various environments in simulation demonstrate that multi-fidelity Bayesian optimization has falsification performance comparable to single-fidelity Bayesian optimization but with much lower cost.

Related Papers

• → Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization(2023)7 cited
• → Initial sampling using multi-fidelity information in simulation optimization of manufacturing systems(2018)1 cited
• Automated Machine Learning - Bayesian Optimization, Meta-Learning & Applications(2018)
• → Uncertainty Quantification of Delamination Failure within Composite Structures using a Submodel Based Multi-Fidelity Approach.(2021)
• → Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization(2022)