No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL
Citations Over Time
Abstract
The performance of reinforcement learning (RL) agents is sensitive to the choice of hyperparameters. In real-world settings like robotics or industrial control systems, however, testing different hyperparameter configurations directly on the environment can be financially prohibitive, dangerous, or time consuming. We propose a new approach to tune hyperparameters from offline logs of data, to fully specify the hyperparameters for an RL agent that learns online in the real world. The approach is conceptually simple: we first learn a model of the environment from the offline data, which we call a calibration model, and then simulate learning in the calibration model to identify promising hyperparameters. We identify several criteria to make this strategy effective, and develop an approach that satisfies these criteria. We empirically investigate the method in a variety of settings to identify when it is effective and when it fails.
Related Papers
- → Grid Search Based Hyperparameter Optimization for Machine Learning Based Non-Intrusive Load Monitoring(2023)6 cited
- → Deep Neural Network with Hyperparameter Tuning for Detection of Heart Disease(2021)23 cited
- → Hyperparameter Tuning on Classification Algorithm with Grid Search(2022)16 cited
- → Novel Suboptimal approaches for Hyperparameter Tuning of Deep Neural Network [under the shelf of Optical Communication](2019)3 cited
- → Hyperparameter Tuning of Dense Neural Network for ECG Signal Classification(2022)