HyperSpace: Distributed Bayesian Hyperparameter Optimization

Citations Over TimeTop 17% of 2018 papers

Abstract

As machine learning models continue to increase in complexity, so does the potential number of free model parameters commonly known as hyperparameters. While there has been considerable progress toward finding optimal configurations of these hyperparameters, many optimization procedures are treated as black boxes. We believe optimization methods should not only return a set of optimized hyperparameters, but also give insight into the effects of model hyperparameter settings. To this end, we present HyperSpace, a parallel implementation of Bayesian sequential model-based optimization. HyperSpace leverages high performance computing (HPC) resources to better understand unknown, potentially non-convex hyperparameter search spaces. We show that it is possible to learn the dependencies between model hyperparameters through the optimization process. By partitioning large search spaces and running many optimization procedures in parallel, we also show that it is possible to discover families of good hyperparameter settings over a variety of models including unsupervised clustering, regression, and classification tasks.

Related Papers

• → Initializing Bayesian Hyperparameter Optimization via Meta-Learning(2015)394 cited
• → Deep Neural Network with Hyperparameter Tuning for Detection of Heart Disease(2021)23 cited
• → Evaluation of Hyperparameter-Optimization Approaches in an Industrial Federated Learning System(2022)14 cited
• → Bayesian Hyperparameter Optimization for Ensemble Learning(2016)35 cited
• → Evaluation of Hyperparameter-Optimization Approaches in an Industrial Federated Learning System(2021)