Neural Optimizer Search with Reinforcement Learning

Citations Over Time

Abstract

We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a domain specific language that describes a mathematical update equation based on a list of primitive functions, such as the gradient, running average of the gradient, etc. The controller is trained with Reinforcement Learning to maximize the performance of a model after a few epochs. On CIFAR-10, our method discovers several update rules that are better than many commonly used optimizers, such as Adam, RMSProp, or SGD with and without Momentum on a ConvNet model. We introduce two new optimizers, named PowerSign and AddSign, which we show transfer well and improve training on a variety of different tasks and architectures, including ImageNet classification and Google's neural machine translation system.

Related Papers

• → A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform(2022)15 cited
• → Performance of distributed multi-agent multi-state reinforcement spectrum management using different exploration schemes(2013)13 cited
• → Machine Translation Using Deep Learning: A Comparison(2020)4 cited
• → Cost-Efficient Reinforcement Learning for Optimal Trade Execution on Dynamic Market Environment(2022)2 cited
• → Classification without gradients: multi-agent reinforcement learning approach to optimization (Conference Presentation)(2023)