Towards Mixed Optimization for Reinforcement Learning with Program Synthesis

Citations Over Time

Abstract

Deep reinforcement learning has led to several recent breakthroughs, though the learned policies are often based on black-box neural networks. This makes them difficult to interpret and to impose desired specification constraints during learning. We present an iterative framework, MORL, for improving the learned policies using program synthesis. Concretely, we propose to use synthesis techniques to obtain a symbolic representation of the learned policy, which can then be debugged manually or automatically using program repair. After the repair step, we use behavior cloning to obtain the policy corresponding to the repaired program, which is then further improved using gradient descent. This process continues until the learned policy satisfies desired constraints. We instantiate MORL for the simple CartPole problem and show that the programmatic representation allows for high-level modifications that in turn lead to improved learning of the policies.

Related Papers

• → Performance of distributed multi-agent multi-state reinforcement spectrum management using different exploration schemes(2013)13 cited
• → Fundamental relations in simple and 0-simple semihypergroups of small size(2012)8 cited
• → Cost-Efficient Reinforcement Learning for Optimal Trade Execution on Dynamic Market Environment(2022)2 cited
• → New Characterizations of Simple Points, Minimal Non-simple Sets and P-Simple Points in 2D, 3D and 4D Discrete Spaces(2008)6 cited
• The [0-Simple] Simple Subsemigroups of Nonnegative Matrices(2000)