Imitation by Predicting Observations
Citations Over Time
Abstract
Imitation learning enables agents to reuse and adapt the hard-won expertise of others, offering a solution to several key challenges in learning behavior. Although it is easy to observe behavior in the real-world, the underlying actions may not be accessible. We present a new method for imitation solely from observations that achieves comparable performance to experts on challenging continuous control tasks while also exhibiting robustness in the presence of observations unrelated to the task. Our method, which we call FORM (for "Future Observation Reward Model") is derived from an inverse RL objective and imitates using a model of expert behavior learned by generative modelling of the expert's observations, without needing ground truth actions. We show that FORM performs comparably to a strong baseline IRL method (GAIL) on the DeepMind Control Suite benchmark, while outperforming GAIL in the presence of task-irrelevant features.
Related Papers
- A Bayesian Approach to Generative Adversarial Imitation Learning(2018)
- Deterministic Policy Imitation Gradient Algorithm(2018)
- → Sample-Efficient Imitation Learning via Generative Adversarial Nets(2018)21 cited
- → TRAIL: Near-Optimal Imitation Learning with Suboptimal Data(2021)5 cited