Flow Contrastive Estimation of Energy-Based Models

Citations Over TimeTop 10% of 2020 papers

Abstract

This paper studies a training method to jointly estimate an energy-based model and a flow-based model, in which the two models are iteratively updated based on a shared adversarial value function. This joint training method has the following traits. (1) The update of the energy-based model is based on noise contrastive estimation, with the flow model serving as a strong noise distribution. (2) The update of the flow model approximately minimizes the Jensen-Shannon divergence between the flow model and the data distribution. (3) Unlike generative adversarial networks (GAN) which estimates an implicit probability distribution defined by a generator model, our method estimates two explicit probabilistic distributions on the data. Using the proposed method we demonstrate a significant improvement on the synthesis quality of the flow model, and show the effectiveness of unsupervised feature learning by the learned energy-based model. Furthermore, the proposed training method can be easily adapted to semi-supervised learning. We achieve competitive results to the state-of-the-art semi-supervised learning methods.

Related Papers

• → Focal points in coordinated divergence(2006)21 cited
• → A Generalized $$\chi ^2$$ Divergence for Multisource Information Fusion(2021)1 cited
• → A THREE-DIMENSIONAL STUDY OF THE HORIZONTAL VELOCITY DIVERGENCE(1955)11 cited
• → Indiscriminate Separation, under the Same Environment, a Cause of Divergence(1890)9 cited
• → Determinants of Economic Divergence Among Accelerating Societies(2013)