Improved Baselines with Momentum Contrastive Learning
arXiv (Cornell University)2020
Citations Over Time
Abstract
Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLR's design improvements by implementing them in the MoCo framework. With simple modifications to MoCo---namely, using an MLP projection head and more data augmentation---we establish stronger baselines that outperform SimCLR and do not require large training batches. We hope this will make state-of-the-art unsupervised learning research more accessible. Code will be made public.
Related Papers
- → Fundamental relations in simple and 0-simple semihypergroups of small size(2012)8 cited
- → New Characterizations of Simple Points, Minimal Non-simple Sets and P-Simple Points in 2D, 3D and 4D Discrete Spaces(2008)6 cited
- → A Note on Maximized Posttest Contrasts(1979)1 cited
- The [0-Simple] Simple Subsemigroups of Nonnegative Matrices(2000)