Learning Correspondence From the Cycle-Consistency of Time

Citations Over TimeTop 1% of 2019 papers

Abstract

We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful for performing cycle-consistent tracking. At test time, we use the acquired representation to find nearest neighbors across space and time. We demonstrate the generalizability of the representation -- without finetuning -- across a range of visual correspondence tasks, including video object segmentation, keypoint tracking, and optical flow. Our approach outperforms previous self-supervised methods and performs competitively with strongly supervised methods.

Related Papers

• → Generalizing Generalizability in Information Systems Research(2003)1,544 cited
• → External Validity, Generalizability, and Knowledge Utilization(2004)250 cited
• → Applying Generalizability Theory using EduG(2011)108 cited
• → Reliability of observers' subjective impressions of families: A generalizability theory approach(2012)20 cited
• → Using Generalizability Theory for the Estimation of Reliability of a Patient Classification System(1994)1 cited