Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning

Citations Over TimeTop 10% of 2021 papers

Abstract

Self-supervised learning has shown great potentials in improving the video representation ability of deep neural networks by getting supervision from the data itself. However, some of the current methods tend to cheat from the background, i.e., the prediction is highly dependent on the video background instead of the motion, making the model vulnerable to background changes. To mitigate the model reliance towards the background, we propose to remove the background impact by adding the background. That is, given a video, we randomly select a static frame and add it to every other frames to construct a distracting video sample. Then we force the model to pull the feature of the distracting video and the feature of the original video closer, so that the model is explicitly restricted to resist the background influence, focusing more on the motion changes. We term our method as Background Erasing (BE). It is worth noting that the implementation of our method is so simple and neat and can be added to most of the SOTA methods without much efforts. Specifically, BE brings 16.4% and 19.1% improvements with MoCo on the severely biased datasets UCF101 and HMDB51, and 14.5% improvement on the less biased dataset Diving48.

Related Papers

• → The origins and early development of the psychological contract construct(1997)231 cited
• → The Construct(2009)12 cited
• Review of journal of construction of modern and contemporary times in China(2005)
• The Generic Cabling System of Construct(2005)
• → The Development of Leader-Member Exchange Construct and the Emergence of Algorithmic Leader-Member Exchange Construct in the Gig Economy(2023)