Self-supervised training for blind multi-frame video denoising

Citations Over Time

Abstract

We propose a self-supervised approach for training multi-frame video denoising networks. These networks predict each frame from a stack of frames around it. Our self-supervised approach benefits from the temporal consistency in the video by minimizing a loss that penalizes the difference between the predicted frame and a neighboring one, after aligning them using an optical flow. We use the proposed strategy to denoise a video contaminated with an unknown noise type, by fine-tuning a pre-trained denoising network on the noisy video. The proposed fine-tuning reaches and sometimes surpasses the performance of state-of-the-art networks trained with supervision. We demonstrate this by showing extensive results on video blind denoising of different synthetic and real noises. In addition, the proposed fine-tuning can be applied to any parameter that controls the denoising performance of the network. We show how this can be expoited to perform joint denoising and noise level estimation for heteroscedastic noise.

Related Papers

• Self-supervised learning of motion capture(2017)
• → A Video Prediction Method Based on Optical Flow Estimation and Pixel Generation(2021)12 cited
• → Optical Flow Estimation through Fusion Network based on Self-supervised Deep Learning(2021)1 cited
• → Deep Learning of Appearance Models for Online Object Tracking(2016)13 cited
• → Robust Temporal Super-Resolution for Dynamic Motion Videos(2019)