Efficient Video Frame Interpolation Using Generative Adversarial Networks
Citations Over TimeTop 23% of 2020 papers
Abstract
Frame interpolation, which generates an intermediate frame given adjacent ones, finds various applications such as frame rate up-conversion, video compression, and video streaming. Instead of using complex network models and additional data involved in the state-of-the-art frame interpolation methods, this paper proposes an approach based on an end-to-end generative adversarial network. A combined loss function is employed, which jointly considers the adversarial loss (difference between data models), reconstruction loss, and motion blur degradation. The objective image quality metric values reach a PSNR of 29.22 dB and SSIM of 0.835 on the UCF101 dataset, similar to those of the state-of-the-art approach. The good visual quality is notably achieved by approximately one-fifth computational time, which entails possible real-time frame rate up-conversion. The interpolated output can be further improved by a GAN based refinement network that better maintains motion and color by image-to-image translation.
Related Papers
- → <title>Estimating motion vectorfields with smoothness constraints</title>(1993)1 cited
- → Interpolation of cinematic sequences(2003)1 cited
- Adaptive motion estimation technique for motion compensated interframe interpolation(1999)
- → Video dynamic interpolation based on weighted shift interframe motion model(2010)
- → Adaptive motion estimation technique for motion compensated interframe interpolation(2003)