Quadratic Video Interpolation for VTSR Challenge

Video interpolation is an important problem in image manipulation, which has drawn increased interests from the vision and graphics communities. In this work, we apply the quadratic video interpolation algorithm to the VTSR challenge of Advances in Image Manipulation (AIM) 2019, and introduce a joint finetuning scheme to exploit more training data. We provide a concise description of the quadratic model and present a detailed analysis of the results in the VTSR challenge. Extensive experiments demonstrate that our network generates high-quality interpolation results and outperforms the state-of-the-arts by a large margin.

[1]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[2]  Feng Liu,et al.  Context-Aware Synthesis for Video Frame Interpolation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  Jian Sun,et al.  Rendering Portraitures from Monocular Camera and Beyond , 2018, ECCV.

[5]  Max Grosse,et al.  Phase-based frame interpolation for video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Feng Liu,et al.  Video Frame Interpolation via Adaptive Convolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ming-Hsuan Yang,et al.  Motion Blur Kernel Estimation via Deep Learning , 2018, IEEE Transactions on Image Processing.

[8]  Xiaoou Tang,et al.  Video Frame Synthesis Using Deep Voxel Flow , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Jan Kautz,et al.  PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Jonathan T. Barron,et al.  Jump: virtual reality video , 2016, ACM Trans. Graph..

[11]  Bohyung Han,et al.  AIM 2019 Challenge on Video Temporal Super-Resolution: Methods and Results , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[12]  Jonathan T. Barron,et al.  Learning to Synthesize Motion Blur , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Wentian Li,et al.  Temporal Feature Correlation for Human Pose Estimation in Videos , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[16]  Feng Liu,et al.  Video Frame Interpolation via Adaptive Separable Convolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[18]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Jan Kautz,et al.  Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Nikolaos G. Bourbakis,et al.  Three-Dimensional Reconstruction of the Digestive Wall in Capsule Endoscopy Videos Using Elastic Video Interpolation , 2011, IEEE Transactions on Medical Imaging.

[21]  Qian Yin,et al.  Quadratic video interpolation , 2019, NeurIPS.

[22]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.