A comprehensive survey on video frame interpolation techniques

[1]  Zhenzhong Chen,et al.  A Multi-Scale Position Feature Transform Network for Video Frame Interpolation , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Zhenzhong Chen,et al.  Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Zhenzhong Chen,et al.  Video Frame Interpolation via Deformable Separable Convolution , 2020, AAAI.

[4]  Bohyung Han,et al.  Channel Attention Is All You Need for Video Frame Interpolation , 2020, AAAI.

[5]  Kyoung Mu Lee,et al.  Scene-Adaptive Video Frame Interpolation via Meta-Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Feng Liu,et al.  Softmax Splatting for Video Frame Interpolation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Li Chen,et al.  Blurry Video Frame Interpolation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ran Li,et al.  A Low-Complex Frame Rate Up-Conversion with Edge-Preserved Filtering , 2020, Electronics.

[9]  Munchurl Kim,et al.  FISR: Deep Joint Frame Interpolation and Super-Resolution with A Multi-scale Temporal Loss , 2019, AAAI.

[10]  Jinwoo Jeong,et al.  A Fast 4K Video Frame Interpolation Using a Multi-Scale Optical Flow Reconstruction Network , 2019, Symmetry.

[11]  Bumjun Park,et al.  PoSNet: 4x Video Frame Interpolation Using Position-Specific Flow , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[12]  Wenxiu Sun,et al.  Quadratic Video Interpolation for VTSR Challenge , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[13]  I. Perera,et al.  Tracking and frame-rate enhancement for real-time 2D human pose estimation , 2019, The Visual Computer.

[14]  Ronggang Wang,et al.  Multi-Frame Pyramid Refinement Network for Video Frame Interpolation , 2019, IEEE Access.

[15]  Thierry Blu,et al.  Lap-Based Video Frame Interpolation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[16]  Weiwei Liu,et al.  Generating Realistic Videos From Keyframes With Concatenated GANs , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Taeoh Kim,et al.  AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Yung-Yu Chuang,et al.  Deep Video Frame Interpolation Using Cyclic Frame Generation , 2019, AAAI.

[19]  Feng Jiang,et al.  Continuous Bidirectional Optical Flow for Video Frame Sequence Interpolation , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[20]  Tomer Peleg,et al.  IM-Net for High Resolution Video Frame Interpolation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Chen Change Loy,et al.  EDVR: Video Restoration With Enhanced Deformable Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Jinwoo Jeong,et al.  A Fast 4K Video Frame Interpolation Using a Hybrid Task-Based Convolutional Neural Network , 2019, Symmetry.

[23]  Mehran Yazdi,et al.  Real-time object tracking based on an adaptive transition model and extended Kalman filter to handle full occlusion , 2019, The Visual Computer.

[24]  Xiaoyun Zhang,et al.  Depth-Aware Video Frame Interpolation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Guangying Ge,et al.  Frame Rate Up-Conversion Based on Edge Information , 2019, 2019 7th International Conference on Information, Communication and Networks (ICICN).

[26]  Yanli Li,et al.  A Spatial Prediction-Based Motion-Compensated Frame Rate Up-Conversion , 2019, Future Internet.

[27]  Yao Zhao,et al.  Optical Flow-Guided Multi-Scale Dense Network for Frame Interpolation , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[28]  Joonhwan Yi,et al.  An interpolation method for strong barrel lens distortion , 2018, The Visual Computer.

[29]  Zhiyong Gao,et al.  MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jian Sun,et al.  Rendering Portraitures from Monocular Camera and Beyond , 2018, ECCV.

[31]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[32]  Feng Jiang,et al.  Video Frame Interpolation Based on Multi-scale Convolutional Network and Adversarial Training , 2018, 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC).

[33]  Markus H. Gross,et al.  PhaseNet for Video Frame Interpolation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Feng Liu,et al.  Context-Aware Synthesis for Video Frame Interpolation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Yi Wang,et al.  Scale-Recurrent Network for Deep Image Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Lizhuang Ma,et al.  Multi-Scale Video Frame-Synthesis Network with Transitive Consistency Loss , 2017, ArXiv.

[37]  Jan Kautz,et al.  Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  W. Freeman,et al.  Video Enhancement with Task-Oriented Flow , 2017, International Journal of Computer Vision.

[39]  Deqing Sun,et al.  PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Feng Liu,et al.  Video Frame Interpolation via Adaptive Separable Convolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[41]  Alain Trémeau,et al.  Residual Conv-Deconv Grid Network for Semantic Segmentation , 2017, BMVC.

[42]  Zhiquan Feng,et al.  An efficient 3D video frame interpolation method using color-depth-motion information , 2017, 2017 4th International Conference on Information, Cybernetics and Computational Social Systems (ICCSS).

[43]  Guillermo Sapiro,et al.  Deep Video Deblurring for Hand-Held Cameras , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Michael J. Black,et al.  Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Ruckmani Divakaran,et al.  True-Motion Estimation Algorithm and Its Application to Motion-Compensated Temporal Frame Interpolation , 2017 .

[46]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[47]  Feng Liu,et al.  Video Frame Interpolation via Adaptive Convolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Raymond A. Yeh,et al.  Video Frame Synthesis Using Deep Voxel Flow , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[49]  Serge J. Belongie,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Michael J. Black,et al.  Optical Flow Estimation Using a Spatial Pyramid Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Taejeong Kim,et al.  MAP-Based Motion Refinement Algorithm for Block-Based Motion-Compensated Frame Interpolation , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[53]  Shahram Shirani,et al.  Frame Rate Upconversion Using Optical Flow and Patch-Based Reconstruction , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[54]  Takeo Igarashi,et al.  Path-based image sequence interpolation guided by feature points , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[55]  Qionghai Dai,et al.  A Polynomial Approximation Motion Estimation Model for Motion-Compensated Frame Interpolation , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[56]  Jiajun Wu,et al.  Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks , 2016, NIPS.

[57]  Luc Van Gool,et al.  A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Weifeng Chen,et al.  Single-Image Depth Perception in the Wild , 2016, NIPS.

[59]  Wenbin Li,et al.  Video interpolation using optical flow and Laplacian smoothness , 2016, Neurocomputing.

[60]  Hongdong Li,et al.  Learning Image Matching by Simply Watching Video , 2016, ECCV.

[61]  Viorica Patraucean,et al.  Spatio-temporal video autoencoder with differentiable memory , 2015, ArXiv.

[62]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[63]  Max Grosse,et al.  Phase-based frame interpolation for video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[65]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[66]  Thierry Blu,et al.  Local All-Pass filters for optical flow estimation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[67]  Hyun Wook Park,et al.  A Region-Based Motion-Compensated Frame Interpolation Method Using a Variance-Distortion Curve , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[68]  Cordelia Schmid,et al.  EpicFlow: Edge-preserving interpolation of correspondences for optical flow , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Xueying Qin,et al.  Depth map enhancement based on color and depth consistency , 2014, The Visual Computer.

[70]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[71]  Myung Hoon Sunwoo,et al.  New Frame Rate Up-Conversion Algorithms With Low Computational Complexity , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[72]  Truong Q. Nguyen,et al.  Depth-Assisted Frame Rate Up-Conversion for Stereoscopic Video , 2014, IEEE Signal Processing Letters.

[73]  Qunsheng Peng,et al.  Depth map enhancement based on color and depth consistency , 2013, The Visual Computer.

[74]  Frédo Durand,et al.  Joint view expansion and filtering for automultiscopic 3D displays , 2013, ACM Trans. Graph..

[75]  Bo Yan,et al.  Low complexity image interpolation method based on path selection , 2013, J. Vis. Commun. Image Represent..

[76]  Houqiang Li,et al.  Multi-Level Video Frame Interpolation: Exploiting the Interaction Among Different Levels , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[77]  Wenbin Li,et al.  Optical Flow Estimation Using Laplacian Mesh Energy , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[78]  Hyun Wook Park,et al.  Iterative True Motion Estimation for Motion-Compensated Frame Interpolation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[79]  Mubarak Shah,et al.  UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild , 2012, ArXiv.

[80]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[81]  Wen Gao,et al.  Multiple Hypotheses Bayesian Frame Rate Up-Conversion by Adaptive Fusion of Motion-Compensated Interpolations , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[82]  Joachim Weickert,et al.  Motion Compensated Frame Interpolation with a Symmetric Optical Flow Constraint , 2012, ISVC.

[83]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[84]  Hyun Wook Park,et al.  Motion estimation with adaptive block size for motion-compensated frame interpolation , 2012, 2012 Picture Coding Symposium.

[85]  Hyun Wook Park,et al.  A Symmetric Motion Estimation Method for Motion-Compensated Frame Interpolation , 2011, IEEE Transactions on Image Processing.

[86]  Horst Bischof,et al.  Optical Flow Guided TV-L1 Video Interpolation and Restoration , 2011, EMMCVPR.

[87]  Lourdes Agapito,et al.  Robust Trajectory-Space TV-L1 Optical Flow for Non-rigid Sequences , 2011, EMMCVPR.

[88]  Sungjoo Yoo,et al.  Dual Motion Estimation for Frame Rate Up-Conversion , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[89]  Truong Q. Nguyen,et al.  A Novel Approach to FRUC Using Discriminant Saliency and Frame Segmentation , 2010, IEEE Transactions on Image Processing.

[90]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[91]  Lin Liang,et al.  AAM based face tracking with temporal matching and face segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[92]  Daniel Cremers,et al.  Large displacement optical flow computation withoutwarping , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[93]  Evan Herbst,et al.  Occlusion Reasoning for Temporal Interpolation using Optical Flow , 2009 .

[94]  Stephen DiVerdi,et al.  Exploring collections of 3D models using fuzzy correspondences , 2012, ACM Trans. Graph..

[95]  Suk-Ju Kang,et al.  Multiframe-based bilateral motion estimation with emphasis on stationary caption processing for frame rate up-conversion , 2008, IEEE Transactions on Consumer Electronics.

[96]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[97]  Kyoung-Rok Cho,et al.  Motion Compensated Frame Rate Up-Conversion Using Extended Bilateral Motion Estimation , 2007, IEEE Transactions on Consumer Electronics.

[98]  Cosmin Ancuti,et al.  Video enhancement using reference photographs , 2007, SIGGRAPH '07.

[99]  Chang-Su Kim,et al.  Motion-Compensated Frame Interpolation Using Bilateral Motion Estimation and Adaptive Overlapped Block Motion Compensation , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[100]  Hui Cheng,et al.  Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection , 2006, ECCV.

[101]  Nebojsa Jojic,et al.  Consistent segmentation for optical flow estimation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[102]  Jiang Li,et al.  A low complexity motion compensated frame interpolation method , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[103]  Jinsong Wang,et al.  Video frame rate up conversion using region based motion compensation , 2004, 2004 IEEE Electro/Information Technology Conference.

[104]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[105]  Rae-Hong Park,et al.  Coarse-to-fine frame interpolation for frame rate up-conversion using pyramid structure , 2003, IEEE Trans. Consumer Electron..

[106]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[107]  Rachid Deriche,et al.  Symmetrical Dense Optical Flow Estimation with Occlusions Detection , 2002, International Journal of Computer Vision.

[108]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[109]  Pierre Moulin,et al.  Frame interpolation and bidirectional prediction of video using compactly encoded optical-flow fields and label fields , 1999, IEEE Trans. Circuits Syst. Video Technol..

[110]  David J. Fleet,et al.  Performance of optical flow techniques , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[111]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[112]  Xiaojun Bi,et al.  Multi-Scale Attention Generative Adversarial Networks for Video Frame Interpolation , 2020, IEEE Access.

[113]  Xiem HoangVan,et al.  Statistical search range adaptation solution for effective frame rate up-conversion , 2018, IET Image Process..

[114]  Kunal Menda,et al.  Frame Interpolation Using Generative Adversarial Networks , 2017 .

[115]  Xiangzhong Fang,et al.  Motion-Compensated Frame Interpolation With Multiframe-Based Occlusion Handling , 2016, Journal of Display Technology.

[116]  Jong Beom Ra,et al.  Frame Rate Up Conversion Based on Variational Image Fusion , 2014, IEEE Transactions on Image Processing.

[117]  Mads Nielsen,et al.  Temporal Super Resolution Using Variational Methods , 2010 .

[118]  Ce Liu.  Beyond pixels: exploring new representations and applications for motion analysis , 2009 .

[119]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .