Path-based image sequence interpolation guided by feature points

We present a method of image sequence interpolation, which can generate a sequence of continuous intermediate frames between two input images. This method is based on a path framework that describes the motion information in the images. A path which starts from one input image, and ends at another input image is constructed for each pixel in the images. The main contribution of this paper is that we take the feature points into consideration. By calculating the position deviation out of the feature points, information and guidance can be given to the process of path optimization, making the interpolation result more plausible and natural. We also increase the conditions and restrictions in the optimization procedure, hence the time and memory cost can be effectively decreased.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[3]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  A. J. Roberts.  Fast and accurate multigrid solution of Poissons equation using diagonally oriented grids , 1999 .

[5]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  R. Zabih,et al.  What energy functions can be minimized via graph cuts , 2004 .

[8]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[10]  Maja Pantic,et al.  Fully automatic facial feature point detection using Gabor feature based boosted classifiers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[11]  P. Koumoutsakos,et al.  Feature point tracking and trajectory analysis for video imaging in cell biology. , 2005, Journal of structural biology.

[12]  J. Weickert,et al.  Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods , 2005 .

[13]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[14]  Samuel Audet,et al.  Image-Based Rendering Using Image-Based Priors , 2006 .

[15]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[16]  P. Belhumeur,et al.  Moving gradients: a path-based method for plausible image interpolation , 2009, SIGGRAPH 2009.

[17]  Lin Liang,et al.  AAM based face tracking with temporal matching and face segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Eli Shechtman,et al.  Regenerative morphing , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Peter H. N. de With,et al.  View Interpolation for Medical Images on Autostereoscopic Displays , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  P. Kanongchaiyos,et al.  Face morphing using critical point filters , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[21]  Bo Yan,et al.  Low complexity image interpolation method based on path selection , 2013, J. Vis. Commun. Image Represent..

[22]  Ramin Samadani,et al.  Stereo Matching and View Interpolation Based on Image Domain Triangulation , 2013, IEEE Transactions on Image Processing.