Multi-image interpolation based on graph-cuts and symmetric optical flow

Multi-image interpolation in space and time has recently received considerable attention. Typically, the interpolated image is synthesized by adaptively blending several forward-warped images. Blending itself is a low-pass filtering operation: the interpolated images are prone to blurring and ghosting artifacts as soon as the underlying correspondence fields are imperfect. We address both issues and propose a multi-image interpolation algorithm that avoids blending. Instead, our algorithm decides for each pixel in the synthesized view from which input image to sample. Combined with a symmetrical long-range optical flow formulation for correspondence field estimation, our approach yields crisp interpolated images without ghosting artifacts.

[1]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[2]  Daniel Cremers,et al.  Advanced Data Terms for Variational Optic Flow Estimation , 2009, VMV.

[3]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[4]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Jitendra Malik,et al.  Large displacement optical flow , 2009, CVPR.

[6]  Marcus A. Magnor,et al.  View and Time Interpolation in Image Space , 2008, Comput. Graph. Forum.

[7]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[8]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[9]  Sung Yong Shin,et al.  Polymorph: Morphing Among Multiple Images , 1998, IEEE Computer Graphics and Applications.

[10]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[11]  Rachid Deriche,et al.  Symmetrical Dense Optical Flow Estimation with Occlusions Detection , 2002, ECCV.

[12]  Janusz Konrad,et al.  Occlusion-Aware Optical Flow Estimation , 2008, IEEE Transactions on Image Processing.

[13]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[14]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[15]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[16]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  P. Belhumeur,et al.  Moving gradients: a path-based method for plausible image interpolation , 2009, SIGGRAPH 2009.

[18]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1998 .

[19]  Marcus A. Magnor,et al.  Perception-motivated interpolation of image sequences , 2008, TAP.

[20]  Maneesh Agrawala,et al.  Using Photographs to Enhance Videos of a Static Scene , 2007, Rendering Techniques.

[21]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Anita Sellent,et al.  Virtual Video Camera: Image‐Based Viewpoint Navigation Through Space and Time , 2010, Comput. Graph. Forum.

[23]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[24]  Daniel Cremers,et al.  Large displacement optical flow computation withoutwarping , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[26]  Baining Guo,et al.  Feature-based light field morphing , 2002, ACM Trans. Graph..