View and Time Interpolation in Image Space

The ability to interpolate between images taken at different time and viewpoints directly in image space opens up new possiblities. The goal of our work is to create plausible in‐between images in real time without the need for an intermediate 3D reconstruction. This enables us to also interpolate between images recorded with uncalibrated and unsynchronized cameras. In our approach we use a novel discontiniuity preserving image deformation model to robustly estimate dense correspondences based on local homographies. Once correspondences have been computed we are able to render plausible in‐between images in real time while properly handling occlusions. We discuss the relation of our approach to human motion perception and other image interpolation techniques.

[1]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[4]  George Wolberg,et al.  Image morphing: a survey , 1998, The Visual Computer.

[5]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[6]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[7]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Dimitri P. Bertsekas,et al.  Auction algorithms for network flow problems: A tutorial introduction , 1992, Comput. Optim. Appl..

[9]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[10]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1992, SIGGRAPH.

[11]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[12]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[13]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[14]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[15]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[16]  J. Canny.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[18]  Rosalee Wolfe.  Riding the wave of technology: the SIGGRAPH 97 educators program , 1997, COMG.

[19]  M. Wertheimer.  Laws of organization in perceptual forms. , 1938 .

[20]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[21]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[22]  M. Wertheimer,et al.  A source book of Gestalt psychology. , 1939 .

[23]  Wojciech Matusik,et al.  3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes , 2004, ACM Trans. Graph..

[24]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[25]  Jitendra Malik,et al.  Matching Shapes , 2001, ICCV.

[26]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[27]  Yuri M. Bayakovsky,et al.  Russia: computer graphics education takes off in the 1990s , 1996, COMG.

[28]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[29]  Carlo Tomasi,et al.  Color edge detection with the compass operator , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[30]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[31]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.

[32]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[33]  Marcus A. Magnor,et al.  Perception-motivated interpolation of image sequences , 2008, APGV.

[34]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.