Evolving Time Fronts: Spatio-Temporal Video Warping

We present evolving time fronts, a new framework for spatiotemporal warping of video. The proposed framework is simple yet general, allowing a large variety of spatio-temporal warps to be specified in an intuitive manner. Specifically, we manipulate the time flow of a video sequence by sweeping an evolving time front surface through the video’s aligned space-time volume. In this paper we first introduce the general framework, and then describe and discuss several specific strategies for time front evolution that we have experimented with so far. These strategies are demonstrated to produce a variety of interesting and useful operations on video, ranging from subtle timing changes to eye-catching special effects, creation of dynamic panoramic mosaics, and parallax effects. CR Categories: I.3.3 [Computer Graphics]: Picture/Image Generation—display algorithms; I.3.6 [Computer Graphics]: Methodology and Techniques—interaction techniques; I.4.9 [Image Processing and Computer Vision]: Applications

[1]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.

[2]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1992, SIGGRAPH.

[3]  Dani Lischinski,et al.  Dynamosaics: video mosaics with non-chronological time , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Yael Pritch,et al.  Omnistereo: Panoramic Stereo Imaging , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Daphna Weinshall,et al.  Mosaicing New Views: The Crossed-Slits Projection , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[7]  Adam Finkelstein,et al.  Stylized video cubes , 2002, SCA '02.

[8]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[9]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[10]  M. Sheelagh T. Carpendale,et al.  Achieving higher magnification in context , 2004, UIST '04.

[11]  P. Anandan,et al.  Direct Recovery of Planar-Parallax from Multiple Frames , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Paul Rademacher,et al.  Multiple-center-of-projection images , 1998, SIGGRAPH.

[13]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[14]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[15]  David Salesin,et al.  Interactive digital photomontage , 2004, ACM Trans. Graph..

[16]  Robert C. Bolles,et al.  Generalizing Epipolar-Plane Image Analysis on the spatiotemporal surface , 2004, International Journal of Computer Vision.

[17]  Stochastic Rigidity: Image Registration for Nowhere-Static Scenes , 2001, ICCV.

[18]  David Salesin,et al.  Multiperspective panoramas for cel animation , 1997, SIGGRAPH.

[19]  Leonard McMillan,et al.  General Linear Cameras , 2004, ECCV.

[20]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[21]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[22]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[23]  Peter-Pike J. Sloan,et al.  Video Cubism , 2001 .