Panoramic video textures

This paper describes a mostly automatic method for taking the output of a single panning video camera and creating a panoramic video texture (PVT): a video that has been stitched into a single, wide field of view and that appears to play continuously and indefinitely. The key problem in creating a PVT is that although only a portion of the scene has been imaged at any given time, the output must simultaneously portray motion throughout the scene. Like previous work in video textures, our method employs min-cut optimization to select fragments of video that can be stitched together both spatially and temporally. However, it differs from earlier work in that the optimization must take place over a much larger set of data. Thus, to create PVTs, we introduce a dynamic programming step, followed by a novel hierarchical min-cut optimization algorithm. We also use gradient-domain compositing to further smooth boundaries between video fragments. We demonstrate our results with an interactive viewer in which users can interactively pan and zoom on high-resolution PVTs.

[1]  Shree K. Nayar,et al.  Catadioptric omnidirectional camera , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[3]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[4]  Richard Szeliski,et al.  Image-based interactive exploration of real-world environments , 2004, IEEE Computer Graphics and Applications.

[5]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[6]  Ulrich Neumann,et al.  Immersive panoramic video , 2000, ACM Multimedia.

[7]  N. Ahuja,et al.  Seamless video editing , 2004, ICPR 2004.

[8]  Richard Szeliski,et al.  High-quality Image-based Interactive Exploration of Real-World Environments 1 , 2003 .

[9]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[10]  Don Kimber,et al.  FlyAbout: spatially indexed panoramic video , 2001, MULTIMEDIA '01.

[11]  A. Fitzgibbon Stochastic rigidity: image registration for nowhere-static scenes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[13]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[14]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[16]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[17]  Dani Lischinski,et al.  Dynamosaics: video mosaics with non-chronological time , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  David Salesin,et al.  Multiresolution video , 1996, SIGGRAPH.

[19]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..