Video textures

This paper introduces a new type of medium, called a video texture, which has qualities somewhere between those of a photograph and a video. A video texture provides a continuous infinitely varying stream of images. While the individual frames of a video texture may be repeated from time to time, the video sequence as a whole is never repeated exactly. Video textures can be used in place of digital photos to infuse a static image with dynamic qualities and explicit actions. We present techniques for analyzing a video clip to extract its structure, and for synthesizing a new, similar looking video of arbitrary length. We combine video textures with view morphing techniques to obtain 3D video textures. We also introduce video-based animation, in which the synthesis of video textures can be guided by a user through high-level interactive controls. Applications of video textures and their extensions include the display of dynamic scenes on web pages, the creation of dynamic backdrops for special effects and games, and the interactive control of video-based animation.

[1]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[2]  Edward H. Adelson,et al.  Motion without movement , 1991, SIGGRAPH.

[3]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[5]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[6]  David Salesin,et al.  Fast multiresolution image querying , 1995, SIGGRAPH.

[7]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[8]  David Salesin,et al.  Hierarchical image caching for accelerated walkthroughs of complex environments , 1996, SIGGRAPH.

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[11]  David Salesin,et al.  Multiresolution video , 1996, SIGGRAPH.

[12]  Jay Torborg,et al.  Talisman: commodity realtime 3D graphics for the PC , 1996, SIGGRAPH.

[13]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[14]  E. Robeson,et al.  The Light of Other Days. , 1996 .

[15]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[16]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[17]  David Beymer,et al.  Feature correspondence by interleaving shape and texture computations , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Ramesh C. Jain,et al.  Reality modeling and visualization from multiple video sequences , 1996, IEEE Computer Graphics and Applications.

[19]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and texture-mapped models , 1997, International Conference on Computer Graphics and Interactive Techniques.

[20]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[21]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[22]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[23]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[24]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[25]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[26]  M. Werman.  Statistical Learning of Multi-Dimensional Textures , 1999 .

[27]  Paul E. Debevec,et al.  Image-based modeling, rendering, and lighting in Fiat Lux , 1999, SIGGRAPH '99.

[28]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[29]  Richard Szeliski,et al.  Construction of panoramic mosaics with global and lo-cal alignment , 2020 .

[30]  Sean Hayes,et al.  View synthesis by trinocular edge matching and transfer , 2000, Image Vis. Comput..

[31]  Richard Szeliski,et al.  Construction of Panoramic Image Mosaics with Global and Local Alignment , 2001 .

[32]  P. Debevec,et al.  Image-based modeling, rendering, and lighting , 2002, IEEE Computer Graphics and Applications.