Novel Approaches for Generating Video Textures

Video texture, a new type of medium, can produce a new video with a continuously varying stream of images from a recorded video. It is synthesized by reordering the input video frames in a way which can be played without any visual discontinuity. However, video texture still experiences few unappealing drawbacks. For instance, video texture techniques can only generate new videos by simply rearranging the order of frames in original videos. Therefore, all the individual frames are the same as before and the result would suffer from "dead-ends" if the current frame could not discover similar frames to make a transition. In this thesis, we propose several new approaches for synthesizing video textures. These approaches adopt dimensionality reduction and regression techniques to generate video textures. Not only the frames in the resulted video textures are new, but also the "Dead end" problem is avoided. First, we have extended die work of applying principal components analysis (PCA) and autoregressive (AR) process to generate video textures by replacing PCA with five other dimension reduction techniques. Based on our experiments, using these dimensionality reduction techniques has improved the quality of video textures compared with extraction of frame signatures using PCA. The synthesized video textures may contain similar motions as the input video and will never be repeated exactly. All frames synthesized have never appeared before. We also propose a new approach for generating video textures using probabilistic principal components analysis (PPCA) and Gaussian process dynamical model (GPDM). GPDM is a nonparametric model for learning high-dimensional nonlinear dynamical data sets. We adopt PPCA and GPDM on several movie clips to synthesize video textures which contain frames that never appeared before and with similar motions as original videos. Furthermore, we have proposed two ways of generating real-time video textures by applying the incremental Isomap and incremental Spati04emporal Isomap (IST-Isomap). Both approaches can produce good real-time video texture results. In particular, IST-Isomap, that we propose, is more suitable for sparse video data (e.g. cartoon)

[1]  竹安 数博,et al.  Time series analysis and its applications , 2007 .

[2]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[3]  Daniel Cremers,et al.  Dynamic texture segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Stochastic Rigidity: Image Registration for Nowhere-Static Scenes , 2001, ICCV.

[5]  Adam Finkelstein,et al.  WYSIWYG NPR: drawing strokes directly on 3D models , 2002, SIGGRAPH.

[6]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[7]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[8]  David Salesin,et al.  Panoramic video textures , 2005, ACM Trans. Graph..

[9]  Harry Shum,et al.  Motion texture: a two-level statistical model for character motion synthesis , 2002, ACM Trans. Graph..

[10]  David J. Fleet,et al.  Gaussian Process Dynamical Models , 2005, NIPS.

[11]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[12]  Victor B. Zordan,et al.  Animated People Textures , 2004 .

[13]  James D. Foley,et al.  Getting There: The Ten Top Problems Left , 2000, IEEE Computer Graphics and Applications.

[14]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[15]  Anil K. Jain,et al.  Incremental nonlinear dimensionality reduction by manifold learning , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Shien-Ming Wu,et al.  Time series and system analysis with applications , 1983 .

[17]  Bobby Bodenheimer,et al.  Cartoon textures , 2004, SCA '04.

[18]  David J. Fleet,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Gaussian Process Dynamical Model , 2007 .

[19]  Okan Arikan,et al.  Interactive motion generation from examples , 2002, ACM Trans. Graph..

[20]  Manuel Menezes de Oliveira Neto.  Image-Based Modeling and Rendering Techniques: A Survey , 2002, RITA.

[21]  Alan Watt,et al.  3D Games: Real-Time Rendering and Software Technology , 2000 .

[22]  Irfan A. Essa,et al.  Controlled animation of video sprites , 2002, SCA '02.

[23]  Iain Murray.  Introduction To Gaussian Processes , 2008 .

[24]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[25]  Irfan A. Essa,et al.  Machine Learning for Video-Based Rendering , 2000, NIPS.

[26]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[27]  Of references. , 1966, JAMA.

[28]  Neill W. Campbell,et al.  Practical generation of video textures using the auto-regressive process , 2004, Image Vis. Comput..

[29]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[30]  Sean Hayes,et al.  View synthesis by trinocular edge matching and transfer , 2000, Image Vis. Comput..

[31]  Chun-Xia Zhang,et al.  A video texture synthesis method based on wavelet transform , 2007, 2007 International Conference on Wavelet Analysis and Pattern Recognition.

[32]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[33]  Mirella M. Moro,et al.  Modelo Temporal de Versões , 2002, RITA.

[34]  Lucas Kovar,et al.  Motion Graphs , 2002, ACM Trans. Graph..

[35]  Maja J. Mataric,et al.  A spatio-temporal extension to Isomap nonlinear dimension reduction , 2004, ICML.

[36]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[37]  David Salesin,et al.  Multiresolution video , 1996, SIGGRAPH.

[38]  Harry Shum,et al.  Review of image-based rendering techniques , 2000, Visual Communications and Image Processing.

[39]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[40]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[41]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[42]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[43]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[44]  Nizar Bouguila,et al.  On video textures generation: A comparison between different dimensionality reduction techniques , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[45]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.