Non-Linear Dynamic Texture Analysis and Synthesis Using Constrained Gaussian Process Latent Variable Model

Linear dynamic system (LDS) has been proposed to model dynamic texture. However, the temporal evolution of dynamic texture is non-linear in general and is not fully captured by the linear model. In this paper, we formulate the dynamic texture learning and synthesis via nonlinear approach. Assuming that dynamic texture is sampled from a low dimensional manifold, the constrained Gaussian process latent variable model (CGPLVM) is proposed to model the dynamic texture as a set of latent states. The essence of dynamic texture is captured as the spatial relationship within the latent states. Moreover, Metropolis-Hastings sampling method is used to sample new states, which hold the spatio-temporal statistics of dynamic texture. Experimental results demonstrate that our approach can produce dynamic texture sequences with promising visual quality.

[1]  Stochastic Rigidity: Image Registration for Nowhere-Static Scenes , 2001, ICCV.

[2]  Payam Saisan,et al.  Dynamic texture recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Q. Shi,et al.  Gaussian Process Latent Variable Models for , 2011 .

[4]  Martin Szummer,et al.  Temporal texture modeling , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[5]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[6]  Stefano Soatto,et al.  Editable dynamic textures , 2002, SIGGRAPH '02.

[7]  Andrea Masiero,et al.  Non Linear Temporal Textures Synthesis: A Monte Carlo Approach , 2006, ECCV.

[8]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[9]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[10]  Narendra Ahuja,et al.  Dynamic Textures Synthesis as Nonlinear Manifold Learning and Traversing , 2006, BMVC.

[11]  Song-Chun Zhu,et al.  A Generative Method for Textured Motion: Analysis and Synthesis , 2002, ECCV.

[12]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[13]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[14]  Harry Shum,et al.  Synthesizing Dynamic Texture with Closed-Loop Linear Dynamic System , 2004, ECCV.

[15]  Wen Gao,et al.  Monocular Tracking 3D People By Gaussian Process Spatio-Temporal Variable Model , 2007, 2007 IEEE International Conference on Image Processing.

[16]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[17]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[18]  A. Fitzgibbon.  Stochastic rigidity: image registration for nowhere-static scenes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.