Dynamic texture recognition based on distributions of spacetime oriented structure

This paper addresses the challenge of recognizing dynamic textures based on their observed visual dynamics. Typically, the term dynamic texture is used with reference to image sequences of various natural processes that exhibit stochastic dynamics (e.g., smoke, water and windblown vegetation); although, it applies equally well to images of simpler dynamics when analyzed in terms of aggregate region properties (e.g., uniform motion of elements in traffic video). In this paper, a novel approach to dynamic texture representation and an associated recognition method are proposed. The approach pursued here recognizes dynamic textures based on matching distributions (histograms) of spacetime orientation structure. Empirical evaluation on a standard database with controls to remove the effects of identical viewpoint demonstrates that the proposed approach achieves superior performance over alternative state-of-the-art methods.

[1]  Suzanne Beauchemin,et al.  The Frequency Structure of 1D Occluding Image Signals , 2000 .

[2]  Patrick Bouthemy,et al.  Motion characterization from temporal cooccurrences of local motion-based measures for video indexing , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[3]  Richard P. Wildes,et al.  Early spatiotemporal grouping with a distributed oriented energy representation , 2009, CVPR.

[4]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[5]  Martin Szummer,et al.  Temporal texture modeling , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[6]  David G. Stork,et al.  Pattern Classification , 1973 .

[7]  Dani Lischinski,et al.  Texture Mixing and Texture Movie Synthesis Using Statistical Learning , 2001, IEEE Trans. Vis. Comput. Graph..

[8]  K. W. Cattermole.  The Fourier Transform and its Applications , 1965 .

[9]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Steven S. Beauchemin,et al.  The Frequency Structure of One-Dimensional Occluding Image Signals , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Alexander J. Smola,et al.  Binet-Cauchy Kernels on Dynamical Systems and its Application to the Analysis of Dynamic Scenes , 2007, International Journal of Computer Vision.

[12]  Song-Chun Zhu,et al.  Modeling textured motion : particle, wave and sketch , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[14]  Weichuan Yu,et al.  Detection and characterization of multiple motion points , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  A. Fitzgibbon.  Stochastic rigidity: image registration for nowhere-static scenes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  T. Poggio,et al.  Visual hyperacuity: spatiotemporal interpolation in human vision , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[17]  Tony Lindeberg,et al.  Linear Spatio-Temporal Scale-Space , 1997, Scale-Space.

[18]  Weixin Xie,et al.  Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[19]  Dmitry Chetverikov,et al.  A Brief Survey of Dynamic Texture Description and Recognition , 2005, CORES.

[20]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[21]  Nuno Vasconcelos,et al.  Probabilistic kernels for the classification of auto-regressive visual processes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Robert Sekuler,et al.  Coherent global motion percepts from stochastic local motions , 1984, Vision Research.

[23]  Richard P. Wildes,et al.  Spatiotemporal stereo via spatiotemporal quadric element (stequel) matching , 2009, CVPR.

[24]  David J. Fleet.  Measurement of image velocity , 1992 .

[25]  Robert Sekuler,et al.  Using metamers to explore motion perception , 1991, Vision Research.

[26]  Payam Saisan,et al.  Dynamic texture recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  James L. Crowley,et al.  Probabilistic recognition of activity using local appearance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  Richard P. Wildes,et al.  Spatiotemporal Oriented Energy Features for Visual Tracking , 2007, ACCV.

[29]  Ramprasad Polana,et al.  Temporal texture and activity recognition , 1994 .

[30]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[31]  Nuno Vasconcelos,et al.  Classifying Video with Kernel Dynamic Textures , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Andrew B. Watson,et al.  A look at motion in the frequency domain , 1983 .

[33]  Konstantinos G. Derpanis,et al.  Three-dimensional nth derivative of Gaussian separable steerable filters , 2005, IEEE International Conference on Image Processing 2005.

[34]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[35]  Richard P. Wildes,et al.  Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation , 2000, ECCV.

[36]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[37]  Andrew W. Fitzgibbon,et al.  Shift-Invariant Dynamic Texture Recognition , 2006, ECCV.

[38]  Randal C. Nelson,et al.  Qualitative recognition of motion using temporal texture , 1992, CVGIP Image Underst..

[39]  Andrew Zisserman,et al.  A Statistical Approach to Material Classification Using Image Patch Exemplars , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  David J. Heeger,et al.  Seeing structure through chaos , 1986 .

[41]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[42]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.