Early spatiotemporal grouping with a distributed oriented energy representation

Spatiotemporal data is associated with vast amounts of raw samples. Given the limited computational resources typically available, an initial organization of this data supporting semantically meaningful lines of inquiry would facilitate efficient processing. In this paper, a new representation for grouping raw image data into a set of coherent spacetime regions is proposed. Unique in this proposal is that coherency is related to a richer description of local spacetime structure than generally considered. In particular, the representation describes the presence of particular oriented spacetime structures in a distributed manner. A key advantage of this representation is its ability to signal the presence of multiple oriented structures at a given spacetime location. More generally, the abstraction allows for the description and grouping of motion and non-motion-related patterns in a uniform manner. Empirical evaluation of the grouping method on synthetic and challenging natural imagery suggests its efficacy.

[1]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[2]  Andrew B. Watson,et al.  A look at motion in the frequency domain , 1983 .

[3]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[4]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Randal C. Nelson,et al.  Qualitative recognition of motion using temporal texture , 1992, CVGIP Image Underst..

[6]  Eero P. Simoncelli,et al.  Separation of transparent motion into layers using velocity-tuned mechanisms , 1994 .

[7]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[8]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[9]  James L. Crowley,et al.  Probabilistic recognition of activity using local appearance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[10]  Richard P. Wildes,et al.  Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation , 2000, ECCV.

[11]  Jitendra Malik,et al.  Efficient spatiotemporal grouping using the Nystrom method , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Daniel DeMenthon,et al.  SPATIO-TEMPORAL SEGMENTATION OF VIDEO BY HIERARCHICAL MEAN SHIFT ANALYSIS , 2002 .

[13]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Gérard G. Medioni,et al.  Layered 4D Representation and Voting for Grouping from Motion , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Daniel Cremers,et al.  Variational space-time motion segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[16]  Daniel Cremers,et al.  Dynamic texture segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Hayit Greenspan,et al.  Probabilistic space-time video modeling via piecewise GMM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Allan D. Jepson,et al.  Quantitative evaluation of a novel image segmentation algorithm , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[20]  Konstantinos G. Derpanis,et al.  Three-dimensional nth derivative of Gaussian separable steerable filters , 2005, IEEE International Conference on Image Processing 2005.

[21]  Thomas Brox,et al.  Variational Motion Segmentation with Level Sets , 2006, ECCV.

[22]  Richard P. Wildes,et al.  Spatiotemporal Oriented Energy Features for Visual Tracking , 2007, ACCV.

[23]  Nuno Vasconcelos,et al.  Modeling, Clustering, and Segmenting Video with Mixtures of Dynamic Textures , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  R. Wildes,et al.  Spatiotemporal stereo via spatiotemporal quadric element (stequel) matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.