Early spatiotemporal grouping with a distributed oriented energy representation

Spatiotemporal data is associated with vast amounts of raw samples. Given the limited computational resources typically available, an initial organization of this data supporting semantically meaningful lines of inquiry would facilitate efficient processing. In this paper, a new representation for grouping raw image data into a set of coherent spacetime regions is proposed. Unique in this proposal is that coherency is related to a richer description of local spacetime structure than generally considered. In particular, the representation describes the presence of particular oriented spacetime structures in a distributed manner. A key advantage of this representation is its ability to signal the presence of multiple oriented structures at a given spacetime location. More generally, the abstraction allows for the description and grouping of motion and non-motion-related patterns in a uniform manner. Empirical evaluation of the grouping method on synthetic and challenging natural imagery suggests its efficacy.

[1]  Richard P. Wildes,et al.  Spatiotemporal stereo via spatiotemporal quadric element (stequel) matching , 2009, CVPR.

[2]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[3]  Konstantinos G. Derpanis,et al.  Three-dimensional nth derivative of Gaussian separable steerable filters , 2005, IEEE International Conference on Image Processing 2005.

[4]  Randal C. Nelson,et al.  Qualitative recognition of motion using temporal texture , 1992, CVGIP Image Underst..

[5]  Eero P. Simoncelli,et al.  Separation of transparent motion into layers using velocity-tuned mechanisms , 1994 .

[6]  Hayit Greenspan,et al.  Probabilistic space-time video modeling via piecewise GMM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[8]  Andrew B. Watson,et al.  A look at motion in the frequency domain , 1983 .

[9]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Allan D. Jepson,et al.  Quantitative evaluation of a novel image segmentation algorithm , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Nuno Vasconcelos,et al.  Modeling, Clustering, and Segmenting Video with Mixtures of Dynamic Textures , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Daniel Cremers,et al.  Variational space-time motion segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[14]  Daniel Cremers,et al.  Dynamic texture segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[16]  Thomas Brox,et al.  Variational Motion Segmentation with Level Sets , 2006, ECCV.

[17]  Gérard G. Medioni,et al.  Layered 4D Representation and Voting for Grouping from Motion , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Jitendra Malik,et al.  Efficient spatiotemporal grouping using the Nystrom method , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  James L. Crowley,et al.  Probabilistic recognition of activity using local appearance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[21]  Richard P. Wildes,et al.  Spatiotemporal Oriented Energy Features for Visual Tracking , 2007, ACCV.

[22]  Richard P. Wildes,et al.  Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation , 2000, ECCV.

[23]  S. Ullman,et al.  The interpretation of visual motion , 1977 .