Detecting Spatiotemporal Structure Boundaries: Beyond Motion Discontinuities

The detection of motion boundaries has been and remains a long-standing challenge in computer vision. In this paper, the recovery of motion boundaries is recast in a broader scope, as focus is placed on the more general problem of detecting spacetime structure boundaries, where motion boundaries constitute a special case. This recasting allows uniform consideration of boundaries between a wider class of spacetime patterns than previously considered in the literature, both coherent motion as well as additional dynamic patterns. Examples of dynamic patterns beyond standard motion that are encompassed by the proposed approach include, flicker, transparency and various dynamic textures (e.g., scintillation). Toward this end, a novel representation and method for detecting these boundaries in raw image sequence data are presented. Central to the representation is the description of oriented spacetime structure in a distributed manner. Empirical evaluation of the proposed boundary detector on challenging natural imagery suggests its efficacy.

[1]  Allan D. Jepson,et al.  Benchmarking Image Segmentation Algorithms , 2009, International Journal of Computer Vision.

[2]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Daphna Weinshall,et al.  Motion Segmentation and Depth Ordering Using an Occlusion Detector , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  David J. Fleet,et al.  Motion feature detection using steerable flow fields , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[5]  William B. Thompson,et al.  Analysis of Accretion and Deletion at Boundaries in Dynamic Scenes , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Martial Hebert,et al.  Local detection of occlusion boundaries in video , 2009, Image Vis. Comput..

[9]  Christopher M. Bishop,et al.  Non-linear Bayesian Image Modelling , 2000, ECCV.

[10]  Robyn A. Owens,et al.  Feature detection from local energy , 1987, Pattern Recognit. Lett..

[11]  David J. Fleet,et al.  Probabilistic Detection and Tracking of Motion Boundaries , 2000, International Journal of Computer Vision.

[12]  Andrew W. Fitzgibbon,et al.  Learning spatiotemporal T-junctions for occlusion detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Richard P. Wildes,et al.  Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation , 2000, ECCV.

[15]  Anselm Spoerri,et al.  The early detection of motion boundaries , 1990, ICCV 1987.

[16]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[17]  C. Tomasi.  Coalescing Texture Descriptors , 1996 .

[18]  Andrew B. Watson,et al.  A look at motion in the frequency domain , 1983 .

[19]  Daniel Cremers,et al.  Dynamic texture segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Konstantinos G. Derpanis,et al.  Three-dimensional nth derivative of Gaussian separable steerable filters , 2005, IEEE International Conference on Image Processing 2005.

[21]  P. Anandan,et al.  Computing Dense Displacement Fields With Confidence Measures In Scenes Containing Occlusion , 1985, Other Conferences.

[22]  Sourabh A. Niyogi,et al.  Detecting kinetic occlusion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[23]  Guillermo Sapiro,et al.  Anisotropic diffusion of multivalued images with applications to color filtering , 1996, IEEE Trans. Image Process..

[24]  Valdis Berzins,et al.  Dynamic Occlusion Analysis in Optical Flow Fields , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.