Local detection of occlusion boundaries in video

Occlusion boundaries are notoriously difficult for many pat ch-based computer vision algorithms, but they also provide potentially useful information about scene structure and shape. Using short video clips, we pres nt a novel method for scoring the degree to which edges exhibit occlusi on. We first utilize a spatio-temporal edge detector which estimates ed ge strength, orientation, and normal motion. By then extracting patches from e ither side of each detected (possibly moving) edglet, we can estimate and compare motion to determine if occlusion is present. This completely l ocal, bottom-up approach is intended to provide powerful low-level informa tion for use by higher-level reasoning methods.

[1]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Bruce A. Maxwell,et al.  Texture Edge Detection Using the Compass Operator , 2003, BMVC.

[3]  Jitendra Malik,et al.  Detecting and localizing edges composed of steps, peaks and roofs , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[4]  Carlo Tomasi,et al.  Color edge detection with the compass operator , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[5]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[6]  Patrick Bouthemy,et al.  A Maximum Likelihood Framework for Determining Moving Edges , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[8]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Takeo Kanade,et al.  A robust subspace approach to layer extraction , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[10]  Martial Hebert,et al.  Using Spatio-Temporal Patches for Simultaneous Estimation of Edge Strength, Orientation, and Motion , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[11]  Emanuele Trucco,et al.  Efficient stereo with multiple windowing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Eli Shechtman,et al.  Space-time behavior based correlation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[14]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[15]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Andrew W. Fitzgibbon,et al.  Learning spatiotemporal T-junctions for occlusion detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Martial Hebert,et al.  Incorporating Background Invariance into Feature-Based Object Recognition , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[18]  Jonathan M. Garibaldi,et al.  Real-Time Correlation-Based Stereo Vision with Reduced Border Errors , 2002, International Journal of Computer Vision.

[19]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Jitendra Malik,et al.  Motion segmentation and tracking using normalized cuts , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[21]  David J. Fleet,et al.  Probabilistic detection and tracking of motion disconti - nuities In International Journal of Comput , 2000 .

[22]  Paul Smith,et al.  Layered motion segmentation and depth ordering by tracking edges , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.