Segmentation and Tracking Multiple Objects Under Occlusion From Multiview Video

In this paper, we present a multiview approach to segment the foreground objects consisting of a group of people into individual human objects and track them across the video sequence. Depth and occlusion information recovered from multiple views of the scene is integrated into the object detection, segmentation, and tracking processes. Adaptive background penalty with occlusion reasoning is proposed to separate the foreground regions from the background in the initial frame. Multiple cues are employed to segment individual human objects from the group. To propagate the segmentation through video, each object region is independently tracked by motion compensation and uncertainty refinement, and the motion occlusion is tackled as layer transition. The experimental results implemented on both our sequences and other's sequence have demonstrated the algorithm's efficiency in terms of subjective performance. Objective comparison with a state-of-the-art algorithm validates the superior performance of our method quantitatively.

[1]  Pascal Fua,et al.  Multicamera People Tracking with a Probabilistic Occupancy Map , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Larry S. Davis,et al.  M2Tracker: A Multi-view Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo , 2002, ECCV.

[3]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[4]  King Ngi Ngan,et al.  Dense Stereo Matching from Separated Views of Wide-Baseline Images , 2010, ACIVS.

[5]  Andrew Blake,et al.  Probabilistic Fusion of Stereo with Color and Contrast for Bi-Layer Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ramakant Nevatia,et al.  Tracking multiple humans in complex situations , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Larry S. Davis,et al.  Multi-camera Tracking and Segmentation of Occluded People on Ground Plane Using Search-Guided Particle Filtering , 2006, ECCV.

[8]  Andrew Blake,et al.  Probabilistic Fusion of Stereo with Color and Contrast for Bilayer Segmentation , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Matthew Klimesh,et al.  Exploiting Calibration-Induced Artifacts in Lossless Compression of Hyperspectral Imagery , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[12]  External Calibration of Multi-camera System Based on Efficient Pair-wise Estimation , 2007, PSIVT.

[13]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[15]  W. Yang,et al.  Edge-preserving regularization of disparity and motion fields , 2003, Proceedings EC-VIP-MC 2003. 4th EURASIP Conference focused on Video/Image Processing and Multimedia Communications (IEEE Cat. No.03EX667).

[16]  V. Vapnik Pattern recognition using generalized portrait method , 1963 .

[17]  Larry S. Davis,et al.  Probabilistic framework for segmenting people under occlusion , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Mubarak Shah,et al.  Tracking Multiple Occluding People by Localizing on Multiple Scene Planes , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Larry S. Davis,et al.  Simultaneous Appearance Modeling and Segmentation for Matching People Under Occlusion , 2007, ACCV.

[20]  Dan Schonfeld,et al.  Distributed Bayesian Multiple-Target Tracking in Crowded Environments Using Multiple Collaborative Cameras , 2007, EURASIP J. Adv. Signal Process..

[21]  A. Murat Tekalp,et al.  Semi-automatic video object segmentation in the presence of occlusion , 2000, IEEE Trans. Circuits Syst. Video Technol..

[22]  King Ngi Ngan,et al.  Multi-view video based multiple objects segmentation using graph cut and spatiotemporal projections , 2010, J. Vis. Commun. Image Represent..

[23]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[24]  Yiannis Aloimonos,et al.  Motion segmentation using occlusions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.