M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene

When occlusion is minimal, a single camera is generally sufficient to detect and track objects. However, when the density of objects is high, the resulting occlusion and lack of visibility suggests the use of multiple cameras and collaboration between them so that an object is detected using information available from all the cameras in the scene.In this paper, we present a system that is capable of segmenting, detecting and tracking multiple people in a cluttered scene using multiple synchronized surveillance cameras located far from each other. The system is fully automatic, and takes decisions about object detection and tracking using evidence collected from many pairs of cameras. Innovations that help us tackle the problem include a region-based stereo algorithm capable of finding 3D points inside an object knowing only the projections of the object (as a whole) in two views, a segmentation algorithm using bayesian classification and the use of occlusion analysis to combine evidence from different camera pairs.The system has been tested using different densities of people in the scene. This helps us determine the number of cameras required for a particular density of people. Experiments have also been conducted to verify and quantify the efficacy of the occlusion analysis scheme.

[1]  Radu Horaud,et al.  Stereo Correspondence Through Feature Grouping and Maximal Cliques , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Laurent Vinet,et al.  Hierarchical region based stereo matching , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[4]  Andy Hopper,et al.  The active badge location system , 1992, TOIS.

[5]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Aaron F. Bobick,et al.  Closed-world tracking , 1995, Proceedings of IEEE International Conference on Computer Vision.

[7]  J. Simonoff.  Multivariate Density Estimation , 1996 .

[8]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[9]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  James W. Davis,et al.  Real-time closed-world tracking , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Andrew Zisserman,et al.  Wide baseline stereo matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[12]  Larry S. Davis,et al.  W/sup 4/: Who? When? Where? What? A real time system for detecting and tracking people , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[13]  Olivier D. Faugeras,et al.  Complete Dense Stereovision Using Level Set Methods , 1998, ECCV.

[14]  Larry S. Davis,et al.  W4S: A real-time system detecting and tracking people in 2 1/2D , 1998, ECCV.

[15]  Jake K. Aggarwal,et al.  Automatic tracking of human motion in indoor scenes across multiple synchronized video streams , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  Ramin Zabih,et al.  Bayesian multi-camera surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[17]  Paolo Remagnino,et al.  Multi-Camera Color Tracking , 1999 .

[18]  Paolo Remagnino,et al.  A multi-agent framework for visual surveillance , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[19]  Larry S. Davis,et al.  Hydra: multiple people detection and tracking using silhouettes , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[20]  Rómer Rosales,et al.  3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[21]  Ramin Zabih,et al.  Counting people from multiple cameras , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[22]  Andrew Blake,et al.  A Probabilistic Exclusion Principle for Tracking Multiple Objects , 2000, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[24]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[25]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[26]  Daniel P. Huttenlocher,et al.  Scene modeling for wide area surveillance and image synthesis , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[27]  R. Zabih,et al.  Exact voxel occupancy with graph cuts , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Larry S. Davis,et al.  Efficient non-parametric adaptive color modeling using fast Gauss transform , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[30]  Trevor Darrell,et al.  Plan-view trajectory estimation with dense stereo background models , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  Larry S. Davis,et al.  Unified multi-camera detection and tracking using region-matching , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[32]  Larry S. Davis,et al.  M2Tracker: A Multi-view Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo , 2002, ECCV.

[33]  Larry S. Davis,et al.  Design and Implementation of the University of Maryland Keck Laboratory for the Analysis of Visual Movement , 2002 .

[34]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[35]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[36]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.