Multi view image surveillance and tracking

The paper presents a set of methods for multi view image tracking using a set of calibrated cameras. We demonstrate how effective the approach is for resolving occlusions and tracking objects between overlapping and non-overlapping camera views. Moving objects are initially detected using background subtraction. Temporal alignment is then performed between each video sequence in order to compensate for the different processing rates of each camera. A Kalman filter is used to track each object in 3D world coordinates and 2D image coordinates. Information is shared between the 2D/3D trackers of each camera view in order to improve the performance of object tracking and trajectory prediction. The system is shown to be robust in resolving dynamic and static object occlusions. Results are presented from a variety of outdoor surveillance video sequences.

[1]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[2]  Takeo Kanade,et al.  A Cooperative Algorithm for Stereo Matching and Occlusion Detection , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Shaogang Gong,et al.  Tracking Multiple People Under Occlusion Using Multiple Cameras , 2000, BMVC.

[4]  C. Jaynes.  Multi-view calibration from planar motion for video surveillance , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[5]  Jake K. Aggarwal,et al.  Human motion analysis: a review , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[6]  Zehang Sun,et al.  A distributed visual surveillance system , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[7]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[8]  Dorin Comaniciu,et al.  Robust detection and tracking of human faces with an active camera , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Jeffrey K. Uhlmann,et al.  A non-divergent estimation algorithm in the presence of unknown correlations , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[11]  Larry S. Davis,et al.  M2Tracker: A Multi-view Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo , 2002, ECCV.

[12]  Tim J. Ellis,et al.  Automatic learning of an activity-based semantic scene model , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[13]  A. M. Tekalp,et al.  Multiple camera fusion for multi-object tracking , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[14]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 2004, International Journal of Computer Vision.

[15]  Jakub Segen,et al.  Performance evaluation of people tracking systems , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[16]  James Orwell,et al.  Learning Surveillance Tracking Models for the Self-Calibrated Ground Plane , 2002, BMVC.

[17]  D. Baker,et al.  Specification of a video test imagery library (VITAL) , 2003, IEEE 37th Annual 2003 International Carnahan Conference onSecurity Technology, 2003. Proceedings..

[18]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[19]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[20]  Larry S. Davis,et al.  An appearance-based body model for multiple people tracking , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  M. Shah,et al.  KNIGHT M : A REAL TIME SURVEILLANCE SYSTEM FOR MULTIPLE OVERLAPPING AND NON-OVERLAPPING CAMERAS , 2003 .

[22]  Stan Sclaroff,et al.  Improved Tracking of Multiple Humans with Trajectory Predcition and Occlusion Modeling , 1998 .

[23]  Ramin Zabih,et al.  Bayesian multi-camera surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[24]  Ramakant Nevatia,et al.  Self-calibration of a camera from video of a walking human , 2002, Object recognition supported by user interaction for service robots.

[25]  Roger D. Boyle,et al.  Tracking multiple sports players through occlusion, congestion and scale , 2001, BMVC.

[26]  Larry S. Davis,et al.  Hydra: multiple people detection and tracking using silhouettes , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[27]  Lily Lee,et al.  Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  G. T. Kogut,et al.  Maintaining the identity of multiple vehicles as they travel through a video network , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[30]  L. Gool,et al.  Color features for tracking non-rigid objects , 2003 .

[31]  Tim J. Ellis,et al.  Partial Observation vs. Blind Tracking through Occlusion , 2002, BMVC.

[32]  Bülent Sankur,et al.  Performance evaluation metrics for object-based video segmentation , 2000, 2000 10th European Signal Processing Conference.

[33]  Gideon P. Stein,et al.  Tracking from multiple view points: Self-calibration of space and time , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[34]  B.M. Junior,et al.  Object detection with multiple cameras , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[35]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[36]  David Beymer,et al.  Person counting using stereo , 2000, Proceedings Workshop on Human Motion.

[37]  Chris Stauffer,et al.  Automated multi-camera planar tracking correspondence modeling , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[38]  Mubarak Shah,et al.  KNIGHT/spl trade/: a real time surveillance system for multiple and non-overlapping cameras , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[39]  Mubarak Shah,et al.  Human Tracking in Multiple Cameras , 2001, ICCV.

[40]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[41]  Mohan M. Trivedi,et al.  Database architecture for autonomous transportation agents for on-scene networked incident management (ATON) , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[42]  Jake K. Aggarwal,et al.  Automatic tracking of human motion in indoor scenes across multiple synchronized video streams , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[43]  Aaron F. Bobick,et al.  Closed-world tracking , 1995, Proceedings of IEEE International Conference on Computer Vision.

[44]  R. German.  Sintering theory and practice , 1996 .

[45]  J. Ferryman,et al.  International Workshop on Performance Evaluation of Tracking and Surveillance ( PETS ’ 2002 ) , 2002 .

[46]  Hironobu Fujiyoshi,et al.  Real-time human motion analysis by image skeletonization , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[47]  David S. Doermann,et al.  Tools and techniques for video performance evaluation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[48]  Mubarak Shah,et al.  Tracking across multiple cameras with disjoint views , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[49]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[50]  A. Murat Tekalp,et al.  Non-rigid object tracking using performance evaluation measures as feedback , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[51]  Jake K. Aggarwal,et al.  Tracking human motion using multiple cameras , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[52]  Fernando Pereira,et al.  MPEG-7 the generic multimedia content description standard, part 1 - Multimedia, IEEE , 2001 .

[53]  Tim J. Ellis,et al.  Spatial and Probabilistic Modelling of Pedestrian Behaviour , 2002, BMVC.

[54]  Larry S. Davis,et al.  Unified multi-camera detection and tracking using region-matching , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[55]  Tim Ellis.  Performance metrics and methods for tracking in surveillance , 2002 .

[56]  Mubarak Shah,et al.  Camera handoff: tracking in multiple uncalibrated stationary cameras , 2000, Proceedings Workshop on Human Motion.

[57]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[58]  Ramin Zabih,et al.  Counting people from multiple cameras , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[59]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[60]  Tim Ellis,et al.  A multi-view surveillance system , 2003 .

[61]  Adrian Hilton,et al.  Towards reliable real-time multiview tracking , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[62]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[63]  Roger D. Boyle,et al.  Performance Evaluation Metrics and Statistics for Positional Tracker Evaluation , 2003, ICVS.

[64]  Paolo Remagnino,et al.  Automated registration of surveillance data for multi-camera fusion , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[65]  James W. Davis,et al.  Real-time closed-world tracking , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[66]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[67]  Mohan M. Trivedi,et al.  Activity monitoring and summarization for an intelligent meeting room , 2000, Proceedings Workshop on Human Motion.

[68]  Takuya Kondo,et al.  Incremental tracking of human actions from multiple views , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[69]  Gérard G. Medioni,et al.  Continuous tracking within and across camera streams , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[70]  James E. Black,et al.  A novel method for video tracking performance evaluation , 2003 .

[71]  Charles V. Stewart,et al.  Robust Parameter Estimation in Computer Vision , 1999, SIAM Rev..

[72]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[73]  Mohammed Yeasin,et al.  Tracking body parts of multiple people: a new approach , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[74]  Ian D. Reid,et al.  A plane measuring device , 1999, Image Vis. Comput..

[75]  Tim J. Ellis,et al.  Illumination-Invariant Motion Detection Using Colour Mixture Models , 2001, BMVC.

[76]  Xiaodong Cai,et al.  MPEG4 over local area mobile surveillance system , 2003 .

[77]  Jeffrey E. Boyd,et al.  MPI-Video infrastructure for dynamic environments , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[78]  James E. Black,et al.  Intelligent Image Surveillance and Monitoring , 2002 .

[79]  Shaogang Gong,et al.  Tracking multiple people with a multi-camera system , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[80]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[81]  Gérard G. Medioni,et al.  Detecting and tracking moving objects for video surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[82]  Paolo Remagnino,et al.  Multi-camera colour tracking , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[83]  Mubarak Shah,et al.  Tracking in uncalibrated cameras with overlapping field of view , 2001 .

[84]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[85]  W. Eric L. Grimson,et al.  Answering Questions about Moving Objects in Surveillance Videos , 2003, New Directions in Question Answering.

[86]  Christopher O. Jaynes,et al.  An Open Development Environment for Evaluation of Video Surveillance Systems , 2002 .

[87]  Jake K. Aggarwal,et al.  Tracking Human Motion in Structured Environments Using a Distributed-Camera System , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[88]  Tim Ellis,et al.  Tracking occluded objects using partial observation , 2003 .

[89]  A. Murat Tekalp,et al.  Metrics for performance evaluation of video object segmentation and tracking without ground-truth , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[90]  Larry S. Davis,et al.  Active outdoor surveillance , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[91]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[92]  Yaron Caspi,et al.  Alignment of non-overlapping sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[93]  Tim Ellis,et al.  Multi-camera image measurement and correspondence , 2002 .

[94]  Myron Flickner,et al.  Detection and tracking of shopping groups in stores , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[95]  Mohan M. Trivedi,et al.  Detecting Moving Shadows: Algorithms and Evaluation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[96]  T. Ellis,et al.  Wide area surveillance with a multi camera network , 2004 .

[97]  Tim J. Ellis,et al.  Bridging the gaps between cameras , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..