Algorithms for cooperative multisensor surveillance

The Video Surveillance and Monitoring (VSAM) team at Carnegie Mellon University (CMU) has developed an end-to-end, multicamera surveillance system that allows a single human operator to monitor activities in a cluttered environment using a distributed network of active video sensors. Video understanding algorithms have been developed to automatically detect people and vehicles, seamlessly track them using a network of cooperating active sensors, determine their three-dimensional locations with respect to a geospatial site model, and present this information to a human operator who controls the system through a graphical user interface. The goal is to automatically collect and disseminate real-time information to improve the situational awareness of security providers and decision makers. The feasibility of real-time video surveillance has been demonstrated within a multicamera testbed system developed on the campus of CMU. This paper presents an overview of the issues and algorithms involved in creating this semiautonomous, multicamera surveillance system.

[1]  Hans-Hellmut Nagel,et al.  Influence of an Explicitly Modelled 3D Scene on the Tracking of Partially Occluded Vehicles , 1997, Comput. Vis. Image Underst..

[2]  Thomas L. Clarke.  Distributed Interactive Simulation Systems for Simulation and Training in the Aerospace Environment , 1995 .

[3]  Hironobu Fujiyoshi,et al.  Real-time human motion analysis by image skeletonization , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[4]  J. E. Smith.  Recent Developments in ModSAF Terrain Representation , 1995 .

[5]  M. Thonnat,et al.  Video sequence interpretation for visual surveillance , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[6]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[7]  Tieniu Tan,et al.  Multi-agent visual surveillance of dynamic scenes , 1998, Image Vis. Comput..

[8]  Matthew Brand,et al.  Discovery and Segmentation of Activities in Video , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Tim J. Ellis,et al.  Image Difference Threshold Strategies and Shadow Detection , 1995, BMVC.

[10]  Robert T. Collins,et al.  Calibration of an outdoor active camera system , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[11]  Patrick Bouthemy,et al.  Real-Time Tracking of Moving Persons by Exploiting Spatio-Temporal Image Slices , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  G. Strang,et al.  Linear Algebra, Geodesy, and GPS , 1997 .

[13]  P. J. Burt,et al.  Change Detection and Tracking Using Pyramid Transform Techniques , 1985, Other Conferences.

[14]  Shaogang Gong,et al.  Visual Surveillance in a Dynamic and Uncertain World , 1995, Artif. Intell..

[15]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[16]  David C. Hogg,et al.  Generating Spatiotemporal Models from Examples , 1995, BMVC.

[17]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[18]  Lambert E. Wixson,et al.  Image alignment for precise camera fixation and aim , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[19]  Hironobu Fujiyoshi,et al.  A System for Video Surveillance and Monitoring CMU VSAM Final Report , 1999 .

[20]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[21]  Pramod K. Varshney,et al.  Special issue on data fusion , 1997 .

[22]  Gian Luca Foresti.  A real-time system for video surveillance of unattended outdoor environments , 1998, IEEE Trans. Circuits Syst. Video Technol..

[23]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Thomas Boutell.  CGI programming in C and Perl - featuring C and Perl 5 source code , 1996 .

[25]  Kurt Maly,et al.  Modeling and Distributed Simulation Techniques for Synthetic Training Environments , 1996 .

[26]  Monique Thonnat,et al.  The PASSWORDS Project [intelligent video image analysis system] , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[27]  Aaron F. Bobick,et al.  Recognition of Visual Activities and Interactions by Stochastic Parsing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[29]  Jitendra Malik,et al.  A real-time computer vision system for measuring traffic parameters , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Alan J. Lipton.  Local Application of Optic Flow to Analyse Rigid versus Non-Rigid Motion , 1999 .

[31]  Stuart J. Russell,et al.  Object Identification: A Bayesian Analysis with Application to Traffic Surveillance , 1998, Artif. Intell..

[32]  Mikel D. Petty.  Computer-generated forces in distributed interactive simulation , 1995, Defense + Commercial Sensing.

[33]  Hans-Hellmut Nagel,et al.  Incremental recognition of traffic situations from video image sequences , 2000, Image Vis. Comput..

[34]  M Brady,et al.  SPECIAL ISSUE ON SENSOR DATA FUSION - FOREWORD , 1988 .

[35]  Rómer Rosales,et al.  3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[36]  Tieniu Tan,et al.  Recognizing objects on the ground-plane , 1994, Image Vis. Comput..

[37]  Hilary Buxton,et al.  Conceptual descriptions from monitoring and watching image sequences , 2000, Image Vis. Comput..

[38]  Jack Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[39]  R. Collins,et al.  Using a DEM to Determine Geospatial Object Trajectories , 1999 .

[40]  Giuseppe Satalino,et al.  Automatic target recognition for naval traffic control using neural networks , 1998, Image Vis. Comput..

[41]  Bir Bhanu,et al.  Adaptive target recognition , 2000, Machine Vision and Applications.

[42]  Paul K. Davis.  Distributed interactive simulation in the evolution of DoD warfare modeling and simulation , 1995 .

[43]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[44]  Daphna Weinshall,et al.  Motion of disturbances: detection and tracking of multi-body non-rigid motion , 1999, Machine Vision and Applications.

[45]  D PettyM.  Computer generated forces in Distributed Interactive Simulation. , 1995 .

[46]  Ian D. Reid,et al.  The Active Recovery of 3D Motion Trajectories and Their Use in Prediction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[48]  Michael J. Black,et al.  Cardboard people: a parameterized model of articulated image motion , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[49]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  A. Ceranowicz,et al.  ModSAF Development Status , 1995 .

[51]  Takeo Kanade,et al.  Advances in Cooperative Multi-Sensor Video Surveillance , 1999 .

[52]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[53]  Gang Xu,et al.  Tracking Human Body Motion Based on a Stick Figure Model , 1994, J. Vis. Commun. Image Represent..

[54]  Lily Lee,et al.  Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Paolo Remagnino,et al.  Multi-camera colour tracking , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[58]  David P. Casasent,et al.  Classifier and shift-invariant automatic target recognition neural networks , 1995, Neural Networks.

[59]  Hans-Hellmut Nagel,et al.  Tracking Persons in Monocular Image Sequences , 1999, Comput. Vis. Image Underst..

[60]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[62]  Glenn Healey,et al.  Segmenting images using normalized color , 1992, IEEE Trans. Syst. Man Cybern..

[63]  Terrance E. Boult,et al.  Frame-rate omnidirectional surveillance and tracking of camouflaged and occluded targets , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[64]  Takashi Matsuyama,et al.  Multiobject Behavior Recognition by Event Driven Selective Attention Method , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[66]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[67]  Keinosuke Fukunaga,et al.  Statistical Pattern Recognition , 1993, Handbook of Pattern Recognition and Computer Vision.

[68]  Paolo Remagnino,et al.  Multi-Camera Color Tracking , 1999 .