Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management

In this paper we present an effective and flexible real-time video analysis system aiming at a wide range of outdoor surveillance and monitoring scenarios, in which robust detection and tracking objects of interest (pedestrians and/or vehicles) are essential to detecting events and understanding scene changes. The approaches employed address properly the challenges found in typical outdoor scenes such as localised and global lighting changes, variations in object size and views, occlusions, and complex object motion and so on. The contributions include use of an effective two-layer shadow removal scheme based on brightness distortion in different colour channels, blob-based object tracking and occlusion handling via multi-state transition management. The system works equally well on highly compressed (JPEG + MPEG-4) video streams as well as raw video data; in the former case, it achieves about 18 frames per second with software MPEG-4 decoder in a modern PC. And the real-time constraint renders most complex tracking algorithms less attractive.

[1]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[2]  Chris Stauffer,et al.  Estimating Tracking Sources and Sinks , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[3]  Li-Qun Xu,et al.  A hybrid blob- and appearance-based framework for multi-object tracking through complex occlusions , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[4]  Rita Cucchiara,et al.  Probabilistic people tracking for occlusion handling , 2004, ICPR 2004.

[5]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Richard Bowden,et al.  A real time adaptive visual surveillance system for tracking low-resolution colour targets in dynamically changing scenes , 2003, Image Vis. Comput..

[7]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[9]  L-Q Xu,et al.  Segmentation and Tracking of Multiple Moving Objects for Intelligent Video Analysis , 2004 .

[10]  Ramakant Nevatia,et al.  Tracking multiple humans in complex situations , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Quming Zhou,et al.  Tracking and Classifying Moving Objects from Video , 2001 .

[12]  B. Lei,et al.  From pixels to objects and trajectories: a generic real-time outdoor video surveillance system , 2005 .

[13]  L. Davis,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002, Proc. IEEE.

[14]  David S. Doermann,et al.  An appearance-based approach for consistent labeling of humans and objects in video , 2004, Pattern Analysis and Applications.

[15]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[16]  Azriel Rosenfeld,et al.  Tracking Groups of People , 2000, Comput. Vis. Image Underst..

[17]  Badrinath Roysam,et al.  Image change detection algorithms: a systematic survey , 2005, IEEE Transactions on Image Processing.

[18]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Tim J. Ellis,et al.  Partial Observation vs. Blind Tracking through Occlusion , 2002, BMVC.

[20]  Montse Pardàs,et al.  Shadow removal with blob-based morphological reconstruction for error correction , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[21]  Michael Harville,et al.  Foreground segmentation using adaptive mixture models in color and depth , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[22]  Mubarak Shah,et al.  Tracking and Object Classification for Automated Surveillance , 2002, ECCV.

[23]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Touradj Ebrahimi,et al.  Detecting shadows in image sequences , 2004 .