A Survey of Automatic Event Detection in Multi-Camera Third Generation Surveillance Systems

Third generation surveillance systems are largely requested for intelligent surveillance of different scenarios such as public areas, urban traffic control, smart homes and so on. They are based on multiple cameras and processing modules that integrate data coming from a large surveillance space. The semantic interpretation of data from a multi-view context is a challenging task and requires the development of image processing methodologies that could support applications in extensive and real-time contexts. This paper presents a survey of automatic event detection functionalities that have been developed for third generation surveillance systems with a particular emphasis on open problems that limit the application of computer vision methodologies to commercial multi-camera systems.

[1]  Tamás Szirányi,et al.  Localizing people in multi-view environment using height map reconstruction in real-time , 2013, Pattern Recognit. Lett..

[2]  Rita Cucchiara,et al.  Event driven software architecture for multi-camera and distributed surveillance research systems , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[3]  Jia-Shung Wang,et al.  A crowd-filter for detection of abandoned objects in crowded area , 2008, 2008 3rd International Conference on Sensing Technology.

[4]  Yael Moses,et al.  Tracking in a Dense Crowd Using Multiple Cameras , 2010, International Journal of Computer Vision.

[5]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Mark S. Nixon,et al.  Performance analysis for automated gait extraction and recognition in multi-camera surveillance , 2010, Multimedia Tools and Applications.

[7]  Keiichi Uchimura,et al.  Vehicle Detection Using Probability Fusion Maps Generated by Multi-camera Systems , 2009, J. Inf. Process..

[8]  Aljoscha Smolic,et al.  3-D reconstruction of a dynamic environment with a fully calibrated background for traffic scenes , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Robert Pless,et al.  An efficient system for vehicle tracking in multi-camera networks , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[10]  Thomas Wiegand,et al.  MULTIPLE VIEW VIDEO STREAMING AND 3D SCENE RECONSTRUCTION FOR TRAFFIC SURVEILLANCE , 2003 .

[11]  Tim J. Ellis,et al.  Multi camera image tracking , 2006, Image Vis. Comput..

[12]  B. S. Manjunath,et al.  A lightweight multiview tracked person descriptor for camera sensor networks , 2008, 2008 15th IEEE International Conference on Image Processing.

[13]  Huiyu Zhou,et al.  Human Localization in a Cluttered Space Using Multiple Cameras , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[14]  Mohiuddin Ahmad,et al.  Human action recognition using shape and CLG-motion flow from multi-view image sequences , 2008, Pattern Recognit..

[15]  Tiziana D'Orazio,et al.  A Semi-automatic System for Ground Truth Generation of Soccer Video Sequences , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[16]  Joonki Paik,et al.  Auto calibration for multiple camera-based surveillance system , 2005, Visual Communications and Image Processing.

[17]  Andreas Girgensohn,et al.  Geometric Tools for Multicamera Surveillance Systems , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[18]  Hamid K. Aghajan,et al.  Multi-View ML Object Tracking With Online Learning on Riemannian Manifolds by Combining Geometric Constraints , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[19]  Mohan M. Trivedi,et al.  Human Pose Estimation and Activity Recognition From Multi-View Videos: Comparative Explorations of Recent Developments , 2012, IEEE Journal of Selected Topics in Signal Processing.

[20]  Jean Ponce,et al.  Detecting Abandoned Objects With a Moving Camera , 2010, IEEE Transactions on Image Processing.

[21]  Ramin Zabih,et al.  Bayesian multi-camera surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[22]  R. Chellappa,et al.  Optimal Multi-View Fusion of Object Locations , 2008, 2008 IEEE Workshop on Motion and video Computing.

[23]  Xue Wang,et al.  Distributed Visual-Target-Surveillance System in Wireless Sensor Networks , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Sergio A. Velastin,et al.  How close are we to solving the problem of automated visual surveillance? , 2008, Machine Vision and Applications.

[25]  Michael Arens,et al.  Person re-identification in multi-camera networks , 2011, CVPR 2011 WORKSHOPS.

[26]  Ajay Divakaran,et al.  Multi-camera calibration, object tracking and query generation , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[27]  Amit K. Roy-Chowdhury,et al.  Tracking and Activity Recognition Through Consensus in Distributed Camera Networks , 2010, IEEE Transactions on Image Processing.

[28]  Alberto Del Bimbo,et al.  Exploiting distinctive visual landmark maps in pan-tilt-zoom camera networks , 2010, Comput. Vis. Image Underst..

[29]  Yoo-Joo Choi,et al.  Learning Spatio-Temporal Topology of a Multi-Camera Network by Tracking Multiple People , 2007 .

[30]  Nicolas Thome,et al.  A Real-Time, Multiview Fall Detection System: A LHMM-Based Approach , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Tiziana D'Orazio,et al.  An Investigation Into the Feasibility of Real-Time Soccer Offside Detection From a Multiple Camera System , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Pierre Vandergheynst,et al.  Cascade of descriptors to detect and track objects across any network of cameras , 2010, Comput. Vis. Image Underst..

[33]  Tiziana D'Orazio,et al.  Color Brightness Transfer Function evaluation for non overlapping multi camera tracking , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[34]  Sridha Sridharan,et al.  Multi-view Intelligent Vehicle Surveillance System , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[35]  Andrew Gilbert,et al.  Tracking Objects Across Cameras by Incrementally Learning Inter-camera Colour Calibration and Patterns of Activity , 2006, ECCV.

[36]  Alessandro Bevilacqua,et al.  Real time detection of stopped vehicles in traffic scenes , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[37]  Tiziana D'Orazio,et al.  People re-identification and tracking from multiple cameras: A review , 2012, 2012 19th IEEE International Conference on Image Processing.

[38]  G. Rigoll,et al.  Multi camera person tracking applying a graph-cuts based foreground segmentation in a homography framework , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[39]  Justin K. Romberg,et al.  Multi-camera tracking on a graph using Markov chain Monte Carlo , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[40]  Rita Cucchiara,et al.  Video Surveillance Online Repository (ViSOR): an integrated framework , 2010, Multimedia Tools and Applications.

[41]  Ioannis Pitas,et al.  View indepedent human movement recognition from multi-view video exploiting a circular invariant posture representation , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[42]  Alessandro Perina,et al.  Multiple-Shot Person Re-identification by HPE Signature , 2010, 2010 20th International Conference on Pattern Recognition.

[43]  Mubarak Shah,et al.  Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views , 2008, Comput. Vis. Image Underst..

[44]  B. S. Manjunath,et al.  Multicamera video summarization and anomaly detection from activity motifs , 2014, TOSN.

[45]  Tiziana D'Orazio,et al.  A system to automatically monitor forbidden areas , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[46]  A. Amato,et al.  Solving the multi object occlusion problem in a multiple camera tracking system , 2009, Pattern Recognition and Image Analysis.

[47]  Bogdan Kwolek,et al.  Multi Camera-Based Person Tracking Using Region Covariance and Homography Constraint , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[48]  L. Marcenaro,et al.  Dual camera intelligent sensor for high definition 360 degrees surveillance , 2005 .

[49]  Andrea Cavallaro,et al.  Person re-identification in crowd , 2012, Pattern Recognit. Lett..

[50]  Jake K. Aggarwal,et al.  Human activity recognition from 3D data: A review , 2014, Pattern Recognit. Lett..

[51]  Bingbing Ni,et al.  3D surveillance system using multiple cameras , 2007, Electronic Imaging.

[52]  Richard I. Hartley,et al.  Person Reidentification Using Spatiotemporal Appearance , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[53]  Sergio A. Velastin,et al.  Intelligent distributed surveillance systems: a review , 2005 .

[54]  Amit K. Roy-Chowdhury,et al.  Robust Tracking in A Camera Network: A Multi-Objective Optimization Framework , 2008, IEEE Journal of Selected Topics in Signal Processing.

[55]  Tiziana D'Orazio,et al.  Object Tracking by Non-overlapping Distributed Camera Network , 2009, ACIVS.

[56]  Jian Yao,et al.  Multi-Camera Multi-Person 3D Space Tracking with MCMC in Surveillance Scenarios , 2008, ECCV 2008.

[57]  Fabien Moutarde,et al.  Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[58]  Mubarak Shah,et al.  A Multiview Approach to Tracking People in Crowded Scenes Using a Planar Homography Constraint , 2006, ECCV.

[59]  W. Eric L. Grimson,et al.  Correspondence-Free Activity Analysis and Scene Modeling in Multiple Camera Views , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  P. L. Venetianer,et al.  The evolution of video surveillance: an overview , 2008, Machine Vision and Applications.

[61]  Aggelos K. Katsaggelos,et al.  Wireless Video Surveillance: A Survey , 2013, IEEE Access.

[62]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[63]  John Hannah,et al.  IEEE International Conference on Image Processing (ICIP) , 1997 .

[64]  Nikolaos G. Bourbakis,et al.  Surveillance Systems for Smart Homes: A Comparative Survey , 2009, 2009 21st IEEE International Conference on Tools with Artificial Intelligence.

[65]  Nick Roussopoulos,et al.  A graph-based approach to vehicle tracking in traffic camera video streams , 2007, DMSN '07.

[66]  Carlos Hitoshi Morimoto,et al.  Multiple camera people detection and tracking using support integration , 2011, Pattern Recognit. Lett..

[67]  Hairong Qi,et al.  Feature Extraction and Representation for Distributed Multi-View Human Action Recognition , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[68]  Paolo Remagnino,et al.  A multi-agent framework for visual surveillance , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[69]  Hassan Foroosh,et al.  View invariant action recognition using projective depth , 2014, Comput. Vis. Image Underst..

[70]  FuaPascal,et al.  Multicamera People Tracking with a Probabilistic Occupancy Map , 2008 .

[71]  Wu-chi Feng,et al.  3-D Target-Based Distributed Smart Camera Network Localization , 2010, IEEE Transactions on Image Processing.

[72]  Luc Van Gool,et al.  A distributed camera system for multi-resolution surveillance , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[73]  James J. Little,et al.  View-Invariant Discriminative Projection for Multi-View Gait-Based Human Identification , 2013, IEEE Transactions on Information Forensics and Security.

[74]  Raúl Mohedano,et al.  Robust multi-camera 3D tracking from mono-camera 2d tracking using Bayesian Association , 2010, IEEE Transactions on Consumer Electronics.

[75]  C.H.M. Donold,et al.  Assessing the human vigilance capacity of control room operators , 1999 .

[76]  Shaogang Gong,et al.  International Journal of Computer Vision (The original publication is available at www.springerlink.com) Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding , 2009 .

[77]  Daw-Tung Lin,et al.  Collaborative pedestrian tracking with multiple cameras: Data fusion and visualization , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[78]  Mark S. Nixon,et al.  Self-Calibrating View-Invariant Gait Biometrics , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[79]  Louahdi Khoudour,et al.  People re-identification by spectral classification of silhouettes , 2010, Signal Process..

[80]  Muhammad Owais Mehmood,et al.  Multi-camera Based Human Tracking with Non-overlapping Fields of View , 2009, 2009 Fifth International Conference on Image and Graphics.

[81]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[82]  Gian Luca Foresti,et al.  Special issue on video communications, processing, and understanding for third generation surveillance systems , 2001 .

[83]  Slawomir Bak,et al.  Person Re-identification Using Haar-based and DCD-based Signature , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[84]  Nikolaos Papanikolopoulos,et al.  Multi-view 3D vehicle tracking with a constrained filter , 2008, 2008 IEEE International Conference on Robotics and Automation.

[85]  Xing Zhang,et al.  Fusing target information from multiple views for robust visual tracking , 2014, IET Comput. Vis..

[86]  Tomi Räty,et al.  Survey on Contemporary Remote Surveillance Systems for Public Safety , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[87]  Harpreet S. Sawhney,et al.  Real-time wide area multi-camera stereo tracking , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[88]  Jian-Huang Lai,et al.  Spatial-temporal consistent labeling of tracked pedestrians across non-overlapping camera views , 2011, Pattern Recognit..

[89]  Arasanathan Anjulan,et al.  Relating "Pace' to Activity Changes in Mono- and Multi-camera Surveillance Videos , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[90]  Aidong Men,et al.  Traffic Surveillance System for Vehicle Flow Detection , 2010, 2010 Second International Conference on Computer Modeling and Simulation.

[91]  Yi-Ping Hung,et al.  An adaptive learning method for target tracking across multiple cameras , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[92]  Kaoru Hirota,et al.  Detection algorithm for color dynamic images by multiple surveillance cameras under low luminance conditions based on fuzzy corresponding map , 2008, Appl. Soft Comput..

[93]  Shaogang Gong,et al.  Detecting and discriminating behavioural anomalies , 2011, Pattern Recognit..

[94]  Mubarak Shah,et al.  Tracking Multiple Occluding People by Localizing on Multiple Scene Planes , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[95]  Marc Pollefeys,et al.  Multi-view Occlusion Reasoning for Probabilistic Silhouette-Based Dynamic Scene Reconstruction , 2010, International Journal of Computer Vision.

[96]  Shahram Payandeh,et al.  Geometry-Based Object Association and Consistent Labeling in Multi-Camera Surveillance , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[97]  Venkatesh Saligrama,et al.  Activity Based Matching in Distributed Camera Networks , 2010, IEEE Transactions on Image Processing.

[98]  Nader Karimi,et al.  View-Invariant Fall Detection System Based on Silhouette Area and Orientation , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[99]  Matti Pietikäinen,et al.  Matching Groups of People by Covariance Descriptor , 2010, 2010 20th International Conference on Pattern Recognition.

[100]  Touradj Ebrahimi,et al.  Multi-view video segmentation and tracking for video surveillance , 2009, Defense + Commercial Sensing.