A comprehensive review on intelligent surveillance systems

Intelligent surveillance system (ISS) has received growing attention due to the increasing demand on security and safety. ISS is able to automatically analyze image, video, audio or other type of surveillance data without or with limited human intervention. The recent developments in sensor devices, computer vision, and machine learning have an important role in enabling such intelligent system. This paper aims to provide general overview of intelligent surveillance system and discuss some possible sensor modalities and their fusion scenarios such as visible camera (CCTV), infrared camera, thermal camera and radar. This paper also discusses main processing steps in ISS: background-foreground segmentation, object detection and classification, tracking, and behavioral analysis.

[1]  Gerhard Rigoll,et al.  Background segmentation with feedback: The Pixel-Based Adaptive Segmenter , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[2]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Kai Oliver Arras,et al.  Tracking people in 3D using a bottom-up top-down detector , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[5]  Jake K. Aggarwal,et al.  Tracking Human Motion in Structured Environments Using a Distributed-Camera System , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Sergio A. Velastin,et al.  Intelligent distributed surveillance systems: a review , 2005 .

[7]  Dmitry B. Goldgof,et al.  How effective is human video surveillance performance? , 2008, 2008 19th International Conference on Pattern Recognition.

[8]  John A. Antoniades,et al.  Autonomous real-time ground ubiquitous surveillance-imaging system (ARGUS-IS) , 2008, SPIE Defense + Commercial Sensing.

[9]  Michael Felsberg,et al.  Accurate Scale Estimation for Robust Visual Tracking , 2014, BMVC.

[10]  Fakhri Karray,et al.  Multisensor data fusion: A review of the state-of-the-art , 2013, Inf. Fusion.

[11]  James Llinas,et al.  An introduction to multisensor data fusion , 1997, Proc. IEEE.

[12]  Jean Meunier,et al.  Robust Video Surveillance for Fall Detection Based on Human Shape Deformation , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[14]  Soraia Raupp Musse,et al.  Crowd Analysis Using Computer Vision Techniques [ A survey ] , 2010 .

[15]  Ramachandran Baskaran,et al.  Automated human behavior analysis from surveillance videos: a survey , 2014, Artificial Intelligence Review.

[16]  Jean-Marc Odobez,et al.  Multi-Layer Background Subtraction Based on Color and Texture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Sridha Sridharan,et al.  Real-time adaptive background segmentation , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[18]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[19]  Vittorio Murino,et al.  Background Subtraction for Automated Multisensor Surveillance: A Comprehensive Review , 2010, EURASIP J. Adv. Signal Process..

[20]  Eduardo F. Nakamura,et al.  Information fusion for wireless sensor networks: Methods, models, and classifications , 2007, CSUR.

[21]  Vassilios Morellas,et al.  DETER: Detection of events for threat evaluation and recognition , 2003, Machine Vision and Applications.

[22]  Stefano Messelodi,et al.  A Kalman Filter Based Background Updating Algorithm Robust to Sharp Illumination Changes , 2005, ICIAP.

[23]  Manoranjan Paul,et al.  Human detection in surveillance videos and its applications - a review , 2013, EURASIP J. Adv. Signal Process..

[24]  Cristiano Premebida,et al.  Pedestrian detection combining RGB and dense LIDAR data , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[26]  Antoine Vacavant,et al.  A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos , 2014, Comput. Vis. Image Underst..

[27]  Xiaogang Wang,et al.  Intelligent multi-camera video surveillance: A review , 2013, Pattern Recognit. Lett..

[28]  Pietro Perona,et al.  The Fastest Pedestrian Detector in the West , 2010, BMVC.

[29]  Haroon Idrees,et al.  Tracking in dense crowds using prominence and neighborhood motion concurrence , 2014, Image Vis. Comput..

[30]  Bing-Fei Wu,et al.  A Real-Time Vision System for Nighttime Vehicle Detection and Traffic Surveillance , 2011, IEEE Transactions on Industrial Electronics.

[31]  Thierry Bouwmans,et al.  Type-2 Fuzzy Mixture of Gaussians Model: Application to Background Modeling , 2008, ISVC.

[32]  Bernt Schiele,et al.  New features and insights for pedestrian detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Mark E Hallenbeck,et al.  Extracting Roadway Background Image , 2006 .

[34]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[35]  Roland Siegwart,et al.  A Layered Approach to People Detection in 3D Range Data , 2010, AAAI.

[36]  Wanqing Li,et al.  Action recognition based on a bag of 3D points , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[37]  Jake K. Aggarwal,et al.  Semantic-level Understanding of Human Actions and Interactions using Event Hierarchy , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[38]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Thierry Chateau,et al.  A Benchmark Dataset for Foreground/Background Extraction , 2012, ACCV 2012.

[40]  Stephen J. Maybank,et al.  The ADVISOR Visual Surveillance System , 2004 .

[41]  Touradj Ebrahimi,et al.  Tracking video objects in cluttered background , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  S. Liaqat,et al.  Automatic target classifier for a Ground Surveillance Radar using linear discriminant analysis and Logistic regression , 2012, 2012 9th European Radar Conference.

[43]  Bruce A. Draper,et al.  Visual object tracking using adaptive correlation filters , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Bertrand Vachon,et al.  Statistical Background Modeling for Foreground Detection: A Survey , 2010 .

[45]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[46]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Teddy Ko,et al.  A survey on behavior analysis in video surveillance for homeland security applications , 2008, 2008 37th IEEE Applied Imagery Pattern Recognition Workshop.

[48]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[49]  Bastian Leibe,et al.  Real-time multi-person tracking with detector assisted structure propagation , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[50]  Lucia Maddalena,et al.  A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications , 2008, IEEE Transactions on Image Processing.

[51]  Xue Wang,et al.  Distributed Visual-Target-Surveillance System in Wireless Sensor Networks , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[52]  Radha Poovendran,et al.  Group Event Detection With a Varying Number of Group Members for Video Surveillance , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[53]  Ying-Wen Bai,et al.  Design and implementation of an embedded home surveillance system with ultra-low alert power , 2011, 2011 IEEE International Conference on Consumer Electronics (ICCE).

[54]  Javier Medina,et al.  Intelligent surveillance system with integration of heterogeneous information for intrusion detection , 2011, Expert Syst. Appl..

[55]  Neil M. Robertson,et al.  Contextual person detection in multi-modal outdoor surveillance , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[56]  Yuri Ivanov,et al.  Robust Abandoned Object Detection Using Dual Foregrounds , 2008, EURASIP J. Adv. Signal Process..

[57]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[58]  Nahum Kiryati,et al.  Real-time abnormal motion detection in surveillance video , 2008, 2008 19th International Conference on Pattern Recognition.

[59]  H. Pourreza,et al.  An eigenspace-based approach for human fall detection using Integrated Time Motion Image and multi-class Support Vector Machine , 2008, 2008 4th International Conference on Intelligent Computer Communication and Processing.

[60]  Alessio Del Bue,et al.  Human behavior analysis in video surveillance: A Social Signal Processing perspective , 2013, Neurocomputing.

[61]  P. KaewTrakulPong,et al.  An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection , 2002 .

[62]  François Brémond,et al.  Video-understanding framework for automatic behavior recognition , 2006, Behavior research methods.

[63]  Thomas B. Moeslund,et al.  Thermal cameras and applications: a survey , 2013, Machine Vision and Applications.

[64]  Thierry Bouwmans,et al.  Recent Advanced Statistical Background Modeling for Foreground Detection - A Systematic Survey , 2011 .

[65]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[66]  Shishir K. Shah,et al.  A survey of approaches and trends in person re-identification , 2014, Image Vis. Comput..

[67]  Csaba Benedek,et al.  3D people surveillance on range data sequences of a rotating Lidar , 2014, Pattern Recognit. Lett..

[68]  Borko Furht,et al.  Neural Network Approach to Background Modeling for Video Object Segmentation , 2007, IEEE Transactions on Neural Networks.

[69]  Badrinath Roysam,et al.  Image change detection algorithms: a systematic survey , 2005, IEEE Transactions on Image Processing.

[70]  Benjamin Höferlin,et al.  Evaluation of background subtraction techniques for video surveillance , 2011, CVPR 2011.

[71]  M. Sigari,et al.  Fuzzy Running Average and Fuzzy Background Subtraction: Concepts and Application , 2008 .