A Review of Anomaly Detection in Automated Surveillance

As surveillance becomes ubiquitous, the amount of data to be processed grows along with the demand for manpower to interpret the data. A key goal of surveillance is to detect behaviors that can be considered anomalous. As a result, an extensive body of research in automated surveillance has been developed, often with the goal of automatic detection of anomalies. Research into anomaly detection in automated surveillance covers a wide range of domains, employing a vast array of techniques. This review presents an overview of recent research approaches on the topic of anomaly detection in automated surveillance. The reviewed studies are analyzed across five aspects: surveillance target, anomaly definitions and assumptions, types of sensors used and the feature extraction processes, learning methods, and modeling algorithms.

[1]  Shaogang Gong,et al.  Incremental and adaptive abnormal behaviour detection , 2008, Comput. Vis. Image Underst..

[2]  Yangsheng Xu,et al.  Abnormal crowd motion analysis , 2009, 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[3]  Shaogang Gong,et al.  Video Behavior Profiling for Anomaly Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Qiang Yang,et al.  Sensor-Based Abnormal Human-Activity Detection , 2008, IEEE Transactions on Knowledge and Data Engineering.

[5]  Marc Ph. Stoecklin Anomaly detection by finding feature distribution outliers , 2006, CoNEXT '06.

[6]  Hoh Peter In,et al.  Situation aware RFID system: evaluating abnormal behavior detecting approach , 2006, The Fourth IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems, and the Second International Workshop on Collaborative Computing, Integration, and Assurance (SEUS-WCCIA'06).

[7]  Youfu Wu,et al.  Anomalous Event Detection Based on Self-Organizing Map for Supermarket Monitoring , 2009, 2009 International Conference on Information Engineering and Computer Science.

[8]  Jun Zhang,et al.  Detecting Pedestrian Abnormal Behavior Based on Fuzzy Associative Memory , 2008, 2008 Fourth International Conference on Natural Computation.

[9]  Chloé Clavel,et al.  Detection and Analysis of Abnormal Situations Through Fear-Type Acoustic Manifestations , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[10]  Shaogang Gong,et al.  Detecting and discriminating behavioural anomalies , 2011, Pattern Recognit..

[11]  Zhanyi Hu,et al.  Pointwise Motion Image (PMI): A Novel Motion Representation and Its Applications to Abnormality Detection and Behavior Recognition , 2009, IEEE Trans. Circuits Syst. Video Technol..

[12]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Ashish Mishra,et al.  Detecting border intrusion using wireless sensor network and artificial neural network , 2010, 2010 6th IEEE International Conference on Distributed Computing in Sensor Systems Workshops (DCOSSW).

[14]  Kian-Ming Lim,et al.  Statistical and entropy based multi purpose human motion analysis , 2010, 2010 2nd International Conference on Signal Processing Systems.

[15]  Wei-bang Chen,et al.  A Multiple Instance Learning and Relevance Feedback Framework for Retrieving Abnormal Incidents in Surveillance Videos , 2010, J. Multim..

[16]  Deborah Goshorn,et al.  Abnormal behavior-detection using sequential syntactical classification in a network of clustered cameras , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[17]  Paulo Cortez,et al.  The OBSERVER: An Intelligent and Automated Video Surveillance System , 2006, ICIAR.

[18]  Carl E. Rasmussen,et al.  Factorial Hidden Markov Models , 1997 .

[19]  Z. Zenn Bien,et al.  A Nonsupervised Learning Framework of Human Behavior Patterns Based on Sequential Actions , 2010, IEEE Transactions on Knowledge and Data Engineering.

[20]  G. Blelloch Introduction to Data Compression * , 2022 .

[21]  Lawrence Carin,et al.  Infinite Hidden Markov Models for Unusual-Event Detection in Video , 2008, IEEE Transactions on Image Processing.

[22]  Yan Meng,et al.  Abnormal Behavior Recognition Using Self-Adaptive Hidden Markov Models , 2009, ICIAR.

[23]  Liu Zhi Abnormal Behavior of Pedestrian Detection Based on Fuzzy Theory , 2010 .

[24]  S. Gong,et al.  Scene event recognition without tracking , 2003 .

[25]  Hatice Gunes,et al.  Suspicious Behavior Assessment for Visual Surveillance Using Neural Network Classifiers , 2003, CISST.

[26]  Yangsheng Xu,et al.  Crowd surveillance using Markov Random Fields , 2008, 2008 IEEE International Conference on Automation and Logistics.

[27]  Li He-Ping,et al.  Behavior Modeling and Abnormality Detection Based on Semi-Supervised Learning Method , 2007 .

[28]  Peter H. N. de With,et al.  Automatic video-based human motion analyzer for consumer surveillance system , 2009, IEEE Transactions on Consumer Electronics.

[29]  Ziyou Xiong,et al.  Traffic Abnormality Detection through Directional Motion Behavior Map , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[30]  Zhu Han,et al.  Catching Attacker(s) for Collaborative Spectrum Sensing in Cognitive Radio Systems: An Abnormality Detection Approach , 2010, 2010 IEEE Symposium on New Frontiers in Dynamic Spectrum (DySPAN).

[31]  Robert B. Fisher,et al.  Modelling Crowd Scenes for Event Detection , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[32]  Christophe Rosenberger,et al.  Abnormal events detection based on spatio-temporal co-occurences , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Michael Spann,et al.  Event detection for intelligent car park video surveillance , 2005, Real Time Imaging.

[34]  David C. Hogg,et al.  On the feasibility of using a cognitive model to filter surveillance data , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[35]  Hilary Buxton,et al.  Learning and understanding dynamic scene activity: a review , 2003, Image Vis. Comput..

[36]  Sandip S. Patil,et al.  Tracking and identification of suspicious and abnormal behaviors using supervised machine learning technique , 2009, ICAC3 '09.

[37]  P. L. Venetianer,et al.  The evolution of video surveillance: an overview , 2008, Machine Vision and Applications.

[38]  Shaogang Gong,et al.  Video behaviour profiling and abnormality detection without manual labelling , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[39]  Chabane Djeraba,et al.  Crowd Behavior Surveillance Using Bhattacharyya Distance Metric , 2010, CompIMAGE.

[40]  Luis Jiménez,et al.  A cognitive surveillance system for detecting incorrect traffic behaviors , 2009, Expert Syst. Appl..

[41]  Sergio A. Velastin,et al.  PRISMATICA: toward ambient intelligence in public transport environments , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[42]  Alireza Rezvanian,et al.  Robust Fall Detection Using Human Shape and Multi-class Support Vector Machine , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[43]  Jean-Marc Odobez,et al.  Topic models for scene analysis and abnormality detection , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[44]  Diane J. Cook,et al.  Automatic Video Classification: A Survey of the Literature , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[45]  Martin Kampel,et al.  Unexpected Human Behavior Recognition in Image Sequences Using Multiple Features , 2010, 2010 20th International Conference on Pattern Recognition.

[46]  Honghai Liu,et al.  Advances in View-Invariant Human Motion Analysis: A Review , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[47]  Venkatesh Saligrama,et al.  Abnormal behavior detection and behavior matching for networked cameras , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[48]  Shaogang Gong,et al.  On-the-fly global activity prediction and anomaly detection , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[49]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[50]  Mubarak Shah,et al.  Automated Visual Surveillance in Realistic Scenarios , 2007, IEEE MultiMedia.

[51]  Zeng Qing-wei,et al.  Application of Support Vector Regression and Particle Swarm Optimization in Traffic Accident Forecasting , 2009, 2009 International Conference on Information Management, Innovation Management and Industrial Engineering.

[52]  Shunzheng Yu,et al.  Hidden semi-Markov models , 2010, Artif. Intell..

[53]  Svetha Venkatesh,et al.  Explicit State Duration HMM for Abnormality Detection in Sequences of Human Activity , 2004, PRICAI.

[54]  Clara Pizzuti,et al.  Distance-based detection and prediction of outliers , 2006, IEEE Transactions on Knowledge and Data Engineering.

[55]  Zhenjiang Miao,et al.  A Home Environment Posture and Behavior Recognition System , 2007, 2007 International Conference on Convergence Information Technology (ICCIT 2007).

[56]  S. Venkatesh,et al.  Incorporating Contextual Audio for an Actively Anxious Smart Home , 2005, 2005 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[57]  Benjamin Z. Yao,et al.  Learning a scene contextual model for tracking and abnormality detection , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[58]  B.B. Orten,et al.  Event Detection in Automated Surveillance Systems , 2006, 2006 IEEE 14th Signal Processing and Communications Applications.

[59]  Andrea Cavallaro,et al.  Performance evaluation of event detection solutions: the CREDS experience , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[60]  Masaki Yoshida,et al.  Algorithm to detect abnormal states of elderly persons for home monitoring , 2007, Systems and Computers in Japan.

[61]  Yangsheng Xu,et al.  Abnormal Behavior Detection by Multi-SVM-Based Bayesian Network , 2007, 2007 International Conference on Information Acquisition.

[62]  David C. Hogg,et al.  Statistical Models of Object Interaction , 2004, International Journal of Computer Vision.

[63]  Luis Jiménez,et al.  A supervised learning approach to automate the acquisition of knowledge in surveillance systems , 2009, Signal Process..

[64]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Venkatesh Saligrama,et al.  Motion segmentation and abnormal behavior detection via behavior clustering , 2008, 2008 15th IEEE International Conference on Image Processing.

[66]  Chen-Chiung Hsieh,et al.  A Simple and Fast Surveillance System for Human Tracking and Behavior Analysis , 2007, 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System.

[67]  Simone Calderara,et al.  Detection of abnormal behaviors using a mixture of Von Mises distributions , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[68]  Hsi-Lin Chen,et al.  A Hidden Markov Model-based approach for recognizing swimmer's behaviors in swimming pool , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[69]  Venkatesh Saligrama,et al.  Modeling background activity for behavior subtraction , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[70]  Ali A. Ghorbani,et al.  IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS 1 Toward Credible Evaluation of Anomaly-Based Intrusion-Detection Methods , 2022 .

[71]  Andrea Cavallaro,et al.  Event monitoring via local motion abnormality detection in non-linear subspace , 2010, Neurocomputing.

[72]  A. Beghdadi,et al.  Local estimation of displacement density for abnormal behavior detection , 2008, 2008 IEEE Workshop on Machine Learning for Signal Processing.

[73]  Amir-Masoud Eftekhari-Moghadam,et al.  Knowledge Discovery of Traffic/People Behaviors Based on Image Mining Approach , 2006, Geometric Modeling and Imaging--New Trends (GMAI'06).

[74]  Shaogang Gong,et al.  A Markov Clustering Topic Model for mining behaviour in video , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[75]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[76]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[77]  A. Mecocci,et al.  Automatic detection of anomalous behavioural events for advanced real-time video surveillance , 2003, The 3rd International Workshop on Scientific Use of Submarine Cables and Related Technologies, 2003..

[78]  Mubarak Shah,et al.  Abnormal crowd behavior detection using social force model , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[79]  Luis Jiménez,et al.  Intelligent Surveillance Based on Normality Analysis to Detect Abnormal Behaviors , 2009, Int. J. Pattern Recognit. Artif. Intell..

[80]  Teddy Ko,et al.  A survey on behavior analysis in video surveillance for homeland security applications , 2008, 2008 37th IEEE Applied Imagery Pattern Recognition Workshop.

[81]  Shaogang Gong,et al.  Learning Rare Behaviours , 2010, ACCV.

[82]  Andrea Cavallaro,et al.  Trajectory Association and Fusion across Partially Overlapping Cameras , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[83]  Ying Wang,et al.  Abnormal Activity Recognition in Office Based on R Transform , 2007, 2007 IEEE International Conference on Image Processing.

[84]  Vangelis Metsis,et al.  Abnormal human behavioral pattern detection in assisted living environments , 2010, PETRA '10.

[85]  James Black,et al.  Multi view image surveillance and tracking , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[86]  Jie Feng,et al.  Online Learning with Self-Organizing Maps for Anomaly Detection in Crowd Scenes , 2010, 2010 20th International Conference on Pattern Recognition.

[87]  Pau-Choo Chung,et al.  A daily behavior enabled hidden Markov model for human behavior understanding , 2008, Pattern Recognit..

[88]  Özgür Ulusoy,et al.  Keyframe labeling technique for surveillance event classification , 2010 .

[89]  W. Eric L. Grimson,et al.  Correspondence-Free Activity Analysis and Scene Modeling in Multiple Camera Views , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[90]  Paulo Cortez,et al.  Prediction of Abnormal Behaviors for Intelligent Video Surveillance Systems , 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining.

[91]  Jun Zhang,et al.  Detecting abnormal motion of pedestrian in video , 2008, 2008 International Conference on Information and Automation.

[92]  Young-Kuk Kim,et al.  A Hybrid Cache Cohrency Scheme for Ubiquitous Mobile Clients , 2007, 2007 International Conference on Convergence Information Technology (ICCIT 2007).

[93]  Yi-ping Tang,et al.  Intelligent Video Analysis Technology for Elevator Cage Abnormality Detection in Computer Vision , 2009, 2009 Fourth International Conference on Computer Sciences and Convergence Information Technology.

[94]  Joakim Rydell,et al.  Estimation of crowd behavior using sensor networks and sensor fusion , 2009, 2009 12th International Conference on Information Fusion.

[95]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[96]  Özgür Ulusoy,et al.  Scenario-based query processing for video-surveillance archives , 2010, Eng. Appl. Artif. Intell..

[97]  Ehud Rivlin,et al.  Understanding Video Events: A Survey of Methods for Automatic Interpretation of Semantic Occurrences in Video , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[98]  Yangsheng Xu,et al.  Surveillance Robot Utilizing Video and Audio Information , 2009, J. Intell. Robotic Syst..

[99]  Tomi Räty,et al.  Survey on Contemporary Remote Surveillance Systems for Public Safety , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[100]  A. Beghdadi,et al.  Abnormal behavior detection using a multi-modal stochastic learning approach , 2008, 2008 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[101]  Jin Young Choi,et al.  Abnormal Traffic Detection Using Intelligent Driver Model , 2010, 2010 20th International Conference on Pattern Recognition.

[102]  Keiichi Yamada,et al.  Predicting unusual right-turn driving behavior at intersection , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[103]  Andrea Cavallaro,et al.  Local Abnormality Detection in Video Using Subspace Learning , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.