Human behavior understanding for video surveillance: Recent advance

With the wide applications of video cameras in surveillance, video analysis technologies have attracted the attention from the researchers in computer vision field. In video analysis, human behavior recognition and understanding is an important research direction. By recognition and understanding the human behaviors, we can predict and recognize the happening of crimes and help to the police or other agencies to react immediately. In the past, large amount of intensive papers have been published on human behavior understanding in videos. Generally speaking, the procedure of human behavior understanding can be divided into the following stages: human segmentation and tracking, and human behavior recognition. In this paper, we provide a comprehensive survey of the recent development of all these stages. We will also discuss the difficulties in behavior understanding and identify possible future directions.

[1]  Christopher Nw Agboso.  User focused Surveillance Systems Integration for Intelligent Transport Systems , 1999 .

[2]  Tieniu Tan,et al.  Fusion of static and dynamic body biometrics for gait recognition , 2004, IEEE Trans. Circuits Syst. Video Technol..

[3]  Robert B. Fisher,et al.  Two Approaches to a Plug-and-Play Vision Architecture – CAVIAR and Psyclone , 2005 .

[4]  Joachim Denzler,et al.  Model based extraction of articulated objects in image sequences for gait analysis , 1997, Proceedings of International Conference on Image Processing.

[5]  Naveen Vignesh Ramaraj.  Location Based Activity Recognition Using Mobile Phones , 2009 .

[6]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  Greg Mori,et al.  IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL., NO. 1 Human Action Recognition by Semi-Latent Topic Models , 2022 .

[8]  M. Shah,et al.  Exploring the Space of an Action for Human Action Recognition , 2005 .

[9]  Hyeran Byun,et al.  Human tracking and silhouette extraction for human–robot interaction systems , 2008, Pattern Analysis and Applications.

[10]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Shyamsundar Rajaram,et al.  Human Activity Recognition Using Multidimensional Indexing , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Hyung Lee-Kwang,et al.  Modeling and recognition of hand gesture using colored Petri nets , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[13]  Tieniu Tan,et al.  Model-Based Localisation and Recognition of Road Vehicles , 1998, International Journal of Computer Vision.

[14]  Henry A. Kautz.  A formal theory of plan recognition , 1987 .

[15]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[16]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Ramesh C. Jain,et al.  On the Analysis of Accumulative Difference Pictures from Image Sequences of Real World Scenes , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Yan Meng,et al.  Abnormal Behavior Recognition Using Self-Adaptive Hidden Markov Models , 2009, ICIAR.

[19]  Ian D. Reid,et al.  A general method for human activity recognition in video , 2006, Comput. Vis. Image Underst..

[20]  I-Cheng Chang,et al.  Ribbon-based motion analysis of human body movements , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[21]  Martial Hebert,et al.  Efficient visual event detection using volumetric features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[22]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[23]  Hironobu Fujiyoshi,et al.  A System for Video Surveillance and Monitoring CMU VSAM Final Report , 1999 .

[24]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[25]  James J. Little,et al.  Tracking and recognizing actions of multiple hockey players using the boosted particle filter , 2009, Image Vis. Comput..

[26]  Shaogang Gong,et al.  Surveillance video behaviour profiling and anomaly detection , 2009, Security + Defence.

[27]  Stuart J. Russell,et al.  Image Segmentation in Video Sequences: A Probabilistic Approach , 1997, UAI.

[28]  Ramakant Nevatia,et al.  Tracking multiple humans in complex situations , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  David J. Fleet,et al.  Temporal motion models for monocular and multiview 3D human body tracking , 2006, Comput. Vis. Image Underst..

[30]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Yang Wang,et al.  Human Action Recognition by Semilatent Topic Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[33]  Shaogang Gong,et al.  Video Behavior Profiling for Anomaly Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Lisa M. Brown,et al.  IBM smart surveillance system (S3): event based video surveillance system with an open and extensible framework , 2008, Machine Vision and Applications.

[35]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[36]  Bob Fisher,et al.  Recognition of coordinated multi agent activities, the individual vs the group , 2006 .

[37]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[38]  Mubarak Shah,et al.  Recognizing human actions in videos acquired by uncalibrated moving cameras , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[39]  Natan Peterfreund,et al.  Robust Tracking of Position and Velocity With Kalman Snakes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Honghai Liu,et al.  Advances in View-Invariant Human Motion Analysis: A Review , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[41]  Sidharth Bhatia,et al.  Tracking loose-limbed people , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[42]  Ankush Mittal,et al.  Study of Robust and Intelligent Surveillance in Visible and Multi-modal Framework , 2007, Informatica.

[43]  Ramakant Nevatia,et al.  Segmentation and Tracking of Multiple Humans in Crowded Environments , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Leslie Pack Kaelbling,et al.  Activity Recognition from Physiological Data using Conditional Random Fields , 2006 .

[45]  Aaron F. Bobick,et al.  A state-based technique for the summarization and recognition of gesture , 1995, Proceedings of IEEE International Conference on Computer Vision.

[46]  Qiang Yang,et al.  Spatio-temporal event detection using dynamic conditional random fields , 2009, IJCAI 2009.

[47]  S. Khalid,et al.  Classifying spatiotemporal object trajectories using unsupervised learning of basis function coefficients , 2005, VSSN@MM.

[48]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[50]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[51]  Carlo S. Regazzoni,et al.  Introduction to the special issue on video object processing for surveillance applications , 2005, Real Time Imaging.

[52]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.