Exploring Techniques for Vision Based Human Activity Recognition: Methods, Systems, and Evaluation

With the wide applications of vision based intelligent systems, image and video analysis technologies have attracted the attention of researchers in the computer vision field. In image and video analysis, human activity recognition is an important research direction. By interpreting and understanding human activities, we can recognize and predict the occurrence of crimes and help the police or other agencies react immediately. In the past, a large number of papers have been published on human activity recognition in video and image sequences. In this paper, we provide a comprehensive survey of the recent development of the techniques, including methods, systems, and quantitative evaluation of the performance of human activity recognition.

[1]  Shaogang Gong,et al.  Video Behavior Profiling for Anomaly Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Lisa M. Brown,et al.  IBM smart surveillance system (S3): event based video surveillance system with an open and extensible framework , 2008, Machine Vision and Applications.

[3]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Jing Zhang,et al.  Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  A. Senior,et al.  Performance Evaluation of Surveillance Systems Under Varying Conditions , 2004 .

[6]  Khalid Choukri,et al.  The CHIL audiovisual corpus for lecture and meeting analysis inside smart rooms , 2007, Lang. Resour. Evaluation.

[7]  Kejun Wang,et al.  Video-Based Abnormal Human Behavior Recognition—A Review , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Rachel Bowers,et al.  PETS vs . VACE Evaluation Programs : A Comparative Study , 2006 .

[9]  Ian D. Reid,et al.  A general method for human activity recognition in video , 2006, Comput. Vis. Image Underst..

[10]  Christopher Nw Agboso.  User focused Surveillance Systems Integration for Intelligent Transport Systems , 1999 .

[11]  Ankush Mittal,et al.  Study of Robust and Intelligent Surveillance in Visible and Multi-modal Framework , 2007, Informatica.

[12]  Shaogang Gong,et al.  Surveillance video behaviour profiling and anomaly detection , 2009, Security + Defence.

[13]  Robert B. Fisher,et al.  Two Approaches to a Plug-and-Play Vision Architecture – CAVIAR and Psyclone , 2005 .

[14]  J. D. Farmer,et al.  State space reconstruction in the presence of noise" Physica D , 1991 .

[15]  Brian C. Lovell,et al.  Kernel analysis over Riemannian manifolds for visual recognition of actions, pedestrians and textures , 2012, 2012 IEEE Workshop on the Applications of Computer Vision (WACV).

[16]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[18]  Wanli Feng,et al.  Exploring techniques for behaviour recognition via the CAVIAR modular vision framework , .

[19]  Svetha Venkatesh,et al.  Efficient duration and hierarchical modeling for human activity recognition , 2009, Artif. Intell..

[20]  Qiang Yang,et al.  Spatio-temporal event detection using dynamic conditional random fields , 2009, IJCAI 2009.

[21]  A. Ellis,et al.  PETS2010 and PETS2009 Evaluation of Results Using Individual Ground Truthed Single Views , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[22]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[23]  Tieniu Tan,et al.  Fusion of static and dynamic body biometrics for gait recognition , 2003, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Alice J. O'Toole,et al.  A video database of moving faces and people , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Leslie Pack Kaelbling,et al.  Activity Recognition from Physiological Data using Conditional Random Fields , 2006 .

[26]  Yui Man Lui,et al.  Advances in matrix manifolds for computer vision , 2012, Image Vis. Comput..

[27]  Carlo S. Regazzoni,et al.  Introduction to the special issue on video object processing for surveillance applications , 2005, Real Time Imaging.

[28]  Mongi A. Abidi,et al.  Optical flow-based real-time object tracking using non-prior training active feature model , 2005, Real Time Imaging.

[29]  S. Khalid,et al.  Classifying spatiotemporal object trajectories using unsupervised learning of basis function coefficients , 2005, VSSN@MM.

[30]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Aaron F. Bobick,et al.  Recognition and interpretation of parametric gesture , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  Dmitry B. Goldgof,et al.  Performance Evaluation of Object Detection and Tracking in Video , 2006, ACCV.

[33]  Tomaso A. Poggio,et al.  Pedestrian detection using wavelet templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Alexander H. Waibel.  CHIL - Computers in the Human Interaction Loop , 2005, MVA.

[35]  Ehud Rivlin,et al.  Understanding Video Events: A Survey of Methods for Automatic Interpretation of Semantic Occurrences in Video , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[36]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[37]  Henry A. Kautz.  A formal theory of plan recognition , 1987 .

[38]  Dieter Fox,et al.  Location-Based Activity Recognition , 2005, KI.

[39]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[40]  Robert T. Collins,et al.  An Open Source Tracking Testbed and Evaluation Web Site , 2005 .

[41]  Shyamsundar Rajaram,et al.  Human Activity Recognition Using Multidimensional Indexing , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  C. Machy,et al.  Performance Evaluation of Frequent Events Detection Systems , 2006 .

[43]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Hironobu Fujiyoshi,et al.  A System for Video Surveillance and Monitoring CMU VSAM Final Report , 1999 .

[45]  James J. Little,et al.  Tracking and recognizing actions of multiple hockey players using the boosted particle filter , 2009, Image Vis. Comput..

[46]  J. Ross Beveridge,et al.  Action classification on product manifolds , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Qiang Yang,et al.  CIGAR: Concurrent and Interleaving Goal and Activity Recognition , 2008 .

[48]  Andrea Cavallaro,et al.  Performance evaluation of event detection solutions: the CREDS experience , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[49]  J.M. Ferryman,et al.  PETS Metrics: On-Line Performance Evaluation Service , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[50]  Manuela M. Veloso,et al.  Conditional random fields for activity recognition , 2007, AAMAS '07.

[51]  François Brémond,et al.  Intelligent Video Systems: A Review of Performance Evaluation Metrics that Use Mapping Procedures , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[52]  Teddy Ko,et al.  A survey on behavior analysis in video surveillance for homeland security applications , 2008, 2008 37th IEEE Applied Imagery Pattern Recognition Workshop.

[53]  François Brémond,et al.  ETISEO, performance evaluation for video surveillance systems , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[54]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Yan Meng,et al.  Abnormal Behavior Recognition Using Self-Adaptive Hidden Markov Models , 2009, ICIAR.

[56]  Robert B. Fisher,et al.  Modelling Crowd Scenes for Event Detection , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[57]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..