Directional Multi-Scale Stationary Wavelet-Based Representation for Human Action Classification

Human action recognition is a very active field in computer vision. Many important applications depend on accurate human action recognition, which is based on accurate representation of the actions. These applications include surveillance, athletic performance analysis, driver assistance, robotics, and human-centered computing. This chapter presents a thorough review of the field, concentrating the recent action representation methods that use spatio-temporal information. In addition, the authors propose a stationary wavelet-based representation of natural human actions in realistic videos. The proposed representation utilizes the 3D Stationary Wavelet Transform to encode the directional multi-scale spatiotemporal characteristics of the motion available in a frame sequence. It was tested using the Weizmann, and KTH datasets, and produced good preliminary results while having reasonable computational complexity when compared to existing state–of–the–art methods.

[1]  Alaa Tharwat,et al.  Principal component analysis - a tutorial , 2016, Int. J. Appl. Pattern Recognit..

[2]  Alex Pentland,et al.  Human Computing and Machine Understanding of Human Behavior: A Survey , 2007, Artifical Intelligence for Human Computing.

[3]  Mubarak Shah,et al.  A 3-dimensional sift descriptor and its application to action recognition , 2007, ACM Multimedia.

[4]  A F Bobick,et al.  Movement, activity and action: the role of knowledge in the perception of motion. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[5]  Shlomo Zilberstein,et al.  How Robots Can Recognize Activities and Plans Using Topic Models , 2014, AAAI 2014.

[6]  Siddharth Swarup Rautaray,et al.  Real Time Multiple Hand Gesture Recognition System for Human Computer Interaction , 2012 .

[7]  Tianzhu Zhang,et al.  Human Action Recognition in Videos Using Hybrid Motion Features , 2010, MMM.

[8]  Jin Young Choi,et al.  Intelligent visual surveillance — A survey , 2010 .

[9]  Young-Koo Lee,et al.  A Unified Framework for Activity Recognition-Based Behavior Analysis and Action Prediction in Smart Homes , 2013, Sensors.

[10]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[11]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[12]  Anupam Agrawal,et al.  A survey on activity recognition and behavior understanding in video surveillance , 2012, The Visual Computer.

[13]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Mohamed F. Tolba,et al.  Spatio-Temporal Motion Detection for Intelligent Surveillance Applications , 2015 .

[15]  Martin D. Levine,et al.  An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions , 2013, Comput. Vis. Image Underst..

[16]  Oliver Brdiczka,et al.  Detecting Human Behavior Models From Multimodal Observation in a Smart Home , 2009, IEEE Transactions on Automation Science and Engineering.

[17]  Luc Van Gool,et al.  Action snippets: How many frames does human action recognition require? , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jake K. Aggarwal,et al.  Human motion analysis: a review , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[19]  Ickjai Lee,et al.  Expert Systems With Applications , 2013 .

[20]  Li Wang,et al.  Structured learning of local features for human action classification and localization , 2012, Image Vis. Comput..

[21]  Alberto Del Bimbo,et al.  Recognizing human actions by fusing spatio-temporal appearance and motion descriptors , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[22]  Hassan Foroosh,et al.  View invariant action recognition using projective depth , 2014, Comput. Vis. Image Underst..

[23]  Alex Pentland,et al.  Human-Centred Intelligent Human-Computer Interaction (HCI2): how far are we from attaining it? , 2008, Int. J. Auton. Adapt. Commun. Syst..

[24]  Alexander G. Hauptmann,et al.  MoSIFT: Recognizing Human Actions in Surveillance Videos , 2009 .

[25]  Zhiquan Wang,et al.  Recognition of human activities using SVM multi-class classifier , 2010, Pattern Recognit. Lett..

[26]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[27]  Qi Tian,et al.  Recognizing human group action by layered model with multiple cues , 2014, Neurocomputing.

[28]  Yannis Avrithis,et al.  Spatiotemporal saliency for event detection and representation in the 3D wavelet domain: potential in human action recognition , 2007, CIVR '07.

[29]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Tadashi Shibata,et al.  A hardware friendly algorithm for action recognition using spatio-temporal motion-field patches , 2013, Neurocomputing.

[31]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Md. Atiqur Rahman Ahad,et al.  Motion history image: its variants and applications , 2012, Machine Vision and Applications.

[33]  Alessio Del Bue,et al.  Human behavior analysis in video surveillance: A Social Signal Processing perspective , 2013, Neurocomputing.

[34]  Noel E. O'Connor,et al.  An Evaluation of Local Action Descriptors for Human Action Classification in the Presence of Occlusion , 2014, MMM.

[35]  Maurice Milgram,et al.  Recognition of human behavior by space-time silhouette characterization , 2008, Pattern Recognit. Lett..

[36]  Miguel A. Patricio,et al.  A probabilistic, discriminative and distributed system for the recognition of human actions from multiple views , 2012, Neurocomputing.

[37]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[38]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[39]  Ling Shao,et al.  Retrieving Human Actions Using Spatio-Temporal Features and Relevance Feedback , 2010 .

[40]  Changsheng Xu,et al.  Boosted multi-class semi-supervised learning for human action recognition , 2011, Pattern Recognit..

[41]  R. Venkatesh Babu,et al.  Recognition of human actions using motion history information extracted from the compressed video , 2004, Image Vis. Comput..

[42]  Shaogang Gong,et al.  Fusing appearance and distribution information of interest points for action recognition , 2012, Pattern Recognit..

[43]  R. Venkatesh Babu,et al.  Human action recognition using a fast learning fully complex-valued classifier , 2012, Neurocomputing.

[44]  Ling Shao,et al.  Human action segmentation and recognition via motion and shape analysis , 2012, Pattern Recognit. Lett..

[45]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[46]  Ronen Basri,et al.  Actions as Space-Time Shapes , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Chabane Djeraba,et al.  Crowd Behavior Surveillance Using Bhattacharyya Distance Metric , 2010, CompIMAGE.

[48]  Arun Sharma,et al.  Wavelet directional histograms for classification of human gestures represented by spatio-temporal templates , 2004, 10th International Multimedia Modelling Conference, 2004. Proceedings..

[49]  James W. Davis,et al.  The representation and recognition of human movement using temporal templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  Duan-Yu Chen,et al.  Motion-based unusual event detection in human crowds , 2011, J. Vis. Commun. Image Represent..

[51]  Rémi Ronfard,et al.  Free viewpoint action recognition using motion history volumes , 2006, Comput. Vis. Image Underst..

[52]  Tieniu Tan,et al.  Vs-star: A visual interpretation system for visual surveillance , 2010, Pattern Recognit. Lett..

[53]  Shiv Ram Dubey,et al.  Human Activity Recognition Using Gait Pattern , 2013, Int. J. Comput. Vis. Image Process..

[54]  M.M. Trivedi,et al.  Visual Modules for Head Gesture Analysis in Intelligent Vehicle Systems , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[55]  Magda B. Fayek,et al.  An enhanced method for human action recognition , 2015, Journal of advanced research.

[56]  Xiaoqin Zhang,et al.  Adaptive learning codebook for action recognition , 2011, Pattern Recognit. Lett..

[57]  Ramakant Nevatia,et al.  3D Human Action Recognition Using Spatio-temporal Motion Templates , 2005, ICCV-HCI.

[58]  Yannis Avrithis,et al.  Spatiotemporal saliency for video classification , 2009, Signal Process. Image Commun..

[59]  Arun Sharma,et al.  Moments and Wavelets for Classification of Human Gestures Represented by Spatio-Temporal Templates , 2004, Australian Conference on Artificial Intelligence.

[60]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, ICPR 2004.

[61]  Jake K. Aggarwal,et al.  Human activity recognition from 3D data: A review , 2014, Pattern Recognit. Lett..