Human Activity Recognition Based on Spatial Distribution of Gradients at Sublevels of Average Energy Silhouette Images

The aim of this paper is to present a unified framework for human action and activity recognition by analysing the effect of computation of spatial distribution of gradients (SDGs) on average energy silhouette images (AESIs). Based on the analysis of SDGs computation at various decomposition levels, an effective approach to compute the SDGs is developed. The AESI is constructed for the representation of the shape of action and activity and these are the reflection of 3-D pose into 2-D pose. To describe the AESIs, the SDGs at various sublevels and sum of the directional pixels (SDPs) variations is computed. The temporal content of the activity is computed through R-transform (RT). Finally, the shape computed through SDGs and SDPs, and temporal evidences through RT of the human body is fused together at the recognition stage, which results in a new powerful unified feature map model. The performance of the proposed framework is evaluated on three different publicly available datasets, i.e., Weizmann, KTH, and Ballet and the recognition accuracy is computed using hybrid classifier. The highest recognition accuracy achieved on these datasets is compared with the similar state-of-the-art techniques and demonstrate the superior performance.

[1]  Kostas Karpouzis,et al.  Exploring trace transform for robust human action recognition , 2013, Pattern Recognit..

[2]  Alfredo Petrosino,et al.  Human activity modeling by spatio temporal textural appearance , 2013, Pattern Recognit. Lett..

[3]  Alexandros Iosifidis,et al.  Discriminant Bag of Words based representation for human action recognition , 2014, Pattern Recognit. Lett..

[4]  Jian-Huang Lai,et al.  A Histogram-Based Chan-Vese Model Driven by Local Contrast Pattern for Texture Image Segmentation , 2014, 2014 22nd International Conference on Pattern Recognition.

[5]  Zafar Ali Khan,et al.  Abnormal human activity recognition system based on R-transform and kernel discriminant technique for elderly home care , 2011, IEEE Transactions on Consumer Electronics.

[6]  Yang Wang,et al.  Human Action Recognition by Semilatent Topic Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Vassilios Morellas,et al.  Action recognition using global spatio-temporal features derived from sparse representations , 2014, Comput. Vis. Image Underst..

[8]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories , 2006 .

[9]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Mohammed Bennamoun,et al.  Discriminative fusion of shape and appearance features for human pose estimation , 2013, Pattern Recognit..

[11]  Alexandros André Chaaraoui,et al.  A review on vision techniques applied to Human Behaviour Analysis for Ambient-Assisted Living , 2012, Expert Syst. Appl..

[12]  Laurent Wendling,et al.  A new shape descriptor defined on the Radon transform , 2006, Comput. Vis. Image Underst..

[13]  Thomas Serre,et al.  A Biologically Inspired System for Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[14]  Ling Shao,et al.  Silhouette Analysis-Based Action Recognition Via Exploiting Human Poses , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Hani Hagras,et al.  A fuzzy logic-based system for the automation of human behavior recognition using machine vision in intelligent environments , 2015, Soft Comput..

[16]  Rajiv Kapoor,et al.  Integrated approach for human action recognition using edge spatial distribution, direction pixel and -transform , 2015, Adv. Robotics.

[17]  Mohiuddin Ahmad,et al.  Variable silhouette energy image representations for recognizing human actions , 2010, Image Vis. Comput..

[18]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[19]  Benjamin Höferlin,et al.  Evaluation of background subtraction techniques for video surveillance , 2011, CVPR 2011.

[20]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[21]  Ickjai Lee,et al.  Expert Systems With Applications , 2013 .

[22]  Ahmed Bouridane,et al.  An FPGA Based Coprocessor for GLCM and Haralick Texture Features and their Application in Prostate Cancer Classification , 2005 .

[23]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[24]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[25]  Rachid Benmokhtar.  Robust human action recognition scheme based on high-level feature fusion , 2012, Multimedia Tools and Applications.

[26]  Greg Mori,et al.  Action recognition by learning mid-level motion features , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[28]  Robert Bergevin,et al.  Semantic human activity recognition: A literature review , 2015, Pattern Recognit..

[29]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[30]  Giulio Iannello,et al.  Multiple subsequence combination in human action recognition , 2014, IET Comput. Vis..

[31]  Ling Shao,et al.  Transform based spatio-temporal descriptors for human action recognition , 2011, Neurocomputing.

[32]  Huchuan Lu,et al.  Combining motion and appearance cues for anomaly detection , 2016, Pattern Recognit..

[33]  Yong Gan,et al.  An active contour model based on fused texture features for image segmentation , 2015, Neurocomputing.

[34]  Rajiv Kapoor,et al.  Hybrid classifier based human activity recognition using the silhouette and cells , 2015, Expert Syst. Appl..

[35]  Marek Gorgon,et al.  Foreground object features extraction with GLCM texture descriptor in FPGA , 2013, 2013 Conference on Design and Architectures for Signal and Image Processing.

[36]  Alexander Belyaev,et al.  GEI + HOG for Action Recognition , 2012 .

[37]  Amiya Kumar Rath,et al.  A Naïve SVM-KNN based stock market trend reversal analysis for Indian benchmark indices , 2015, Appl. Soft Comput..

[38]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[39]  Ling Shao,et al.  Combining appearance and structural features for human action recognition , 2013, Neurocomputing.

[40]  Jiwen Lu,et al.  Activity-based human identification , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[41]  Rajiv Kapoor,et al.  Recognition of abnormal human activity using the changes in orientation of silhouette in key frames , 2015, 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom).

[42]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[43]  Ling Shao,et al.  Spatio-Temporal Laplacian Pyramid Coding for Action Recognition , 2014, IEEE Transactions on Cybernetics.

[44]  Shaogang Gong,et al.  Fusing appearance and distribution information of interest points for action recognition , 2012, Pattern Recognit..

[45]  Ling Shao,et al.  Human action segmentation and recognition via motion and shape analysis , 2012, Pattern Recognit. Lett..

[46]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[47]  Ronen Basri,et al.  Actions as Space-Time Shapes , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Lun-zheng Tan,et al.  Human action recognition based on chaotic invariants , 2013, Journal of Central South University.

[49]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[50]  Jianfang Dou,et al.  Robust human action recognition based on spatio-temporal descriptors and motion temporal templates , 2014 .

[51]  Yue Wang,et al.  Adaptive texture-color based background subtraction for video surveillance , 2012, 2012 19th IEEE International Conference on Image Processing.

[52]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Redha Touati,et al.  MDS-based Multi-axial Dimensionality Reduction Model for Human Action Recognition , 2014, 2014 Canadian Conference on Computer and Robot Vision.

[54]  Deepu Rajan,et al.  Human action recognition using Pose-based discriminant embedding , 2012, Signal Process. Image Commun..

[55]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Rajiv Kapoor,et al.  An efficient interpretation of hand gestures to control smart interactive television , 2017, Int. J. Comput. Vis. Robotics.

[57]  Ling Shao,et al.  Action recognition by spatio-temporal oriented energies , 2014, Inf. Sci..

[58]  Rajiv Kapoor,et al.  Simple and intelligent system to recognize the expression of speech-disabled person , 2012, 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI).

[59]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[60]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[61]  Alexandros André Chaaraoui,et al.  Silhouette-based human action recognition using sequences of key poses , 2013, Pattern Recognit. Lett..

[62]  Jitendra Malik,et al.  SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).