Deep Learning-Based Action Recognition Using 3D Skeleton Joints Information

Human action recognition has turned into one of the most attractive and demanding fields of research in computer vision and pattern recognition for facilitating easy, smart, and comfortable ways of human-machine interaction. With the witnessing of massive improvements to research in recent years, several methods have been suggested for the discrimination of different types of human actions using color, depth, inertial, and skeleton information. Despite having several action identification methods using different modalities, classifying human actions using skeleton joints information in 3-dimensional space is still a challenging problem. In this paper, we conceive an efficacious method for action recognition using 3D skeleton data. First, large-scale 3D skeleton joints information was analyzed and accomplished some meaningful pre-processing. Then, a simple straight-forward deep convolutional neural network (DCNN) was designed for the classification of the desired actions in order to evaluate the effectiveness and embonpoint of the proposed system. We also conducted prior DCNN models such as ResNet18 and MobileNetV2, which outperform existing systems using human skeleton joints information.

[1]  Tieniu Tan,et al.  Skeleton-based action recognition with hierarchical spatial reasoning and temporal stack learning network , 2020, Pattern Recognit..

[2]  Pichao Wang,et al.  Joint Distance Maps Based Action Recognition With Convolutional Neural Networks , 2017, IEEE Signal Processing Letters.

[3]  Petros Maragos,et al.  Fusing Body Posture With Facial Expressions for Joint Recognition of Affect in Child–Robot Interaction , 2019, IEEE Robotics and Automation Letters.

[4]  Qian Wang,et al.  Deep Learning-Based Gait Recognition Using Smartphones in the Wild , 2018, IEEE Transactions on Information Forensics and Security.

[5]  Wei Liu,et al.  Discriminative Multi-instance Multitask Learning for 3D Action Recognition , 2017, IEEE Transactions on Multimedia.

[6]  Pichao Wang,et al.  Skeleton Optical Spectra-Based Action Recognition Using Convolutional Neural Networks , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Gholamreza Anbarjafari,et al.  Emotion Recognition from Skeletal Movements , 2019, Entropy.

[8]  Balasubramanian Raman,et al.  Evaluating fusion of RGB-D and inertial sensors for multimodal human action recognition , 2020, J. Ambient Intell. Humaniz. Comput..

[9]  Chee Sun Won,et al.  A Survey of Human Action Recognition Approaches that use an RGB-D Sensor , 2015 .

[10]  Xiaogang Wang,et al.  A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Xing Liu,et al.  Learning shape and motion representations for view invariant skeleton-based action recognition , 2020, Pattern Recognit..

[12]  Ling Shao,et al.  Silhouette Analysis-Based Action Recognition Via Exploiting Human Poses , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Hairong Qi,et al.  Spatio-temporal feature extraction and representation for RGB-D human action recognition , 2014, Pattern Recognit. Lett..

[14]  Yonghong Hou,et al.  ConvNets-based action recognition from skeleton motion maps , 2019, Multimedia Tools and Applications.

[15]  Jing Zhang,et al.  Action Recognition From Depth Maps Using Deep Convolutional Neural Networks , 2016, IEEE Transactions on Human-Machine Systems.

[16]  Dong-Seong Kim,et al.  Image representation of pose-transition feature for 3D skeleton-based action recognition , 2020, Inf. Sci..