Dynamic Gesture Recognition Algorithm Based on 3D Convolutional Neural Network

Gesture recognition is one of the important ways of human-computer interaction, which is mainly detected by visual technology. The temporal and spatial features are extracted by convolution of the video containing gesture. However, compared with the convolution calculation of a single image, multiframe image of dynamic gestures has more computation, more complex feature extraction, and more network parameters, which affects the recognition efficiency and real-time performance of the model. To solve above problems, a dynamic gesture recognition model based on CBAM-C3D is proposed. Key frame extraction technology, multimodal joint training, and network optimization with BN layer are used for making the network performance better. The experiments show that the recognition accuracy of the proposed 3D convolutional neural network combined with attention mechanism reaches 72.4% on EgoGesture dataset, which is improved greatly compared with the current main dynamic gesture recognition methods, and the effectiveness of the proposed algorithm is verified.

[1]  W. Marsden.  I and J , 2012 .

[2]  Gongfa Li,et al.  Decomposition algorithm for depth image of human health posture based on brain health , 2019, Neural Computing and Applications.

[3]  Hao Wu,et al.  Occlusion gesture recognition based on improved SSD , 2020, Concurr. Comput. Pract. Exp..

[4]  H. R. Nandi Vardhan,et al.  Smart gloves for hand gesture recognition: Sign language to speech conversion system , 2016, 2016 International Conference on Robotics and Automation for Humanitarian Applications (RAHA).

[5]  Tanima Dutta,et al.  A Continuous Hand Gestures Recognition Technique for Human-Machine Interaction Using Accelerometer and Gyroscope Sensors , 2016, IEEE Sensors Journal.

[6]  Yi Zhu,et al.  Hidden Two-Stream Convolutional Networks for Action Recognition , 2017, ACCV.

[7]  Hanqing Lu,et al.  EgoGesture: A New Dataset and Benchmark for Egocentric Hand Gesture Recognition , 2018, IEEE Transactions on Multimedia.

[8]  Andrew Zisserman,et al.  Convolutional Two-Stream Network Fusion for Video Action Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Honghai Liu,et al.  Research on gesture recognition of smart data fusion features in the IoT , 2019, Neural Computing and Applications.

[11]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[12]  장윤희,et al.  Y. , 2003, Industrial and Labor Relations Terms.

[13]  Wei Li,et al.  Dynamic gesture recognition based on feature fusion network and variant ConvLSTM , 2020, IET Image Process..

[14]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[15]  Mustapha Hamerlain,et al.  SLAM based on Adaptive SVSF for Cooperative Unmanned Vehicles in Dynamic environment , 2019 .

[16]  Gongfa Li,et al.  An effective and unified method to derive the inverse kinematics formulas of general six-DOF manipulator with simple geometry , 2021 .

[17]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[18]  Du Jiang,et al.  Gesture recognition based on multi‐modal feature weight , 2020, Concurr. Comput. Pract. Exp..

[19]  Guodong Cong,et al.  Monitoring and Recognizing Enterprise Public Opinion from High-Risk Users Based on User Portrait and Random Forest Algorithm , 2021, Axioms.

[20]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[21]  Yaoqing Weng,et al.  Enhancement of real‐time grasp detection by cascaded deep convolutional neural networks , 2020, Concurr. Comput. Pract. Exp..

[22]  Pavlo Molchanov,et al.  Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[24]  Junfeng Wang,et al.  Research on the visual elements of augmented reality assembly processes , 2019, Virtual Real. Intell. Hardw..

[25]  Gongfa Li,et al.  Gesture recognition based on surface electromyography‐feature image , 2020, Concurr. Comput. Pract. Exp..

[26]  Guodong Cong,et al.  Analysis of User Needs on Downloading Behavior of English Vocabulary APPs Based on Data Mining for Online Comments , 2021, Mathematics.

[27]  Tinggui Chen,et al.  Analysis of Social Effects on Employment Promotion Policies for College Graduates Based on Data Mining for Online Use Review in China during the COVID-19 Pandemic , 2021, Healthcare.

[28]  Karl F. MacDorman,et al.  Review of constraints on vision-based gesture recognition for human-computer interaction , 2018, IET Comput. Vis..

[29]  Vibhav Vineet,et al.  Struck: Structured Output Tracking with Kernels , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Hao Wu,et al.  Dynamic Gesture Recognition in the Internet of Things , 2019, IEEE Access.

[31]  Gongfa Li,et al.  Grip strength forecast and rehabilitative guidance based on adaptive neural fuzzy inference system using sEMG , 2019, Personal and Ubiquitous Computing.

[32]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Honghai Liu,et al.  Gesture recognition based on an improved local sparse representation classification algorithm , 2017, Cluster Computing.

[34]  Hujun Bao,et al.  Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality , 2019, Virtual Real. Intell. Hardw..

[35]  Naixue Xiong,et al.  Research on strong agile response task scheduling optimization enhancement with optimal resource usage in green cloud computing , 2021, Future Gener. Comput. Syst..

[36]  Carlos R. del-Blanco,et al.  Tiny hand gesture recognition without localization via a deep convolutional network , 2017, IEEE Transactions on Consumer Electronics.

[37]  Bo Tao,et al.  Gesture recognition based on skeletonization algorithm and CNN with ASL database , 2018, Multimedia Tools and Applications.

[38]  Stefan Wermter,et al.  An analysis of Convolutional Long Short-Term Memory Recurrent Neural Networks for gesture recognition , 2017, Neurocomputing.

[39]  Luc Van Gool,et al.  Temporal Segment Networks: Towards Good Practices for Deep Action Recognition , 2016, ECCV.

[40]  Weili Wu,et al.  Two-Phase Multidocument Summarization Through Content-Attention-Based Subtopic Detection , 2021, IEEE Transactions on Computational Social Systems.

[41]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[42]  Tao Mei,et al.  Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[43]  Hui Yu,et al.  Gesture recognition based on binocular vision , 2018, Cluster Computing.

[44]  Gongfa Li,et al.  Manipulator grabbing position detection with information fusion of color image and depth image using deep learning , 2021, Journal of Ambient Intelligence and Humanized Computing.

[45]  Yi Wu,et al.  Gesture recognition based on deep deformable 3D convolutional neural networks , 2020, Pattern Recognit..

[46]  Yongkang Wong,et al.  Surface-Electromyography-Based Gesture Recognition by Multi-View Deep Learning , 2019, IEEE Transactions on Biomedical Engineering.

[47]  Feng Guo,et al.  Human action recognition based on HOIRM feature fusion and AP clustering BOW , 2019, PloS one.

[48]  Gongfa Li,et al.  Semantic segmentation for multiscale target based on object recognition using the improved Faster-RCNN model , 2021, Future Gener. Comput. Syst..

[49]  Hongying Meng,et al.  Holoscopic 3D Microgesture Recognition by Deep Neural Network Model Based on Viewpoint Images and Decision Fusion , 2021, IEEE Transactions on Human-Machine Systems.

[50]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[51]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[52]  Pavlo Molchanov,et al.  Hand gesture recognition with 3D convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[53]  J. Kautz,et al.  Hand Gesture Recognition with 3 D Convolutional Neural Networks , 2015 .