Smart Healthcare Hand Gesture Recognition Using CNN-Based Detector and Deep Belief Network

Gesture recognition in dynamic images is challenging in computer vision, automation and medical field. Hand gesture tracking and recognition between both human and computer must have symmetry in real world. With advances in sensor technology, numerous researchers have recently proposed RGB gesture recognition techniques. In our research paper, we introduce a reliable hand gesture tracking and recognition model that is accurate despite any complex environment, it can track and recognise RGB dynamic gestures. Firstly, videos are converted into frames. After images light intensity adjustment and noise removal, images are passed through CNN for hand gesture extraction. Then from the extracted hand, features are extracted from full hand. Neural gas and locomotion thermal mapping are extracted to make the feature vector. The feature vector are then passed through the fuzzy optimiser to reduce the uncertainties and the fuzziness. The optimised features are then passed to the classifier Deep Belief Network (DBW) for the classification of the gestures. Egogesture and Jester datasets are used for the validation of proposed systems. The experimental results over Egogesture and Jester datasets demonstrate overall accuracies of 90.73% and 89.33% respectively. The experiments proves our system readability and suitability of our proposed model with the other state of the arts model.

[1]  Toktam Khatibi,et al.  Patient’s actions recognition in hospital’s recovery department based on RGB-D dataset , 2022, Multimedia Tools and Applications.

[2]  Jie Yang,et al.  Recurring the Transformer for Video Action Recognition , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  L. Nicolescu,et al.  Human-Computer Interaction in Customer Service: The Experience with AI Chatbots—A Systematic Literature Review , 2022, Electronics.

[4]  Amit K. Roy-Chowdhury,et al.  Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations , 2021, NeurIPS.

[5]  Fethi Fkih,et al.  Similarity measures for Collaborative Filtering-based Recommender Systems: Review and experimental comparison , 2021, J. King Saud Univ. Comput. Inf. Sci..

[6]  Pin-Chao Liao,et al.  Research Trends of Human–Computer Interaction Studies in Construction Hazard Recognition: A Bibliometric Review , 2021, Sensors.

[7]  Bo Tao,et al.  Dynamic Gesture Recognition Algorithm Based on 3D Convolutional Neural Network , 2021, Comput. Intell. Neurosci..

[8]  Rytis Maskeliūnas,et al.  Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model , 2021, Applied Sciences.

[9]  Munkhjargal Gochoo,et al.  Hand Gesture Recognition Based on Auto-Landmark Localization and Reweighted Genetic Algorithm for Healthcare Muscle Activities , 2021 .

[10]  Phaneendra K. Yalavarthy,et al.  Design and Implementation of Deep Learning Based Contactless Authentication System Using Hand Gestures , 2021, Electronics.

[11]  Ahmed M. Soliman,et al.  Smart healthcare solutions using the internet of medical things for hand gesture recognition system , 2020, Complex & Intelligent Systems.

[12]  Fabiane Maia Garcia,et al.  Professor , 2006, Encyclopedia of Evolutionary Psychological Science.

[13]  Javaan S. Chahl,et al.  Hand Gesture Recognition Based on Computer Vision: A Review of Techniques , 2020, J. Imaging.

[14]  Anouar Ben Khalifa,et al.  Hand-Gesture-Based Touchless Exploration of Medical Images with Leap Motion Controller , 2020, 2020 17th International Multi-Conference on Systems, Signals & Devices (SSD).

[15]  Ying Sun,et al.  Intelligent human computer interaction based on non redundant EMG signal , 2020, Alexandria Engineering Journal.

[16]  R. Suguna,et al.  An efficient method for human hand gesture detection and recognition using deep learning convolutional neural networks , 2020, Soft Computing.

[17]  Syed Omer Gilani,et al.  Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG , 2020, Sensors.

[18]  Ahmet Gunduz,et al.  Online Dynamic Hand Gesture Recognition Including Efficiency Analysis , 2020, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[19]  Lin Chen,et al.  Hand Gesture Recognition Using Compact CNN via Surface Electromyography Signals , 2020, Sensors.

[20]  Muhammad Ghulam,et al.  Hand Gesture Recognition Using 3D-CNN Model , 2020, IEEE Consumer Electronics Magazine.

[21]  Giancarlo Fortino,et al.  Human emotion recognition using deep belief network architecture , 2019, Inf. Fusion.

[22]  Joanna Materzynska,et al.  The Jester Dataset: A Large-Scale Video Dataset of Human Gestures , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[23]  Aljosa Smolic,et al.  Simultaneous Segmentation and Recognition: Towards More Accurate Ego Gesture Recognition , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[24]  Alper Basturk,et al.  Transfer learning-based convolutional neural networks with heuristic optimization for hand gesture recognition , 2019, Neural Computing and Applications.

[25]  Ying Zhang,et al.  Intrusion Detection for IoT Based on Improved Genetic Algorithm and Deep Belief Network , 2019, IEEE Access.

[26]  Anibal T. de Almeida,et al.  Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier , 2018, Biomed. Signal Process. Control..

[27]  Namkug Kim,et al.  Enhancement of gesture recognition for contactless interface using a personalized classifier in the operating room , 2018, Comput. Methods Programs Biomed..

[28]  Honghai Liu,et al.  Hand gesture recognition based on convolution neural network , 2017, Cluster Computing.

[29]  Rashima Mahajan,et al.  Medical Images Contrast Enhancement using Quad Weighted Histogram Equalization with Adaptive Gama Correction and Homomorphic Filtering , 2017 .

[30]  Woei-Chyn Chu,et al.  Hand gesture recognition for post-stroke rehabilitation using leap motion , 2017, 2017 International Conference on Applied System Innovation (ICASI).

[31]  Youngwook Kim,et al.  Hand Gesture Recognition Using Micro-Doppler Signatures With Convolutional Neural Network , 2016, IEEE Access.

[32]  Andrew Clark,et al.  A System for a Hand Gesture-Manipulated Virtual Reality Environment , 2016, SAICSIT '16.

[33]  Pavlo Molchanov,et al.  Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Ana-Maria Cretu,et al.  Static and Dynamic Hand Gesture Recognition in Depth Data Using Dynamic Time Warping , 2016, IEEE Transactions on Instrumentation and Measurement.

[35]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Ian D. Walker,et al.  Use of kinect depth data and Growing Neural Gas for gesture based robot control , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[37]  Naoyuki Kubota,et al.  Growing neural gas for information extraction in gesture recognition and reproduction of robot partners , 2012, 2012 International Symposium on Micro-NanoMechatronics and Human Science (MHS).

[38]  Luc Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, WACV.

[39]  D. R. Neog,et al.  A Hand Gesture-operated System for Rehabilitation using an End-to-End Detection Framework , 2023, IEEE Transactions on Artificial Intelligence.

[40]  Aurobinda Routray,et al.  A Human-Computer Interaction framework for emotion recognition through time-series thermal video sequences , 2021, Comput. Electr. Eng..

[41]  Eiad Yafi,et al.  A Systematic Review of Human-Computer Interaction and Explainable Artificial Intelligence in Healthcare with Artificial Intelligence Techniques , 2021, IEEE Access.

[42]  Med Salim Bouhlel,et al.  LeapGestureDB: A Public Leap Motion Database Applied for Dynamic Hand Gesture Recognition in Surgical Procedures , 2020 .

[43]  Hao Wu,et al.  Dynamic Gesture Recognition in the Internet of Things , 2019, IEEE Access.

[44]  Weihua Gui,et al.  A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network. , 2019, ISA transactions.

[45]  Angelo Cenedese,et al.  A Random Forest-based Approach for Hand Gesture Recognition with Wireless Wearable Motion Capture Sensors , 2019, IFAC-PapersOnLine.

[46]  Rajiv Ranjan Sahay,et al.  Deep Gesture: Static Hand Gesture Recognition Using CNN , 2016, CVIP.

[47]  L. Van Gool,et al.  Combining RGB and ToF cameras for real-time 3D hand gesture interaction , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).