HAND-REHA: dynamic hand gesture recognition for game-based wrist rehabilitation

Hand-gesture recognition systems have recently gained more popularity. Moreover, there is a growing interest in building games for other purposes apart from entertainment, such as education and rehabilitation. This paper focuses on developing a novel game-based system for wrist rehabilitation, called HandReha. The idea is to automatically recognize pre-defined hand gestures using a web-camera, so to control an avatar in a three dimensional maze run game. The pre-defined gestures are picked from a pool of well-defined gestures suitable for wrist rehabilitation. Deep learning techniques were utilized to perform real-time hand gesture recognition from the images. To evaluate the performance of the developed wrist rehabilitation system, a preliminary study with 12 healthy participants was conducted. The results showed that the developed wrist rehabilitation system is intuitive and engages the user, which is crucial for rehabilitation purposes.

[1]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[2]  Meng Ma,et al.  MirrARbilitation: A clinically-related gesture recognition interactive tool for an AR rehabilitation system , 2016, Comput. Methods Programs Biomed..

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  Pei Xu,et al.  A Real-time Hand Gesture Recognition and Human-Computer Interaction System , 2017, ArXiv.

[5]  Iosr Journals,et al.  Detecting and tracking the hand to create an augmented reality system , 2014 .

[6]  Zhiquan Feng,et al.  Multi-sensors Based 3D Gesture Recognition and Interaction in Virtual Block Game , 2017, 2017 International Conference on Virtual Reality and Visualization (ICVRV).

[7]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[8]  Vassilis Athitsos,et al.  Analysis of hand segmentation on challenging hand over face scenario , 2019, PETRA.

[9]  Siew Wen Chin,et al.  Game-based human computer interaction using gesture recognition for rehabilitation , 2013, 2013 IEEE International Conference on Control System, Computing and Engineering.

[10]  Fillia Makedon,et al.  An automated assessment system for embodied cognition in children: from motion data to executive functioning , 2019, iWOAR.

[11]  Fillia Makedon,et al.  An Intelligent Action Recognition System to assess Cognitive Behavior for Executive Function Disorder , 2019, 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE).

[12]  Edgar Rodríguez Ramírez,et al.  A tangible interface and augmented reality game for facilitating sit-to-stand exercises for stroke rehabilitation , 2018, IOT.

[13]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Fillia Makedon,et al.  Real-Time Static Gesture Recognition for Upper Extremity Rehabilitation Using the Leap Motion , 2015, HCI.

[15]  Rachel Proffitt,et al.  User-Centered Design of a Controller-Free Game for Hand Rehabilitation. , 2015, Games for health journal.

[16]  Yassine Bouteraa,et al.  Training of Hand Rehabilitation Using Low Cost Exoskeleton and Vision-Based Game Interface , 2019, Journal of Intelligent & Robotic Systems.

[17]  Philip Beesley,et al.  Architectural Design in Open-Source Software Developing MeasureIt-ARCH, an Open Source tool to create Dimensioned and Annotated Architectural drawings within the Blender 3D creation suite. , 2019 .

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Xiaoling Xia,et al.  Inception-v3 for flower classification , 2017, 2017 2nd International Conference on Image, Vision and Computing (ICIVC).

[20]  S N Karishma,et al.  Fusion of Skin Color Detection and Background Subtraction for Hand Gesture Segmentation , 2014 .

[21]  Mike Y. Chen,et al.  FitBird: Improving Free-weight Training Experience using Wearable Sensors for Game Control , 2019, CHI PLAY.

[22]  Li-Yang Huang,et al.  HP2: Using Machine Learning Model to Play Serious Game with IMU Smart Suit , 2018, MUM.

[23]  Kuan-Ta Chen,et al.  What Do Stroke Patients Look for in Game-Based Rehabilitation , 2016, Medicine.

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  E. S. Gedraite,et al.  Investigation on the effect of a Gaussian Blur in image filtering and segmentation , 2011, Proceedings ELMAR-2011.

[26]  Mahmoud A. Mofaddel,et al.  Fast and accurate approaches for image and moving object segmentation , 2011, The 2011 International Conference on Computer Engineering & Systems.

[27]  Sergio Escalera,et al.  Challenges in Multi-modal Gesture Recognition , 2017, Gesture Recognition.