Approach to Hand Tracking and Gesture Recognition Based on Depth-Sensing Cameras and EMG Monitoring

In this paper, a new approach for hand tracking and gesture recognition based on the Leap Motion device and surface electromyography (SEMG) is presented. The system is about to process the depth image information and the electrical activity produced by skeletal muscles on forearm. The purpose of such combination is enhancement in the gesture recognition rate. As a first we analyse the conventional approaches toward hand tracking and gesture recognition and present the results of various researches. Successive topic gives brief overview of depth-sensing cameras with focus on Leap motion device where we test its accuracy of fingers recognition. The vision-SEMG-based system is to be potentially applicable to many areas of human computer interaction.

[1]  Mauro Donadeo,et al.  Combining multiple depth-based descriptors for hand gesture recognition , 2014, Pattern Recognit. Lett..

[2]  Changsheng Xu,et al.  Discriminative Exemplar Coding for Sign Language Recognition With Kinect , 2013, IEEE Transactions on Cybernetics.

[3]  Yi Jiang,et al.  Applying infrared radiation image sensor to step-on interface: Touched point detection and tracking , 2012, 2012 IEEE/SICE International Symposium on System Integration (SII).

[4]  M. A. C. García,et al.  Surface electromyography: Why, when and how to use it , 2011 .

[5]  Qiang Li,et al.  A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices , 2014, IEEE Transactions on Human-Machine Systems.

[6]  Francisco Javier Díaz Pernas,et al.  A Kinect-based system for cognitive rehabilitation exercises monitoring , 2014, Comput. Methods Programs Biomed..

[7]  Yao-Jen Chang,et al.  A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. , 2011, Research in developmental disabilities.

[8]  B. Katirji.  Electromyography in Clinical Practice: A Case Study Approach , 1998 .

[9]  Yong Peng,et al.  EEG-based emotion classification using deep belief networks , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[10]  Xiangyang Zhu,et al.  Dynamical Characteristics of Surface EMG Signals of Hand Grasps via Recurrence Plot , 2014, IEEE Journal of Biomedical and Health Informatics.

[11]  Christian Stolte,et al.  The Molecular Control Toolkit: Controlling 3D molecular graphics via gesture and voice , 2013, 2013 IEEE Symposium on Biological Data Visualization (BioVis).

[12]  Kandarpa Kumar Sarma,et al.  Gloved and free hand tracking based hand gesture recognition , 2013, 2013 1st International Conference on Emerging Trends and Applications in Computer Science.

[13]  Zhizeng Luo,et al.  Pattern Classification of Surface Electromyography Based on AR Model and High-order Neural Network , 2006, 2006 2nd IEEE/ASME International Conference on Mechatronics and Embedded Systems and Applications.

[14]  Andrew Eisen,et al.  Electromyography in clinical practice. A case study approach , 2000 .

[15]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Saeid Nahavandi,et al.  Measuring depth accuracy in RGBD cameras , 2013, 2013, 7th International Conference on Signal Processing and Communication Systems (ICSPCS).

[17]  Satish Chandra,et al.  Gesture recognition using kinect for sign language translation , 2013, 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013).

[18]  Jose-Juan Hernandez-Lopez,et al.  Detecting objects using color and depth segmentation with Kinect sensor , 2012 .

[19]  Honghai Liu,et al.  Human Hand Motion Analysis With Multisensory Information , 2014, IEEE/ASME Transactions on Mechatronics.

[20]  Jana Abhijit.  Kinect for Windows SDK Programming Guide , 2012 .

[21]  Mohamed Jemni,et al.  3D Motion Trajectory Analysis Approach to Improve Sign Language 3D-based Content Recognition , 2012, INNS-WC.