Feasibility of Wrist-Worn, Real-Time Hand, and Surface Gesture Recognition via sEMG and IMU Sensing

While most wearable gesture recognition approaches focus on the forearm or fingers, the wrist may be a more suitable location for practical use. We present the design and validation of a real-time gesture recognition wristband based on surface electromyography and inertial measurement unit sensing fusion, which can recognize 8 air gestures and 4 surface gestures with 2 distinct force levels. Ten healthy subjects performed an initial gesture recognition experiment, followed by a second experiment 1 h later and a third experiment 1 day later. Classification accuracies for the initial experiment were 92.6% and 88.8% for air and surface gestures, respectively, and there were no changes in accuracy results during testing 1 h. and 1 day later (<inline-formula><tex-math notation="LaTeX"> $p$</tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX">$>$</tex-math></inline-formula> 0.05). These results demonstrate the feasibility of wrist-based gesture recognition paving the way for potential future integration in to a smart watch or other wrist-worn wearable for intuitive human computer interaction.

[1]  T. Gharbi,et al.  MMG measurement: a high-sensitivity microphone-based sensor for clinical use , 1998, IEEE Transactions on Biomedical Engineering.

[2]  Xinjun Sheng,et al.  Development of a Multi-Channel Compact-Size Wireless Hybrid sEMG/NIRS Sensor System for Prosthetic Manipulation , 2016, IEEE Sensors Journal.

[3]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[4]  Nooritawati Md Tahir,et al.  Review in Sign Language Recognition Systems , 2012, 2012 IEEE Symposium on Computers & Informatics (ISCI).

[5]  Carlo J. De Luca,et al.  The Use of Surface Electromyography in Biomechanics , 1997 .

[6]  Weihua Sheng,et al.  Wearable Sensor-Based Hand Gesture and Daily Activity Recognition for Robot-Assisted Living , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[7]  Yasue Mitsukura,et al.  Wrist EMG signals identification using neural network , 2009, 2009 35th Annual Conference of IEEE Industrial Electronics.

[8]  Marco Platzner,et al.  Fluctuating emg signals: Investigating long-term effects of pattern matching algorithms , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[9]  Kongqiao Wang,et al.  Hand gesture recognition and virtual game control based on 3D accelerometer and EMG sensors , 2009, IUI.

[10]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[11]  Xinjun Sheng,et al.  Reduced Daily Recalibration of Myoelectric Prosthesis Classifiers Based on Domain Adaptation , 2016, IEEE Journal of Biomedical and Health Informatics.

[12]  Jun Rekimoto,et al.  GestureWrist and GesturePad: unobtrusive wearable interaction devices , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[13]  Nianfeng Wang,et al.  The recognition of multi-finger prehensile postures using LDA , 2013, Biomed. Signal Process. Control..

[14]  Patrick Olivier,et al.  Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor , 2012, UIST.

[15]  Kentaro Takemura,et al.  Hand pose estimation based on active bone-conducted sound sensing , 2016, UbiComp Adjunct.

[16]  Kui Wang,et al.  Gesture-based telemanipulation of a humanoid robot for home service tasks , 2015, 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).

[17]  Asha,et al.  A Hand Gesture Recognition Framework and Wearable Gesture Based Interaction Prototype for Mobile Devices , 2015 .

[18]  Thomas Garvey,et al.  REACHING FOR HOLOGRAMS : ASSESSING THE ERGONOMICS OF THE MICROSOFT™ HOLOLENS™ 3D GESTURE KNOWN AS THE "AIR TAP" , 2015 .

[19]  Susan Goldin-Meadow,et al.  Creating a communication system from scratch: gesture beats vocalization hands down , 2014, Front. Psychol..

[20]  Huzefa Rangwala,et al.  Novel Method for Predicting Dexterous Individual Finger Movements by Imaging Muscle Activity Using a Wearable Ultrasonic System , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Levi J. Hargrove,et al.  A Comparison of Surface and Intramuscular Myoelectric Signal Classification , 2007, IEEE Transactions on Biomedical Engineering.

[22]  Pyeong-Gook Jung,et al.  A Wearable Gesture Recognition Device for Detecting Muscular Activities Based on Air-Pressure Sensors , 2015, IEEE Transactions on Industrial Informatics.

[23]  Antonio Krüger,et al.  EMPress: Practical Hand Gesture Classification with Wrist-Mounted EMG and Pressure Sensing , 2016, CHI.

[24]  Han Liu,et al.  An Efficient Method to Estimate Labelled Sample Size for Transductive LDA(QDA/MDA) Based on Bayes Risk , 2004, ECML.

[25]  Yasue Mitsukura,et al.  Identification of wrist EMG signals using dry type electrodes , 2009, 2009 ICCAS-SICE.

[26]  Joseph A. Paradiso,et al.  WristFlex: low-power gesture input with wrist-worn pressure sensors , 2014, UIST.

[27]  Xinjun Sheng,et al.  Boosting training for myoelectric pattern recognition using Mixed-LDA , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[28]  Xinjun Sheng,et al.  Toward an Enhanced Human–Machine Interface for Upper-Limb Prosthesis Control With Combined EMG and NIRS Signals , 2017, IEEE Transactions on Human-Machine Systems.

[29]  Max Ortiz-Catalan,et al.  BioPatRec: A modular research platform for the control of artificial limbs based on pattern recognition algorithms , 2013, Source Code for Biology and Medicine.

[30]  Kent Lyons,et al.  The Gesture Watch: A Wireless Contact-free Gesture based Wrist Interface , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[31]  Loredana Zollo,et al.  NLR, MLP, SVM, and LDA: a comparative analysis on EMG data from people with trans-radial amputation , 2017, Journal of NeuroEngineering and Rehabilitation.

[32]  Erik J. Scheme,et al.  Selective Classification for Improved Robustness of Myoelectric Control Under Nonideal Conditions , 2011, IEEE Transactions on Biomedical Engineering.

[33]  Ulrik Schroeder,et al.  Wearable Sensors for eLearning of Manual Tasks: Using Forearm EMG in Hand Hygiene Training , 2016, Sensors.

[34]  A. Chan,et al.  Surface Electromyographic Signals Using Dry Electrodes , 2010, IEEE Transactions on Instrumentation and Measurement.

[35]  Nadia Magnenat-Thalmann,et al.  AR in Hand: Egocentric Palm Pose Tracking and Gesture Recognition for Augmented Reality Applications , 2015, ACM Multimedia.

[36]  Huosheng Hu,et al.  Support Vector Machine-Based Classification Scheme for Myoelectric Control Applied to Upper Limb , 2008, IEEE Transactions on Biomedical Engineering.

[37]  Christopher Assad,et al.  Gesture-based robot control with variable autonomy from the JPL BioSleeve , 2013, 2013 IEEE International Conference on Robotics and Automation.

[38]  J.W. Sensinger,et al.  Adaptive Pattern Recognition of Myoelectric Signals: Exploration of Conceptual Framework and Practical Algorithms , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.