A Proportional Pattern Recognition Control Scheme for Wearable A-mode Ultrasound Sensing

It is evident that the prevailing solution, myoelectric pattern recognition for prosthetic manipulation, constrains gesture-based interaction because of the lack of proportional control information such as exerted force. This paper reports an attempt, named simultaneous gesture recognition and muscle contraction force estimation, to realize proportional pattern recognition (PPR) control taking advantage of arm muscle deformation via wearable ultrasound sensing. We experiment with eight types of predefined hand motions, with a range of 0–60% maximum voluntary contraction (MVC) using a wearable multichannel A-mode ultrasound system. The experiment result demonstrates that above 93.7% of gestures are correctly recognized during dynamic muscle contraction forces (0–60% MVC), albeit only training at a slight force level (<6% MVC). Besides, the adopted nonparametric Gaussian process regression estimates the muscle contraction forces accurately and synchronously, with average coefficient of determination, ${\bf R}^{\bf {2}}$, and normalized root-mean-square error ($nRMSE$) of 0.927 and 0.102, respectively. These research outcomes demonstrate the feasibility of ultrasound-based PPR control, paving the way for musculature-driven applications such as finer prosthetic control, remote manipulation, and rehabilitation treatment.

[1]  Xinjun Sheng,et al.  Feasibility of Wrist-Worn, Real-Time Hand, and Surface Gesture Recognition via sEMG and IMU Sensing , 2018, IEEE Transactions on Industrial Informatics.

[2]  Claudio Castellini,et al.  Ultrasound imaging as a human-machine interface in a realistic scenario , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[4]  Dario Farina,et al.  Myoelectric Control of Artificial Limbs¿Is There a Need to Change Focus? [In the Spotlight] , 2012, IEEE Signal Process. Mag..

[5]  Nathan E. Bunderson,et al.  Quantification of Feature Space Changes With Experience During Electromyogram Pattern Recognition Control , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  Mike Fraser,et al.  EchoFlex: Hand Gesture Recognition using Ultrasound Imaging , 2017, CHI.

[7]  Honghai Liu,et al.  Ultrasound-Based Sensing Models for Finger Motion Classification , 2018, IEEE Journal of Biomedical and Health Informatics.

[8]  Erik Scheme,et al.  Motion Normalized Proportional Control for Improved Pattern Recognition-Based Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Erik Scheme,et al.  Training Strategies for Mitigating the Effect of Proportional Control on Classification in Pattern Recognition–Based Myoelectric Control , 2013, Journal of prosthetics and orthotics : JPO.

[10]  Honghai Liu,et al.  Towards Wearable A-Mode Ultrasound Sensing for Real-Time Finger Motion Recognition , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  Geoffrey E. Hinton,et al.  Evaluation of Gaussian processes and other methods for non-linear regression , 1997 .

[12]  C. Castellini,et al.  Using Ultrasound Images of the Forearm to Predict Finger Positions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[13]  Nicolas D. Georganas,et al.  Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques , 2011, IEEE Transactions on Instrumentation and Measurement.

[14]  Giulio Sandini,et al.  Surface EMG for force control of mechanical hands , 2008, 2008 IEEE International Conference on Robotics and Automation.

[15]  Xiangyang Zhu,et al.  Dual-Frequency Ultrasound Transducers for the Detection of Morphological Changes of Deep-Layered Muscles , 2018, IEEE Sensors Journal.

[16]  Ann M. Simon,et al.  A comparison of proportional control methods for pattern recognition control , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  T. Abe,et al.  Associations between Handgrip Strength and Ultrasound-Measured Muscle Thickness of the Hand and Forearm in Young Men and Women. , 2015, Ultrasound in medicine & biology.

[18]  Huzefa Rangwala,et al.  Novel Method for Predicting Dexterous Individual Finger Movements by Imaging Muscle Activity Using a Wearable Ultrasonic System , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Jana Kosecka,et al.  Real-Time Classification of Hand Motions Using Ultrasound Imaging of Forearm Muscles , 2016, IEEE Transactions on Biomedical Engineering.

[20]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[21]  Thomas L. Szabo,et al.  Diagnostic Ultrasound Imaging: Inside Out , 2004 .

[22]  Yuefeng Li,et al.  Human-machine interface based on multi-channel single-element ultrasound transducers: A preliminary study , 2016, 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom).

[23]  O. Stavdahl,et al.  Control of Upper Limb Prostheses: Terminology and Proportional Myoelectric Control—A Review , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  Patrick van der Smagt,et al.  Surface EMG in advanced hand prosthetics , 2008, Biological Cybernetics.

[25]  Erik J Scheme,et al.  A proportional control scheme for high density force myography , 2018, Journal of neural engineering.

[26]  Giulio Sandini,et al.  Fine detection of grasp force and posture by amputees via surface electromyography , 2009, Journal of Physiology-Paris.

[27]  Honghai Liu,et al.  A New Wearable Ultrasound Muscle Activity Sensing System for Dexterous Prosthetic Control , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[28]  Jing-Yi Guo,et al.  Recognition of Finger Flexion from Ultrasound Image with Optical Flow: A Preliminary Study , 2010, 2010 International Conference on Biomedical Engineering and Computer Science.

[29]  Qinghua Huang,et al.  Continuous Monitoring of Sonomyography, Electromyography and Torque Generated by Normal Upper Arm Muscles During Isometric Contraction: Sonomyography Assessment for Arm Muscles , 2008, IEEE Transactions on Biomedical Engineering.

[30]  Christian Cipriani,et al.  Ultrasound imaging for hand prosthesis control: a comparative study of features and classification methods , 2015, 2015 IEEE International Conference on Rehabilitation Robotics (ICORR).

[31]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[32]  Claudio Castellini,et al.  A realistic implementation of ultrasound imaging as a human-machine interface for upper-limb amputees , 2013, Front. Neurorobot..