Towards Wearable A-Mode Ultrasound Sensing for Real-Time Finger Motion Recognition

It is evident that surface electromyography (sEMG) based human–machine interfaces (HMI) have inherent difficulty in predicting dexterous musculoskeletal movements such as finger motions. This paper is an attempt to investigate a plausible alternative to sEMG, ultrasound-driven HMI, for dexterous motion recognition due to its characteristic of detecting morphological changes of deep muscles and tendons. A multi-channel A-mode ultrasound lightweight device is adopted to evaluate the performance of finger motion recognition; an experiment is designed for both widely acceptable offline and online algorithms with eight able-bodied subjects employed. The experiment result presents that the offline recognition accuracy is up to 98.83% ± 0.79%. The real-time motion completion rate is 95.4% ± 8.7% and online motion selection time is 0.243 ± 0.127 s. The outcomes confirm the feasibility of A-mode ultrasound based wearable HMI and its prosperous applications in prosthetic devices, virtual reality, and remote manipulation.

[1]  Jing-Yi Guo,et al.  Dynamic monitoring of forearm muscles using one-dimensional sonomyography system. , 2008, Journal of rehabilitation research and development.

[2]  Honghai Liu,et al.  Ultrasound-Based Sensing Models for Finger Motion Classification , 2018, IEEE Journal of Biomedical and Health Informatics.

[3]  Jing-Yi Guo,et al.  Performances of one-dimensional sonomyography and surface electromyography in tracking guided patterns of wrist extension. , 2009, Ultrasound in medicine & biology.

[4]  Jana Kosecka,et al.  Real-time, ultrasound-based control of a virtual hand by a trans-radial amputee , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[5]  Claudio Castellini,et al.  Ultrasound image features of the wrist are linearly related to finger positions , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  A. Nitz,et al.  Measurement of lumbar multifidus muscle contraction with rehabilitative ultrasound imaging. , 2007, Manual therapy.

[7]  Xiangyang Zhu,et al.  Combining Improved Gray-Level Co-Occurrence Matrix With High Density Grid for Myoelectric Control Robustness to Electrode Shift , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Levi J. Hargrove,et al.  A Comparison of Surface and Intramuscular Myoelectric Signal Classification , 2007, IEEE Transactions on Biomedical Engineering.

[9]  George N. Saridis,et al.  EMG Pattern Analysis and Classification for a Prosthetic Arm , 1982, IEEE Transactions on Biomedical Engineering.

[10]  Jana Kosecka,et al.  Real-time, ultrasound-based control of a virtual hand by a trans-radial amputee. , 2016, Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference.

[11]  T. Kuiken,et al.  Quantifying Pattern Recognition—Based Myoelectric Control of Multifunctional Transradial Prostheses , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  Xinjun Sheng,et al.  Toward an Enhanced Human–Machine Interface for Upper-Limb Prosthesis Control With Combined EMG and NIRS Signals , 2017, IEEE Transactions on Human-Machine Systems.

[13]  Thomas Schmitz-Rode,et al.  Surface electromyography and muscle force: limits in sEMG-force relationship and new approaches for applications. , 2009, Clinical biomechanics.

[14]  Yuefeng Li,et al.  Human-machine interface based on multi-channel single-element ultrasound transducers: A preliminary study , 2016, 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom).

[15]  Y. Zheng,et al.  Assessment of muscle fatigue using sonomyography: muscle thickness change detected from ultrasound images. , 2007, Medical engineering & physics.

[16]  Brian Waryck,et al.  Comparison Of Two Myoelectric Multi-Articulating Prosthetic Hands , 2011 .

[17]  S. Gandevia,et al.  Measurement of muscle contraction with ultrasound imaging , 2003, Muscle & nerve.

[18]  Richard F. Weir,et al.  A Comparison of the Effects of Electrode Implantation and Targeting on Pattern Classification Accuracy for Prosthesis Control , 2008, IEEE Transactions on Biomedical Engineering.

[19]  Honghai Liu,et al.  Multi-Modal Sensing Techniques for Interfacing Hand Prostheses: A Review , 2015, IEEE Sensors Journal.

[20]  Xiangyang Zhu,et al.  A Multichannel Surface EMG System for Hand Motion Recognition , 2015, Int. J. Humanoid Robotics.

[21]  Huosheng Hu,et al.  Support Vector Machine-Based Classification Scheme for Myoelectric Control Applied to Upper Limb , 2008, IEEE Transactions on Biomedical Engineering.

[22]  Xinjun Sheng,et al.  Cascaded Adaptation Framework for Fast Calibration of Myoelectric Control , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Jørgen Arendt Jensen,et al.  Ultrasound Imaging and Its Modeling , 2002 .

[24]  Xiao Liu,et al.  Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset , 2016, Neurocomputing.

[25]  Claudio Castellini,et al.  A realistic implementation of ultrasound imaging as a human-machine interface for upper-limb amputees , 2013, Front. Neurorobot..

[26]  Levi J. Hargrove,et al.  A training strategy to reduce classification degradation due to electrode displacements in pattern recognition based myoelectric control , 2008, Biomed. Signal Process. Control..

[27]  Ravinder Agarwal,et al.  Study of issues in the development of surface EMG controlled human hand , 2009, Journal of materials science. Materials in medicine.

[28]  D. Stegeman,et al.  Multichannel surface EMG: Basic aspects and clinical utility , 2003, Muscle & nerve.

[29]  Shihui Ying,et al.  Multimodal Neuroimaging Feature Learning With Multimodal Stacked Deep Polynomial Networks for Diagnosis of Alzheimer's Disease , 2018, IEEE Journal of Biomedical and Health Informatics.

[30]  Jing-Yi Guo,et al.  Recognition of Finger Flexion from Ultrasound Image with Optical Flow: A Preliminary Study , 2010, 2010 International Conference on Biomedical Engineering and Computer Science.

[31]  Qinghua Huang,et al.  Continuous Monitoring of Sonomyography, Electromyography and Torque Generated by Normal Upper Arm Muscles During Isometric Contraction: Sonomyography Assessment for Arm Muscles , 2008, IEEE Transactions on Biomedical Engineering.

[32]  Dapeng Yang,et al.  Experimental Study of an EMG-Controlled 5-DOF Anthropomorphic Prosthetic Hand for Motion Restoration , 2014, J. Intell. Robotic Syst..

[33]  E. Biddiss,et al.  Upper limb prosthesis use and abandonment: A survey of the last 25 years , 2007, Prosthetics and orthotics international.

[34]  Dario Farina,et al.  EMG-based simultaneous and proportional estimation of wrist/hand kinematics in uni-lateral trans-radial amputees , 2011, Journal of NeuroEngineering and Rehabilitation.

[35]  Todd A. Kuiken,et al.  The Effects of Electrode Size and Orientation on the Sensitivity of Myoelectric Pattern Recognition Systems to Electrode Shift , 2011, IEEE Transactions on Biomedical Engineering.

[36]  Todd A. Kuiken,et al.  Improving Myoelectric Pattern Recognition Robustness to Electrode Shift by Changing Interelectrode Distance and Electrode Configuration , 2012, IEEE Transactions on Biomedical Engineering.

[37]  M. Swiontkowski.  Targeted Muscle Reinnervation for Real-time Myoelectric Control of Multifunction Artificial Arms , 2010 .

[38]  Honghai Liu,et al.  A New Wearable Ultrasound Muscle Activity Sensing System for Dexterous Prosthetic Control , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[39]  Thomas L. Szabo,et al.  Diagnostic Ultrasound Imaging: Inside Out , 2004 .

[40]  Stefano Stramigioli,et al.  Myoelectric forearm prostheses: state of the art from a user-centered perspective. , 2011, Journal of rehabilitation research and development.

[41]  Huzefa Rangwala,et al.  Novel Method for Predicting Dexterous Individual Finger Movements by Imaging Muscle Activity Using a Wearable Ultrasonic System , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[42]  Yinfeng Fang,et al.  Interface Prostheses With Classifier-Feedback-Based User Training , 2017, IEEE Transactions on Biomedical Engineering.

[43]  C. J. Luca,et al.  SURFACE ELECTROMYOGRAPHY : DETECTION AND RECORDING , 2022 .

[44]  Zhi-Hong Mao,et al.  Limitations of Surface EMG Signals of Extrinsic Muscles in Predicting Postures of Human Hand , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[45]  Guanglin Li,et al.  Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees , 2012, Journal of NeuroEngineering and Rehabilitation.

[46]  C. Castellini,et al.  Using Ultrasound Images of the Forearm to Predict Finger Positions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[47]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[48]  Jing-Yi Guo,et al.  Recognition of finger flexion motion from ultrasound image: a feasibility study. , 2012, Ultrasound in medicine & biology.

[49]  Daniel R Merrill,et al.  Development of an implantable myoelectric sensor for advanced prosthesis control. , 2011, Artificial organs.

[50]  Jana Kosecka,et al.  Real-Time Classification of Hand Motions Using Ultrasound Imaging of Forearm Muscles , 2016, IEEE Transactions on Biomedical Engineering.

[51]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[52]  Hong Liu,et al.  Classification of Multiple Finger Motions During Dynamic Upper Limb Movements , 2017, IEEE Journal of Biomedical and Health Informatics.