Evaluation of Finger Flexion Classification at Reduced Lateral Spatial Resolutions of Ultrasound

The objective of this paper is to investigate the effect that lateral spatial resolution of ultrasound has on finger flexion classification. The objective’s purpose leads toward the development of a wearable human machine interface (HMI) using single-element ultrasonic sensors with non-focused ultrasound. Ultrasound radiofrequency (RF) signals were acquired using a linear array ultrasound probe in B-mode while performing the individual finger flexions from five healthy volunteers. Each B-mode frame is composed of 127 parallel ultrasound RF signals along the lateral direction within a 40-mm width. To reduce the lateral resolution of ultrasound data artificially, the RF signals were averaged into a reduced number of lateral columns. Across ten independent arm experiments the classification accuracy at 127 channels (full resolution) resulted in the first and third quartile to be 80–92%. Averaging into four RF signals (simulating 10-mm wide ultrasound beams from each channel) could achieve a median classification accuracy of 87% using the proposed feature extraction method with the discrete wavelet transform. Our results show low resolutions could achieve high accuracies to that of full resolution. We also conducted a preliminary study using a multichannel single-element ultrasound system with lightweight, flexible, and wearable ultrasonic sensors (WUSs) using non-focused ultrasound. Each WUS had an ultrasound sensing area of 20mm by 20mm. Three WUSs were attached on one subject’s forearm and ultrasound RF signals were acquired during individual finger flexions. A mean classification accuracy of 98% was obtained with F1 scores ranging between 95 – 98% (across five finger flexions).

[1]  R.F. Weir,et al.  The Optimal Controller Delay for Myoelectric Prostheses , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Aaron O'Leary,et al.  PyWavelets: A Python package for wavelet analysis , 2019, J. Open Source Softw..

[3]  Mike Fraser,et al.  EchoFlex: Hand Gesture Recognition using Ultrasound Imaging , 2017, CHI.

[4]  Jing-Yi Guo,et al.  Recognition of finger flexion motion from ultrasound image: a feasibility study. , 2012, Ultrasound in medicine & biology.

[5]  Claudio Castellini,et al.  A realistic implementation of ultrasound imaging as a human-machine interface for upper-limb amputees , 2013, Front. Neurorobot..

[6]  Honghai Liu,et al.  A Lightweight Ultrasound Probe for Wearable Human–Machine Interfaces , 2019, IEEE Sensors Journal.

[7]  Yatindra Kumar,et al.  Feature extraction and classification for EMG signals using linear discriminant analysis , 2016, 2016 2nd International Conference on Advances in Computing, Communication, & Automation (ICACCA) (Fall).

[8]  Honghai Liu,et al.  A New Wearable Ultrasound Muscle Activity Sensing System for Dexterous Prosthetic Control , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[9]  Claudio Castellini,et al.  State of the Art and Perspectives of Ultrasound Imaging as a Human-Machine Interface , 2014 .

[10]  Claudio Castellini,et al.  Ultrasound image features of the wrist are linearly related to finger positions , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Ning Jiang,et al.  Wrist and Finger Gesture Recognition With Single-Element Ultrasound Signals: A Comparison With Single-Channel Surface Electromyogram , 2019, IEEE Transactions on Biomedical Engineering.

[12]  Effect of Lateral Resolution on Classifying Individual Finger Flexions using Ultrasound , 2019 .

[13]  P. Gallagher,et al.  Environmental barriers, activity limitations and participation restrictions experienced by people with major limb amputation , 2011, Prosthetics and orthotics international.

[14]  Y. Zheng,et al.  Sonomyography: monitoring morphological changes of forearm muscles in actions with the feasibility for the control of powered prosthesis. , 2006, Medical engineering & physics.

[15]  Panagiotis Artemiadis,et al.  Neuro-Robotics : From Brain Machine Interfaces to Rehabilitation Robotics , 2014 .

[16]  Can Eyupoglu,et al.  A cepstrum analysis-based classification method for hand movement surface EMG signals , 2019, Medical & Biological Engineering & Computing.

[17]  Jing-Yi Guo,et al.  Recognition of Finger Flexion from Ultrasound Image with Optical Flow: A Preliminary Study , 2010, 2010 International Conference on Biomedical Engineering and Computer Science.

[18]  Ibrahim AlMohimeed,et al.  Development of wearable and flexible ultrasonic sensor for skeletal muscle monitoring , 2013, 2013 IEEE International Ultrasonics Symposium (IUS).

[19]  Kathryn Ziegler-Graham,et al.  Estimating the prevalence of limb loss in the United States: 2005 to 2050. , 2008, Archives of physical medicine and rehabilitation.

[20]  Jana Kosecka,et al.  Real-time, ultrasound-based control of a virtual hand by a trans-radial amputee , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[21]  Christian Cipriani,et al.  Ultrasound imaging for hand prosthesis control: a comparative study of features and classification methods , 2015, 2015 IEEE International Conference on Rehabilitation Robotics (ICORR).

[22]  C. Castellini,et al.  Using Ultrasound Images of the Forearm to Predict Finger Positions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Geethanjali Purushothaman,et al.  Identification of a feature selection based pattern recognition scheme for finger movement recognition from multichannel EMG signals , 2018, Australasian Physical & Engineering Sciences in Medicine.

[24]  V. R. Thool,et al.  Wavelet transform based EMG feature extraction and evaluation using scatter graphs , 2015, 2015 International Conference on Industrial Instrumentation and Control (ICIC).

[25]  Jana Kosecka,et al.  Real-Time Classification of Hand Motions Using Ultrasound Imaging of Forearm Muscles , 2016, IEEE Transactions on Biomedical Engineering.

[26]  Honghai Liu,et al.  Ultrasound-Based Sensing Models for Finger Motion Classification , 2018, IEEE Journal of Biomedical and Health Informatics.

[27]  A. J. Fernandes,et al.  Flexible and Wearable Ultrasonic Sensors and Method for Classifying Individual Finger Flexions , 2020, 2020 IEEE International Conference on Flexible and Printable Sensors and Systems (FLEPS).

[28]  Siddhartha Sikdar,et al.  Evaluation of the Role of Proprioception During Proportional Position Control Using Sonomyography: Applications in Prosthetic Control , 2019, 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR).

[29]  Yinfeng Fang,et al.  Simultaneous Prediction of Wrist/Hand Motion via Wearable Ultrasound Sensing , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Stéphane Mallat,et al.  Multifrequency channel decompositions of images and wavelet models , 1989, IEEE Trans. Acoust. Speech Signal Process..

[31]  Ahmed Hamza Osman,et al.  A Novel Feature Selection Based on One-Way ANOVA F-Test for E-Mail Spam Classification , 2014 .

[32]  Max Ortiz-Catalan,et al.  Evaluation of surface EMG-based recognition algorithms for decoding hand movements , 2019, Medical & Biological Engineering & Computing.

[33]  Yuefeng Li,et al.  Human-machine interface based on multi-channel single-element ultrasound transducers: A preliminary study , 2016, 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom).

[34]  Ilja Kuzborskij,et al.  On the challenge of classifying 52 hand movements from surface electromyography , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[35]  Honghai Liu,et al.  Towards Wearable A-Mode Ultrasound Sensing for Real-Time Finger Motion Recognition , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[36]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[37]  Huzefa Rangwala,et al.  Novel Method for Predicting Dexterous Individual Finger Movements by Imaging Muscle Activity Using a Wearable Ultrasonic System , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[38]  Manas Agarwal,et al.  Wearable Ultrasonic Sensor Using Double-Layer PVDF Films for Monitoring Tissue Motion , 2018, 2018 IEEE Canadian Conference on Electrical & Computer Engineering (CCECE).

[39]  Rozaimi Ghazali,et al.  Feature extraction of surface electromyography (sEMG) and signal processing technique in wavelet transform: A review , 2016, 2016 IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS).