Intelligent Human–Computer Interaction: Combined Wrist and Forearm Myoelectric Signals for Handwriting Recognition

Recent studies have highlighted the possibility of using surface electromyographic (EMG) signals to develop human–computer interfaces that are also able to recognize complex motor tasks involving the hand as the handwriting of digits. However, the automatic recognition of words from EMG information has not yet been studied. The aim of this study is to investigate the feasibility of using combined forearm and wrist EMG probes for solving the handwriting recognition problem of 30 words with consolidated machine-learning techniques and aggregating state-of-the-art features extracted in the time and frequency domains. Six healthy subjects, three females and three males aged between 25 and 40 years, were recruited for the study. Two tests in pattern recognition were conducted to assess the possibility of classifying fine hand movements through EMG signals. The first test was designed to assess the feasibility of using consolidated myoelectric control technology with shallow machine-learning methods in the field of handwriting detection. The second test was implemented to assess if specific feature extraction schemes can guarantee high performances with limited complexity of the processing pipeline. Among support vector machine, linear discriminant analysis, and K-nearest neighbours (KNN), the last one showed the best classification performances in the 30-word classification problem, with a mean accuracy of 95% and 85% when using all the features and a specific feature set known as TDAR, respectively. The obtained results confirmed the validity of using combined wrist and forearm EMG data for intelligent handwriting recognition through pattern recognition approaches in real scenarios.

[1]  David Sussillo,et al.  A generic noninvasive neuromotor interface for human-computer interaction , 2024, bioRxiv.

[2]  S. Fioretti,et al.  On the Decoding of Shoulder Joint Intent of Motion From Transient EMG: Feature Evaluation and Classification , 2023, IEEE Transactions on Medical Robotics and Bionics.

[3]  Shashank Kumar Singh,et al.  Leveraging deep feature learning for wearable sensors based handwritten character recognition , 2023, Biomed. Signal Process. Control..

[4]  G. Murakami,et al.  Distally-extending muscle fibers across involved joints: study of long muscles and tendons of wrist and ankle in late-term fetuses and adult cadavers , 2022, Anatomy & cell biology.

[5]  S. Fioretti,et al.  Identification of Neurodegenerative Diseases From Gait Rhythm Through Time Domain and Time-Dependent Spectral Descriptors , 2022, IEEE Journal of Biomedical and Health Informatics.

[6]  Erik Scheme,et al.  Electromyography-Based Gesture Recognition: Is It Time to Change Focus From the Forearm to the Wrist? , 2022, IEEE Transactions on Industrial Informatics.

[7]  Kianoush Nazarpour,et al.  Decoding HD-EMG Signals for Myoelectric Control - How Small Can the Analysis Window Size be? , 2021, IEEE Robotics and Automation Letters.

[8]  Federica Verdini,et al.  Improving EMG Signal Change Point Detection for Low SNR by Using Extended Teager-Kaiser Energy Operator , 2020, IEEE Transactions on Medical Robotics and Bionics.

[9]  Jose Ruiz-Pinales,et al.  Multi-Stroke handwriting character recognition based on sEMG using convolutional-recurrent neural networks. , 2020, Mathematical biosciences and engineering : MBE.

[10]  Reza Langari,et al.  A Multi-Window Majority Voting Strategy to Improve Hand Gesture Recognition Accuracies Using Electromyography Signal , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  Eduardo Palermo,et al.  Automatic Detection of Faults in Race Walking: A Comparative Analysis of Machine-Learning Algorithms Fed with Inertial Sensor Data , 2019, Sensors.

[12]  H. Kao,et al.  Chinese calligraphy handwriting (CCH): a case of rehabilitative awakening of a coma patient after stroke , 2018, Neuropsychiatric disease and treatment.

[13]  Wim Vandenberghe,et al.  Handwriting training in Parkinson’s disease: A trade-off between size, speed and fluency , 2017, PloS one.

[14]  Marcos Faúndez-Zanuy,et al.  Evaluation of handwriting kinematics and pressure for differential diagnosis of Parkinson's disease , 2016, Artif. Intell. Medicine.

[15]  Elizaveta V Okorokova,et al.  A dynamical model improves reconstruction of handwriting from multichannel electromyographic recordings , 2015, Front. Neurosci..

[16]  F. Micheli,et al.  Handwriting Rehabilitation in Parkinson Disease: A Pilot Study , 2015, Annals of rehabilitation medicine.

[17]  Jean-Luc Velay,et al.  Basic and supplementary sensory feedback in handwriting , 2015, Front. Psychol..

[18]  R. Khushaba Correlation Analysis of Electromyogram Signals for Multiuser Myoelectric Interfaces , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  R. Kronland-Martinet,et al.  Handwriting Movement Sonification for the Rehabilitation of Dysgraphia , 2013 .

[20]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[21]  Lin Yao,et al.  Improvements on EMG-based handwriting recognition with DTW algorithm , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[22]  Gamini Dissanayake,et al.  Toward improved control of prosthetic fingers using surface electromyogram (EMG) signals , 2012, Expert Syst. Appl..

[23]  Songthip Ounpraseuth,et al.  Micrographia and related deficits in Parkinson's disease: a cross-sectional study , 2012, BMJ Open.

[24]  Annie McCluskey,et al.  A review of factors that influence adult handwriting performance. , 2011, Australian occupational therapy journal.

[25]  Dario Farina,et al.  Influence of the training set on the accuracy of surface EMG classification in dynamic contractions for the control of multifunction prostheses , 2011, Journal of NeuroEngineering and Rehabilitation.

[26]  M. Pąchalska,et al.  Early neurorehabilitation in a patient with severe traumatic brain injury to the frontal lobes. , 2010, Medical science monitor : international medical journal of experimental and clinical research.

[27]  Dingguo Zhang,et al.  An EMG-based handwriting recognition through dynamic time warping , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[28]  S Micera,et al.  Control of Hand Prostheses Using Peripheral Information , 2010, IEEE Reviews in Biomedical Engineering.

[29]  Chia-Hung Lin,et al.  Portable hand motion classifier for multi-channel surface electromyography recognition using grey relational analysis , 2010, Expert Syst. Appl..

[30]  H. Kao Calligraphy therapy: A complementary approach to psychotherapy , 2010 .

[31]  Mikhail A. Lebedev,et al.  Recognition of Handwriting from Electromyography , 2009, PloS one.

[32]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[33]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[34]  L. Burattini,et al.  Handwritten Digits Recognition From sEMG: Electrodes Location and Feature Selection , 2023, IEEE Access.

[35]  E. Clancy,et al.  Measuring Neuromuscular Electrophysiological Activities to Decode HD-sEMG Biometrics for Cross-Application Discrepant Personal Identification With Unknown Identities , 2022, IEEE Transactions on Instrumentation and Measurement.

[36]  Fady S. Botros,et al.  Day-to-Day Stability of Wrist EMG for Wearable-Based Hand Gesture Recognition , 2022, IEEE Access.

[37]  K. Raoof,et al.  Hand Gesture Recognition With Acoustic Myography and Wavelet Scattering Transform , 2022, IEEE Access.

[38]  Wei-Chun Hsu,et al.  Surface EMG vs. High-Density EMG: Tradeoff Between Performance and Usability for Head Orientation Prediction in VR Application , 2021, IEEE Access.

[39]  Minas Liarokapis,et al.  Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality , 2021, IEEE Access.

[40]  Giuseppe Pirlo,et al.  Dynamic Handwriting Analysis for the Assessment of Neurodegenerative Diseases: A Pattern Recognition Perspective , 2019, IEEE Reviews in Biomedical Engineering.

[41]  Hong-Bo Xie,et al.  Ant colony optimization-based feature selection method for surface electromyography signals classification , 2012, Comput. Biol. Medicine.

[42]  Todd R Farrell,et al.  Determining delay created by multifunctional prosthesis controllers. , 2011, Journal of rehabilitation research and development.

[43]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[44]  E. van Functional relationship between the abductor pollicis longus and abductor pollicis brevis muscles : an EMG analysis , 2022 .