A Novel Hybrid Model for Drawing Trace Reconstruction from Multichannel Surface Electromyographic Activity

Recently, several researchers have considered the problem of reconstruction of handwriting and other meaningful arm and hand movements from surface electromyography (sEMG). Although much progress has been made, several practical limitations may still affect the clinical applicability of sEMG-based techniques. In this paper, a novel three-step hybrid model of coordinate state transition, sEMG feature extraction and gene expression programming (GEP) prediction is proposed for reconstructing drawing traces of 12 basic one-stroke shapes from multichannel surface electromyography. Using a specially designed coordinate data acquisition system, we recorded the coordinate data of drawing traces collected in accordance with the time series while 7-channel EMG signals were recorded. As a widely-used time domain feature, Root Mean Square (RMS) was extracted with the analysis window. The preliminary reconstruction models can be established by GEP. Then, the original drawing traces can be approximated by a constructed prediction model. Applying the three-step hybrid model, we were able to convert seven channels of EMG activity recorded from the arm muscles into smooth reconstructions of drawing traces. The hybrid model can yield a mean accuracy of 74% in within-group design (one set of prediction models for all shapes) and 86% in between-group design (one separate set of prediction models for each shape), averaged for the reconstructed x and y coordinates. It can be concluded that it is feasible for the proposed three-step hybrid model to improve the reconstruction ability of drawing traces from sEMG.

[1]  Kejun Zhang,et al.  An Upper-Limb Power-Assist Exoskeleton Using Proportional Myoelectric Control , 2014, Sensors.

[2]  Xinjun Sheng,et al.  User adaptation in long-term, open-loop myoelectric training: implications for EMG pattern recognition in prosthesis control , 2015, Journal of neural engineering.

[3]  Cândida Ferreira Gene Expression Programming in Problem Solving , 2002 .

[4]  Dario Farina,et al.  Simultaneous and Proportional Force Estimation for Multifunction Myoelectric Prostheses Using Mirrored Bilateral Training , 2011, IEEE Transactions on Biomedical Engineering.

[5]  Réjean Plamondon,et al.  A New Methodology to Improve Myoelectric Signal Processing Using Handwriting , 2008 .

[6]  I. J. Gelb A study of writing , 1954 .

[7]  James A. Landay,et al.  Sketching Interfaces: Toward More Human Interface Design , 2001, Computer.

[8]  A. Geurts,et al.  Definition dependent properties of the cortical silent period in upper-extremity muscles, a methodological study , 2014, Journal of NeuroEngineering and Rehabilitation.

[9]  Toshio Asano,et al.  Visual interface system by character handwriting gestures in the air , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[10]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[11]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[12]  Scott Robertson,et al.  How to Draw: Drawing and Sketching Objects and Environments from Your Imagination , 2013 .

[13]  Robert D. Lipschutz,et al.  Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. , 2009, JAMA.

[14]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[15]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Øyvind Stavdahl,et al.  System training and assessment in simultaneous proportional myoelectric prosthesis control , 2013, Journal of NeuroEngineering and Rehabilitation.

[17]  Dario Farina,et al.  EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis , 2015, Journal of NeuroEngineering and Rehabilitation.

[18]  Markus Nowak,et al.  Low-cost wearable multichannel surface EMG acquisition for prosthetic hand control , 2015, 2015 6th International Workshop on Advances in Sensors and Interfaces (IWASI).

[19]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[20]  Dingguo Zhang,et al.  An EMG-based handwriting recognition through dynamic time warping , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[21]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[22]  Cândida Ferreira,et al.  The Entities of Gene Expression Programming , 2006 .

[23]  Desney S. Tan,et al.  Introduction to the Special Issue on Physiological Computing for Human-Computer Interaction , 2015, ACM Trans. Comput. Hum. Interact..

[24]  Alvin W. Y. Su,et al.  EMG based rehabilitation systems - approaches for ALS patients in different stages , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[25]  Jianping Wang,et al.  Surface EMG based handgrip force predictions using gene expression programming , 2016, Neurocomputing.

[26]  Mikhail A. Lebedev,et al.  Recognition of Handwriting from Electromyography , 2009, PloS one.

[27]  Dario Farina,et al.  Influence of the training set on the accuracy of surface EMG classification in dynamic contractions for the control of multifunction prostheses , 2011, Journal of NeuroEngineering and Rehabilitation.

[28]  C. K. Battye,et al.  The use of myo-electric currents in the operation of prostheses. , 1955, The Journal of bone and joint surgery. British volume.

[29]  D. Farina,et al.  Spatial Correlation of High Density EMG Signals Provides Features Robust to Electrode Number and Shift in Pattern Recognition for Myocontrol , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  H. Nichols Surgical anatomy of the hand. , 1959, Clinical orthopaedics.

[31]  Huosheng Hu,et al.  Myoelectric control systems - A survey , 2007, Biomed. Signal Process. Control..

[32]  Aiping Liu,et al.  A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors , 2015, Sensors.

[33]  Mohamed Benrejeb,et al.  Parametric identification of handwriting system based on RLS algorithm , 2011, 2011 11th International Conference on Control, Automation and Systems.

[34]  G J Kemp,et al.  Electromyographic assessment of muscle fatigue in massive rotator cuff tear. , 2015, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[35]  C. Turati,et al.  Three-year-olds' rapid facial electromyographic responses to emotional facial expressions and body postures. , 2016, Journal of experimental child psychology.

[36]  T. Kuiken,et al.  Quantifying Pattern Recognition—Based Myoelectric Control of Multifunctional Transradial Prostheses , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  Cândida Ferreira,et al.  Gene Expression Programming: Mathematical Modeling by an Artificial Intelligence , 2014, Studies in Computational Intelligence.

[38]  Levent Burak Kara,et al.  An image-based, trainable symbol recognizer for hand-drawn sketches , 2005, Comput. Graph..

[39]  Lin Yao,et al.  Improvements on EMG-based handwriting recognition with DTW algorithm , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[40]  O. Kisi,et al.  Comparison of Gene Expression Programming with neuro-fuzzy and neural network computing techniques in estimating daily incoming solar radiation in the Basque Country (Northern Spain) , 2012 .

[41]  Sougata Karmakar,et al.  Muscle Computer Interface: A Review , 2013 .

[42]  Barbara Tversky,et al.  Visualizing Thought , 2011, Top. Cogn. Sci..

[43]  Ping Zhou,et al.  Complexity Analysis of Surface EMG for Overcoming ECG Interference toward Proportional Myoelectric Control , 2016, Entropy.

[44]  Todd A. Kuiken,et al.  Dual Window Pattern Recognition Classifier for Improved Partial-Hand Prosthesis Control , 2016, Front. Neurosci..

[45]  Jian Huang,et al.  A real-time EMG pattern recognition method for virtual myoelectric hand control , 2014, Neurocomputing.

[46]  M. Swiontkowski Targeted Muscle Reinnervation for Real-time Myoelectric Control of Multifunction Artificial Arms , 2010 .

[47]  Kevin Englehart,et al.  Multifunction control of prostheses using the myoelectric signal , 2001 .

[48]  Jianping Wang,et al.  Physiological and subjective responses to breathing resistance of N95 filtering facepiece respirators in still-sitting and walking , 2016 .

[49]  Weimin Xiao,et al.  Evolving accurate and compact classification rules with gene expression programming , 2003, IEEE Trans. Evol. Comput..

[50]  N. P. Reddy,et al.  Neural network committees for finger joint angle estimation from surface EMG signals , 2009, Biomedical engineering online.

[51]  Aaron J. Young,et al.  Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses , 2014, Journal of neural engineering.

[52]  V. Goel Sketches of thought , 1995 .

[53]  Desney S. Tan,et al.  Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces , 2008, CHI.

[54]  Levi J. Hargrove,et al.  A training strategy to reduce classification degradation due to electrode displacements in pattern recognition based myoelectric control , 2008, Biomed. Signal Process. Control..

[55]  Kejun Zhang,et al.  Web music emotion recognition based on higher effective gene expression programming , 2013, Neurocomputing.

[56]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[57]  Wen-Yo Lee,et al.  A Vision-Based Fingertip-Writing Character Recognition System , 2016 .

[58]  Elizaveta V Okorokova,et al.  A dynamical model improves reconstruction of handwriting from multichannel electromyographic recordings , 2015, Front. Neurosci..

[59]  J H Blok,et al.  Surface EMG models: properties and applications. , 2000, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[60]  Hongnian Yu,et al.  Impact of Load Variation on Joint Angle Estimation From Surface EMG Signals , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[61]  Jiangping Wang,et al.  Eyebrow emotional expression recognition using surface EMG signals , 2015, Neurocomputing.

[62]  Wenwei Yu,et al.  Hand motion reconstruction using EEG and EMG , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[63]  Zhongliang Yang,et al.  Surface EMG-based Sketching Recognition Using Two Analysis Windows and Gene Expression Programming , 2016, Front. Neurosci..

[64]  Shumin Zhai,et al.  SHARK2: a large vocabulary shorthand writing system for pen-based computers , 2004, UIST '04.

[65]  Jianda Han,et al.  Missing-Data Classification With the Extended Full-Dimensional Gaussian Mixture Model: Applications to EMG-Based Motion Recognition , 2015, IEEE Transactions on Industrial Electronics.

[66]  Marimuthu Palaniswami,et al.  Subtle Hand Gesture Identification for HCI Using Temporal Decorrelation Source Separation BSS of Surface EMG , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).