Recognition of sketching from surface electromyography

The main objective of this study is to recognize sketching precisely and effectively in human–computer interaction. A surface electromyography (sEMG)-based sketching recognition method is proposed. We conducted an experiment in which we recorded the sEMG signals from the forearm muscles of two participants who were instructed to sketch seven basic one-stroke shapes. Subsequently, seven features of the sEMG time domain were extracted. After reducing data dimensionality using principal component analysis, these features were used as input vectors to a sketching recognition model based on support vector machines (SVMs). The performance of this model was compared to two other recognition models based on multilayer perceptron (MLP) neural networks and self-organization feature map (SOFM) neural networks. The average recognition rates for the seven basic one-stroke shapes of two participants achieved by the SVM-based and MLP-based models were both 98.5% in the test set, which were slightly superior to the performance of the SOFM classifier. Our results demonstrate the feasibility of converting forearm sEMG signals into sketching patterns.

[1]  Beryl Plimmer,et al.  Automatic evaluation of sketch recognizers , 2009, SBIM '09.

[2]  Othman Omran Khalifa,et al.  EMG signal classification for human computer interaction: a review , 2009 .

[3]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[4]  Yang Li.  Incremental Sketch Understanding for Intention Extraction in Sketch-based User Interfaces , 2003 .

[5]  Desney S. Tan,et al.  Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces , 2008, CHI.

[6]  Elisabeth André,et al.  EMG-based hand gesture recognition for realtime biosignal interfacing , 2008, IUI '08.

[7]  Ellen Yi-Luen Do,et al.  Ambiguous intentions: a paper-like interface for creative design , 1996, UIST '96.

[8]  Ian W. McKeague,et al.  A Statistical Model for Signature Verification , 2005 .

[9]  Kongqiao Wang,et al.  Hand Gesture Recognition Research Based on Surface EMG Sensors and 2D-accelerometers , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[10]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[11]  I. Cosic,et al.  Strategies to identify changes in SEMG due to muscle fatigue during cycling , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[12]  Volkan Y. Senyurek,et al.  Performance Comparison of Artificial Neural Network and Gaussian Mixture Model in Classifying Hand Motions by Using sEMG Signals , 2013 .

[13]  Guangyou Xu,et al.  Subject-independent natural action recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[14]  Mikhail A. Lebedev,et al.  Recognition of Handwriting from Electromyography , 2009, PloS one.

[15]  Lingyun Sun,et al.  Creative Segment: A descriptive theory applied to computer-aided sketching , 2014 .

[16]  Sougata Karmakar,et al.  Muscle Computer Interface: A Review , 2013 .

[17]  David Gur,et al.  Automated freehand sketch segmentation using radial basis functions , 2009, Comput. Aided Des..

[18]  N. Georganas,et al.  Human emotion recognition from body language of the head using soft computing techniques , 2012 .

[19]  Mehdi Nikoo,et al.  Determination of compressive strength of concrete using Self Organization Feature Map (SOFM) , 2013, Engineering with Computers.

[20]  James A. Landay,et al.  Sketching Interfaces: Toward More Human Interface Design , 2001, Computer.

[21]  Masanori Goka,et al.  Constructive simulation of creative concept generation process in design: a research method for difficult-to-observe design-thinking processes , 2012 .

[22]  Shaogang Gong,et al.  Free-hand sketch recognition by multi-kernel feature learning , 2015, Comput. Vis. Image Underst..

[23]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[24]  Jiangping Wang,et al.  Eyebrow emotional expression recognition using surface EMG signals , 2015, Neurocomputing.

[25]  V. Goel.  Sketches of thought , 1995 .

[26]  Abdulhamit Subasi,et al.  Muscle Fatigue Detection in EMG Using Time–Frequency Methods, ICA and Neural Networks , 2009, Journal of Medical Systems.

[27]  Yu Chen,et al.  Sketching in the air: A vision-based system for 3D object design , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Mehdi Nikoo,et al.  Principal Component Analysis combined with a Self Organization Feature Map to determine the pull-off adhesion between concrete layers , 2015 .

[29]  Euljoon Park,et al.  Adaptive filtering of the electromyographic signal for prosthetic control and force estimation , 1995, IEEE Transactions on Biomedical Engineering.

[30]  M. W Gardner,et al.  Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences , 1998 .

[31]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[32]  Mehdi Nikoo,et al.  Corrosion current density prediction in reinforced concrete by imperialist competitive algorithm , 2014, Neural Computing and Applications.

[33]  Lin Wu,et al.  Bead geometry prediction for robotic GMAW-based rapid manufacturing through a neural network and a second-order regression analysis , 2012, Journal of Intelligent Manufacturing.

[34]  A. Lansari,et al.  A novel electromyography (EMG) based classification approach for Arabic handwriting , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[35]  N. Shoylev,et al.  Neural Networks for Online Classification of Hand and Finger Movements Using Surface EMG signals , 2006, 2006 8th Seminar on Neural Network Applications in Electrical Engineering.

[36]  Lin Yao,et al.  Improvements on EMG-based handwriting recognition with DTW algorithm , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[37]  Ahmet Alkan,et al.  Classification of EEG Recordings by Using Fast Independent Component Analysis and Artificial Neural Network , 2008, Journal of Medical Systems.

[38]  Marimuthu Palaniswami,et al.  Multi run ICA and surface EMG based signal processing system for recognising hand gestures , 2008, 2008 8th IEEE International Conference on Computer and Information Technology.

[39]  Mario Costa Sousa,et al.  Sketch-based modeling: A survey , 2009, Comput. Graph..

[40]  Xin Wang,et al.  Human emotion and cognition recognition from body language of the head using soft computing techniques , 2012, Journal of Ambient Intelligence and Humanized Computing.

[41]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[42]  Thomas Schmitz-Rode,et al.  Surface electromyography and muscle force: limits in sEMG-force relationship and new approaches for applications. , 2009, Clinical biomechanics.

[43]  J. Potvin.  Effects of muscle kinematics on surface EMG amplitude and frequency during fatiguing dynamic contractions. , 1997, Journal of applied physiology.

[44]  Donald A. Sch.  The reflective practitioner: how professionals think in action , 1983 .

[45]  Qi-yi Tang,et al.  Data Processing System (DPS) software with experimental design, statistical analysis and data mining developed for use in entomological research , 2013, Insect science.

[46]  Elif Derya Übeyli.  Combined neural network model employing wavelet coefficients for EEG signals classification , 2009, Digit. Signal Process..

[47]  Jianping Wang,et al.  Surface EMG based handgrip force predictions using gene expression programming , 2016, Neurocomputing.

[48]  Carlo J. De Luca,et al.  The Use of Surface Electromyography in Biomechanics , 1997 .

[49]  Jianping Wang,et al.  Physiological and subjective responses to breathing resistance of N95 filtering facepiece respirators in still-sitting and walking , 2016 .

[50]  Hiroshi Yokoi,et al.  Development of a multi-DOF electromyography prosthetic system using the adaptive joint mechanism , 2006 .

[51]  Kejun Zhang,et al.  Web music emotion recognition based on higher effective gene expression programming , 2013, Neurocomputing.

[52]  Elizaveta V Okorokova,et al.  Corrigendum: A dynamical model improves reconstruction of handwriting from multichannel electromyographic recordings , 2016, Front. Neurosci..

[53]  Dingguo Zhang,et al.  An EMG-based handwriting recognition through dynamic time warping , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[54]  Yoshiaki Hayashi,et al.  An EMG-Based Control for an Upper-Limb Power-Assist Exoskeleton Robot , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).