Signer-independent classification of American sign language word signs using surface EMG

The field of Sign Language Recognition (SLR) has become an increasingly popular research topic. The goal of this study is an SLR system that will be capable of identifying a subset of 50 of the most common American Sign Language (ASL) word signs using surface electromyography and accelerometer data for multiple signers. All data was collected from deaf, fluent ASL users. A windowing approach is used with different time domain features for feature extraction. The samples are divided into one and two-handed signs, each of which are used to train a Support Vector Machine classifier. Samples from all but one subject are used to train the classifiers. The classifiers are then tested on both data held out from the subjects used for training and the subject that was left out. The resulting system had an average accuracy of 59.96% for trained subjects and 33.66% for the subject left out. To compare this approach to others, 40-word and 7-word sign sets are trained and tested using this method. The proposed system performed comparably with literature for the 40-word set, and better for the 7-word set.

[1]  Lale Akarun,et al.  A multi-class classification strategy for Fisher scores: Application to signer independent sign language recognition , 2010, Pattern Recognit..

[2]  Ajat Shatru Arora,et al.  Multi-Class SVM Classification of Surface EMG Signal for Upper Limb Function , 2009, 2009 3rd International Conference on Bioinformatics and Biomedical Engineering.

[3]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[4]  Xu Zhang,et al.  Random Forest-Based Recognition of Isolated Sign Language Subwords Using Data from Accelerometers and Surface Electromyographic Sensors , 2016, Sensors.

[5]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[6]  Jun Morimoto,et al.  Bilinear Modeling of EMG Signals to Extract User-Independent Features for Multiuser Myoelectric Interface , 2013, IEEE Transactions on Biomedical Engineering.

[7]  Roozbeh Jafari,et al.  Real-time American Sign Language Recognition using wrist-worn motion and surface EMG sensors , 2015, 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[8]  Kongqiao Wang,et al.  Automatic recognition of sign language subwords based on portable accelerometer and EMG sensors , 2010, ICMI-MLMI '10.

[9]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[10]  Ferat Sahin,et al.  American Sign Language Recognition system by using surface EMG signal , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[11]  Changsheng Xu,et al.  Latent support vector machine for sign language recognition with Kinect , 2013, 2013 IEEE International Conference on Image Processing.

[12]  Vasiliki Kosmidou,et al.  Using sample entropy for automated sign language recognition on sEMG and accelerometer data , 2010, Medical & Biological Engineering & Computing.

[13]  Ferat Sahin,et al.  Real-Time American Sign Language Recognition System Using Surface EMG Signal , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[14]  Kirsti Grobel,et al.  Video-Based Sign Language Recognition Using Hidden Markov Models , 1997, Gesture Workshop.

[15]  Phornphop Naiyanetr,et al.  Sign language-Thai alphabet conversion based on Electromyogram (EMG) , 2014, The 7th 2014 Biomedical Engineering International Conference.

[16]  Wen Gao,et al.  Signer-independent sign language recognition based on SOFM/HMM , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[17]  Johannes Wagner,et al.  Bi-channel sensor fusion for automatic sign language recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[18]  R. Mitchell,et al.  How Many People Use ASL in the United States? Why Estimates Need Updating , 2006 .

[19]  Mikhail Kuznetsov,et al.  Filtering the surface EMG signal: Movement artifact and baseline noise contamination. , 2010, Journal of biomechanics.

[20]  Ferat Sahin,et al.  A study of recent classification algorithms and a novel approach for EEG data classification , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[21]  Khaled Assaleh,et al.  Video-based signer-independent Arabic sign language recognition using hidden Markov models , 2009, Appl. Soft Comput..