Sign Language Recognition Using Intrinsic-Mode Sample Entropy on sEMG and Accelerometer Data

Sign language forms a communication channel among the deaf; however, automated gesture recognition could further expand their communication with the hearers. In this work, data from five-channel surface electromyogram and 3-D accelerometer from the signer's dominant hand were analyzed using intrinsic-mode entropy (IMEn) for the automated recognition of Greek sign language (GSL) isolated signs. Discriminant analysis was used to identify the effective scales of the intrinsic-mode functions and the window length for the calculation of the IMEn that contributes to the efficient classification of the GSL signs. Experimental results from the IMEn analysis applied to GSL signs corresponding to 60-word lexicon repeated ten times by three native signers have shown more than 93% mean classification accuracy using IMEn as the only source of the classification feature set. This provides a promising bed-set toward the automated GSL gesture recognition.

[1]  D. Ruelle,et al.  Ergodic theory of chaos and strange attractors , 1985 .

[2]  Trevor Darrell,et al.  Hidden Conditional Random Fields for Gesture Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Dimitris N. Metaxas,et al.  Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes , 1999, Gesture Workshop.

[4]  P. N. Vassilia,et al.  Towards an assistive tool for Greek sign language communication , 2003, Proceedings 3rd IEEE International Conference on Advanced Technologies.

[5]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[6]  Roberto Cipolla,et al.  Real-time Interpretation of Hand Motions using a Sparse Bayesian Classifier on Motion Gradient Orientation Images , 2005, BMVC.

[7]  Paul Lukowicz,et al.  Using multiple sensors for mobile sign language recognition , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[8]  Geoffrey E. Hinton,et al.  Glove-TalkII: Mapping Hand Gestures to Speech Using Neural Networks , 1994, NIPS.

[9]  David J. Hewson,et al.  Intrinsic Mode Entropy for Nonlinear Discriminant Analysis , 2007, IEEE Signal Processing Letters.

[10]  R.Fff. Weir,et al.  A heuristic fuzzy logic approach to EMG pattern recognition for multifunctional prosthesis control , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  Mohammed Waleed Kadous,et al.  Machine Recognition of Auslan Signs Using PowerGloves: Towards Large-Lexicon Recognition of Sign Lan , 1996 .

[12]  George N. Saridis,et al.  EMG Pattern Analysis and Classification for a Prosthetic Arm , 1982, IEEE Transactions on Biomedical Engineering.

[13]  Wojtek J. Krzanowski,et al.  Principles of multivariate analysis : a user's perspective. oxford , 1988 .

[14]  Leontios J. Hadjileontiadis,et al.  Intrinsic mode entropy: An enhanced classification means for automated Greek Sign Language gesture recognition , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[15]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[16]  Robert Shaw.  Strange Attractors, Chaotic Behavior, and Information Flow , 1981 .

[17]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[18]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[19]  Wen Gao,et al.  Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Qing Chen,et al.  Hand Gesture Recognition Using Haar-Like Features and a Stochastic Context-Free Grammar , 2008, IEEE Transactions on Instrumentation and Measurement.

[21]  Sang-Won Lee,et al.  Hand Gesture Recognition System Using Fuzzy Algorithm and RDBMS for Post PC , 2005, FSKD.

[22]  Surendra Ranganath,et al.  Real-time gesture recognition system and application , 2002, Image Vis. Comput..

[23]  Wen Gao,et al.  A continuous Chinese sign language recognition system , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[24]  A E Marble,et al.  Image processing system for interpreting motion in American Sign Language. , 1992, Journal of biomedical engineering.

[25]  Alan Wexelblat,et al.  A feature-based approach to continuous-gesture analysis , 1994 .

[26]  Robert W. Lindeman,et al.  A new instrumented approach for translating American Sign Language into sound and text , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[27]  Richard Bowden,et al.  Large Lexicon Detection of Sign Language , 2007, ICCV-HCI.

[28]  J. Richman,et al.  Physiological time-series analysis using approximate entropy and sample entropy. , 2000, American journal of physiology. Heart and circulatory physiology.

[29]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[30]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[31]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[32]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[33]  L.J. Hadjileontiadis,et al.  Evaluation of surface EMG features for the recognition of American Sign Language gestures , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[34]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Richard A. Tennant,et al.  The American Sign Language Handshape Dictionary , 1998 .

[36]  M. Vuskovic,et al.  Mahalanobis distance-based ARTMAP network , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[37]  S M Pincus,et al.  Approximate entropy as a measure of system complexity. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[38]  Tomoichi Takahashi,et al.  Hand gesture coding based on experiments using a hand gesture interface device , 1991, SGCH.

[39]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[40]  Dimitris N. Metaxas,et al.  Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.