Recognition of Mexican Sign Language from Frames in Video Sequences

The development of vision systems capable to extracting discriminative features that enhance the generalization power of a classifier is still a very challenging problem. In this paper, is presented a methodology to improve the classification performance of Mexican Sign Language (MSL). The proposed method explores some frames in video sequences for each sign. 743 features were extracted from these frames, and a genetic algorithm is employed to select a subset of sensitive features by removing the irrelevant features. The genetic algorithm permits to obtain the most discriminative features. Support Vector Machines (SVM) are used to classify signs based on these features. The experiments show that the proposed method can be successfully used to recognize the MSL with accuracy results individually above 97 % on average. The proposed feature extraction methodology and the GA used to extract the most discriminative features is a promising method to facilitate the communication of deaf people.

[1]  José Sergio Ruiz Castilla,et al.  Color Characterization Comparison for Machine Vision-Based Fruit Recognition , 2015, ICIC.

[2]  Wen Gao,et al.  Large vocabulary sign language recognition based on fuzzy decision trees , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[3]  Jacques de Villiers,et al.  New tools for interactive speech and language training: Using animated conversational agents in the classrooms of profoundly deaf children , 1999 .

[4]  Khaled Assaleh,et al.  Video-based signer-independent Arabic sign language recognition using hidden Markov models , 2009, Appl. Soft Comput..

[5]  Javier Macías Guarasa,et al.  Speech to sign language translation system for Spanish , 2008, Speech Commun..

[6]  Ronald A. Cole,et al.  Perceptive animated interfaces: first steps toward a new paradigm for human-computer interaction , 2003, Proc. IEEE.

[7]  Peter Vamplew Recognition of sign language gestures using neural networks , 1996 .

[8]  Jair Cervantes,et al.  Clasificación del lenguaje de señas mexicano con SVM generando datos artificiales Classification of Mexican Sign Language SVM generating data , 2013 .

[9]  Hermann Ney,et al.  Spoken language processing techniques for sign language recognition and translation , 2008, Technology and Disability.

[10]  Hermann Ney,et al.  Geometric Features for Improving Continuous Appearance-based Sign Language Recognition , 2006, BMVC.

[11]  Antonio García,et al.  Spoken Spanish generation from sign language , 2010, Interact. Comput..

[12]  Kirsti Grobel,et al.  Video-Based Sign Language Recognition Using Hidden Markov Models , 1997, Gesture Workshop.

[13]  Hermann Hienz,et al.  Relevant features for video-based continuous sign language recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).