Persian sign language (PSL) recognition using wavelet transform and neural networks

This paper presents a system for recognizing static gestures of alphabets in Persian sign language (PSL) using Wavelet transform and neural networks (NN). The required images for the selected alphabets are obtained using a digital camera. The color images are cropped, resized, and converted to grayscale images. Then, the discrete wavelet transform (DWT) is applied on the gray scale images, and some features are extracted. Finally, the extracted features are used to train a Multi-Layered Perceptron (MLP) NN. Our recognition system does not use any gloves or visual marking systems. This system only requires the images of the bare hand for the recognition. The system is implemented and tested using a data set of 640 samples of Persian sign images; 20 images for each sign. Experimental results show that our system is able to recognize 32 selected PSL alphabets with an average classification accuracy of 94.06%.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  Kirsti Grobel,et al.  Video-Based Sign Language Recognition Using Hidden Markov Models , 1997, Gesture Workshop.

[3]  Truong Q. Nguyen,et al.  Wavelets and filter banks , 1996 .

[4]  Wen Gao,et al.  A Real-Time Large Vocabulary Continuous Recognition System for Chinese Sign Language , 2001, IEEE Pacific Rim Conference on Multimedia.

[5]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[6]  Dimitris N. Metaxas,et al.  Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[7]  Wen Gao,et al.  CSLDS: Chinese sign language dialog system , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[8]  Alfred Mertins,et al.  Signal Analysis: Wavelets, Filter Banks, Time-Frequency Transforms and Applications , 1999 .

[9]  Duy-Dinh Le,et al.  Hand gesture classification using boosted cascade of classifiers , 2006, 2006 International Conference onResearch, Innovation and Vision for the Future.

[10]  Sanjay Kumar,et al.  Visual Hand Gestures Classification Using Wavelet Transforms , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[11]  Nikolaos Mitianoudis,et al.  International Symposium on Signal Processing and its Applications , 2003 .

[12]  Peter Vamplew.  Recognition of sign language gestures using neural networks , 1996 .

[13]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[14]  Patrick Oonincx,et al.  Second generation wavelets and applications , 2005 .

[15]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1993 .

[16]  Mohamed A. Deriche,et al.  Image based arabic sign language recognition , 2005, Proceedings of the Eighth International Symposium on Signal Processing and Its Applications, 2005..

[17]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Yung-Hui Lee,et al.  Taiwan sign language (TSL) recognition based on 3D data and neural networks , 2009, Expert Syst. Appl..

[19]  Munib Qutaishat,et al.  American sign language (ASL) recognition based on Hough transform and neural networks , 2007, Expert Syst. Appl..

[20]  Wen Gao,et al.  Mahalanobis distance based Polynomial Segment Model for Chinese Sign Language Recogniton , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[21]  Sanjay Kumar,et al.  Visual Hand Gestures Classification Using Wavelet Transform and Moment Based Features , 2005, Int. J. Wavelets Multiresolution Inf. Process..

[22]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[24]  M. B. Waldron,et al.  Isolated ASL sign recognition system for deaf persons , 1995 .

[25]  Dimitris N. Metaxas,et al.  Handshapes and Movements: Multiple-Channel American Sign Language Recognition , 2003, Gesture Workshop.

[26]  KwangYun Wohn,et al.  Recognition of space-time hand-gestures using hidden Markov model , 1996, VRST.