Recognition of a real-time signer-independent static Farsi sign language based on fourier coefficients amplitude

A sign language is a language which uses manual communication and body language to convey meaning, as opposed to acoustically conveyed sound patterns. This can involve simultaneously combining hand shapes, orientation and movement of the hands, arms or body, and facial expressions to fluidly express a speaker’s thoughts. Sign language is a preliminary communication way for individuals with hearing and speech problems. Considering that more than a hundred million people all around the world are annoyed by hearing loss and impaired speech, it is needed to design a system for automatic sign language interpreter as an interface between deaf-and-dumb and ordinary people can feel it strongly. Given this, in this article we aimed to design such a computer vision-based translation interface. Farsi sign language recognition is one of the most challenging fields of study is given because of some features such as the wide range of similar gestures, hands orientation, complicated background, and ambient light variation. A Farsi sign language recognition system is based on computer vision which is capable of real-time gesture processing and is independent of the signer. Furthermore, there is no need to use glove or marker by the signer in the proposed system. After hand segmentation from video frames, the proposed algorithm extracts boundary of the dominant hand to compute the normalized accumulation angle and represents the boundary, so that the invariance to transition and scale change of the features is realized at this stage. Afterward, Fourier coefficients amplitude is extracted as preferred features at the frequency domain, while invariance to rotation of the features is added at this point. Then the frequency features, as extracted features for gesture recognition, are applied to inputs of feed-forward multilayer perception neural network. The proposed method is presented to make recognition system independent of the signer and retrofit it against signer’s distance changes from camera using features of powerful invariant extraction against transition, scale change, and rotation. Data classification is carried out by three classifiers including Bayes, K-NN, and neural network. Performance of the classifiers was also compared. Training set of gestures comprised 250 samples for 10 gestures and 5 positions and orientations that were performed by 5 individuals. Recognition results showed an outstanding recognition rate of the system.

[1]  Dimitris N. Metaxas,et al.  Parallel hidden Markov models for American sign language recognition , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Howard B. Demuth,et al.  Neutral network toolbox for use with Matlab , 1995 .

[3]  S. Padmavathi,et al.  Computer vision based approach for Indian Sign Language character recognition , 2012, 2012 International Conference on Machine Vision and Image Processing (MVIP).

[4]  F. Wong,et al.  Hidden Markov Model-Based Gesture Recognition with Overlapping Hand-Head/Hand-Hand Estimated Using Kalman Filter , 2012, 2012 Third International Conference on Intelligent Systems Modelling and Simulation.

[5]  Phillip Ho,et al.  Sign Language Classification Using Webcam Images , 2011 .

[6]  Ali Karami,et al.  Persian sign language (PSL) recognition using wavelet transform and neural networks , 2011, Expert Syst. Appl..

[7]  Nitin V. Pujari,et al.  Finger Detection for Sign Language Recognition , 2009 .

[8]  Chin-Shyurng Fahn,et al.  A Human-Machine Interaction Technique: Hand Gesture Recognition Based on Hidden Markov Models with Trajectory of Hand Motion , 2011 .

[9]  Richard J. Prokop,et al.  A survey of moment-based techniques for unoccluded object representation and recognition , 1992, CVGIP Graph. Model. Image Process..

[10]  Omar M. Al-Jarrah,et al.  Recognition of gestures in Arabic sign language using neuro-fuzzy systems , 2001, Artif. Intell..

[11]  Vasif V. Nabiyev,et al.  TOWARDS A BIOMETRIC PURPOSE IMAGE FILTER ACCORDING TO SKIN DETECTION , 2008 .

[12]  Oguz Altun,et al.  Turkish fingerspelling recognition system using Generalized Hough Transform, interest regions, and local descriptors , 2011, Pattern Recognit. Lett..

[13]  Paul Goh,et al.  Automatic Recognition of Auslan Finger-spelling using Hidden Markov Models , 2005 .

[14]  Wen Gao,et al.  Recognition of strong and weak connection models in continuous sign language , 2002, Object recognition supported by user interaction for service robots.

[15]  Ayush S Parashar,et al.  Representation and Interpretation of Manual and Non-Manual Information for Automated American Sign Language Recognition , 2003 .

[16]  Paul Lukowicz,et al.  Using multiple sensors for mobile sign language recognition , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[17]  Vit Niennattrakul,et al.  TFRS: Thai finger-spelling sign language recognition system , 2012, 2012 Second International Conference on Digital Information and Communication Technology and it's Applications (DICTAP).

[18]  Nikolaos G. Bourbakis,et al.  A survey of skin-color modeling and detection methods , 2007, Pattern Recognit..

[19]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[20]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[21]  Mark S. Nixon,et al.  Feature Extraction and Image Processing , 2002 .

[22]  M. Z. Susic,et al.  Recognition and classification of deaf signs using neural networks , 2012, 11th Symposium on Neural Network Applications in Electrical Engineering.

[23]  Sumit Kumar,et al.  Recognition of Single Handed Sign Language Gestures using Contour Tracing Descriptor , 2022 .

[24]  Gang Hua,et al.  Dynamic hand gesture recognition: An exemplar-based approach from motion divergence fields , 2012, Image Vis. Comput..

[25]  Rajiv V. Dharaskar,et al.  Video gestures identification and recognition using Fourier descriptor and general fuzzy minmax neural network for subset of Indian sign language , 2012, 2012 12th International Conference on Hybrid Intelligent Systems (HIS).