Arabic sign language continuous sentences recognition using PCNN and graph matching

Many previous researchers have tried developing sign languages recognition systems in general and Arabic sign language specifically. They succeeded to achieve acceptable results for isolated gestures level, but none of them investigated the recognition of connected sequence of gestures. This paper focuses on how to recognize real-time connected sequence of gestures using graph-matching technique, also how the continuous input gestures are segmented and classified. Graphs are a general and powerful data structure useful for the representation of various objects and concepts. This work is a component of a real-time Arabic Sign Language Recognition system that applied pulse-coupled neural network for static posture recognition in its first phase. This work can be adapted and applied to different sign languages and other recognition problems.

[1]  Reinhard Eckhorn,et al.  Feature Linking via Synchronization among Distributed Assemblies: Simulations of Results from Cat Visual Cortex , 1990, Neural Computation.

[2]  Khaled Assaleh,et al.  Recognition of Arabic Sign Language Alphabet Using Polynomial Classifiers , 2005, EURASIP J. Adv. Signal Process..

[3]  Hermann Ney,et al.  Signspeak--understanding, recognition, and translation of sign languages , 2010 .

[4]  Mohamed A. Deriche,et al.  Image based arabic sign language recognition , 2005, Proceedings of the Eighth International Symposium on Signal Processing and Its Applications, 2005..

[5]  Omar M. Al-Jarrah,et al.  Recognition of gestures in Arabic sign language using neuro-fuzzy systems , 2001, Artif. Intell..

[6]  Hermann Hienz,et al.  Relevant features for video-based continuous sign language recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[7]  Jason M. Kinser,et al.  Pulse image processing using centripetal autowaves , 2000, SPIE Defense + Commercial Sensing.

[8]  Chao-Hung Lin,et al.  A graph‐based shape matching scheme for 3D articulated objects , 2011, Comput. Animat. Virtual Worlds.

[9]  Horst Bunke,et al.  A decision tree approach to graph and subgraph isomorphism detection , 1999, Pattern Recognit..

[10]  Edwin R. Hancock,et al.  Graph Partition for Matching , 2003, GbRPR.

[11]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[12]  Mohammad A. Al-Rousan,et al.  Automatic Recognition of Arabic Sign Language Finger Spelling , 2001, Int. J. Comput. Their Appl..

[13]  Manjari Gupta,et al.  Design pattern mining using greedy algorithm for multi-labelled graphs , 2011, Int. J. Inf. Commun. Technol..

[14]  Karl-Friedrich Kraiss,et al.  Robust Person-Independent Visual Sign Language Recognition , 2005, IbPRIA.

[15]  Jean-Marc Ogier,et al.  Local Structural Analysis: A Primer , 2003, GREC.

[16]  I. Mokris,et al.  Feature generation improving by optimized PCNN , 2008, 2008 6th International Symposium on Applied Machine Intelligence and Informatics.

[17]  Kostas Topouzelis.  A Neural Network Approach to Oil Spill Detection using SAR Data , 2003 .

[18]  J. van Leeuwen,et al.  Graph Based Representations in Pattern Recognition , 2003, Lecture Notes in Computer Science.

[19]  Yoshiaki Shirai,et al.  Extraction of Hand Features for Recognition of Sign Language Words , 2002 .