Dynamic Hand Gesture Recognition Using Generalized Time Warping and Deep Belief Networks

Body gestures play an important role in human communications, specially hand gestures are the most distinctive features in sign languages. Several works have been proposed in order to recognize hand gestures using static and dynamic approaches. Nevertheless, due to the high variety of signs and the dynamic changes exhibited in different hand motions, a strategy for modeling these dynamic changes in hand signs must be fulfilled. In this work we propose a framework for dynamic hand gesture recognition using a well known method for alignment of time series as the Generalized Time Warping (GTW). Several features are extracted from the aligned sequences of hand gestures based on texture descriptors. Then a methodology for hand motion recognition is carried out based on Convolutional Neural Networks. The obtained results show that the methodology proposed allows an accurate recognition of several hand gestures obtained from the RVL-SLLL American Sign Language Database.

[1]  Soumik Mondal,et al.  Recognition of Isolated Indian Sign Language Gesture in Real Time , 2010, BAIP.

[2]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[3]  Hélène Laurent,et al.  Hand gesture recognition using a dedicated geometric descriptor , 2012, 2012 3rd International Conference on Image Processing Theory, Tools and Applications (IPTA).

[4]  Avinash C. Kak,et al.  Proceedings of IEEE International Conference on Multimodel Interfaces, 2002 , 2022 .

[5]  M. Younus Javed,et al.  A Survey on Sign Language Recognition , 2011, 2011 Frontiers of Information Technology.

[6]  S. Foo,et al.  Hand pose estimation for American sign language recognition , 2004, Thirty-Sixth Southeastern Symposium on System Theory, 2004. Proceedings of the.

[7]  Honglak Lee,et al.  Unsupervised feature learning for audio classification using convolutional deep belief networks , 2009, NIPS.

[8]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[9]  M. S. Sinith,et al.  Sign Gesture Recongnition Using Support Vector Machine , 2012, 2012 International Conference on Advances in Computing and Communications.

[10]  Hong Li,et al.  Model-based segmentation and recognition of dynamic gestures in continuous video streams , 2011, Pattern Recognit..

[11]  Bahman Zanj,et al.  A Neural Network based system for Persian sign language recognition , 2009, 2009 IEEE International Conference on Signal and Image Processing Applications.

[12]  A. Abdelbaky Ahmed,et al.  Appearance-based Arabic Sign Language recognition using Hidden Markov Models , 2014, 2014 International Conference on Engineering and Technology (ICET).