Finger spelling recognition using neural network

Finger spelling is a way of communication by expressing words using hand signs in order to ensure deaf and dumb community can communicate with others effectively. Therefore, a system that can understand finger spelling is needed. As a result of that, this work is conducted to primarily develop a tutoring system for finger spelling. To develop a robust real-time finger spelling tutoring system, it is necessary to ensure the accuracy of the finger spelling recognition. Even though there are existing solutions available for a decade, but most of them are just focusing on improving accuracy rate without implementing their solutions as a complete tutoring system for finger spelling. Consequently, it inspires this research project to develop a tutoring system for finger spelling. Microsoft Kinect sensor is used to acquire color images and depth images of the finger spells. Depth images are used to perform segmentation on the color images. After that, the segmented images are used as input and pass into a two hidden layers backpropagation neural network for classification.

[1]  Ming C. Leu,et al.  Recognition of Finger Spelling of American Sign Language with Artificial Neural Network Using Position/Orientation Sensors and Data Glove , 2005, ISNN.

[2]  Nicolas Pugeault,et al.  Spelling it out: Real-time ASL fingerspelling recognition , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[3]  Yaya Heryadi,et al.  American sign language-based finger-spelling recognition using k-Nearest Neighbors classifier , 2015, 2015 3rd International Conference on Information and Communication Technology (ICoICT).

[4]  G.R.S. Murthy,et al.  Hand gesture recognition using neural networks , 2010, 2010 IEEE 2nd International Advance Computing Conference (IACC).

[5]  Xiaolong Zhu,et al.  Single-frame hand gesture recognition using color and depth kernel descriptors , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[6]  K. Otiniano Rodriguez,et al.  Finger Spelling Recognition from RGB-D Information Using Kernel Descriptor , 2013, 2013 XXVI Conference on Graphics, Patterns and Images.

[7]  Ayoub Al-Hamadi,et al.  Utilizing Invariant Descriptors for Finger Spelling American Sign Language Using SVM , 2010, ISVC.