Bengali Sign language to text conversion using artificial neural network and support vector machine

This paper presents a novel system that converts Bengali Sign language to text using an optimum system comprising of artificial neural networks and support vector machine (SVM). Microsoft Kinect is used to take the input, which is the hand sign performed in front of the camera. The captured hand sign is eventually recognized, after joint and wrist detection and by assessing the contours. Contour feature is extracted and is run through a SVM for classification of the sign. The contour finding algorithm utilizes the convex hull method, and the features extracted after detection is passed through the support vector model for recognition. To validate the performance of the proposed model, a dataset that consists of both male and female hand gesture images is utilized. Experimental results demonstrate 84.11% classification accuracy for our tested dataset.

[1]  Trevor Darrell,et al.  Hidden Conditional Random Fields for Gesture Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  C. Shufflebarger,et al.  What is neutral network , 1992 .

[3]  Michal Kawulok,et al.  Hand landmarks detection and localization in color images , 2016, Multimedia Tools and Applications.

[4]  Brandon Garcia,et al.  Real-time American Sign Language Recognition with Convolutional Neural Networks , 2022 .

[5]  Angur M. Jarman,et al.  An Automated Bengali Sign Language Recognition System Based on Fingertip Finder Algorithm , 2015 .

[6]  Jessica Lowell.  Neural Network , 2001 .

[7]  Ednaldo Brigante Pizzolato,et al.  Sign Language Recognition with Support Vector Machines and Hidden Conditional Random Fields: Going from Fingerspelling to Natural Articulated Words , 2013, MLDM.

[8]  Yunzhe Jia,et al.  An Integrated Approach of Real-time Hand Gesture Recognition Based on Feature Points , 2015, MUE 2015.

[9]  Mohammad Alauddin,et al.  Deafness in Bangladesh , 2004 .

[10]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[11]  Chih-Jen Lin,et al.  Feature Ranking Using Linear SVM , 2008, WCCI Causation and Prediction Challenge.

[12]  Mrs. A. R. Patil,et al.  On Vision Based Hand Gesture Recognition Approach Using Support Vector Machines . , .

[13]  Naomi Krishnarajah,et al.  Capturing Hand Gesture Movement: A Survey on Tools, Techniques and Logical Considerations , 2011 .

[14]  Guang Li,et al.  Sign Language Recognition and Translation with Kinect , 2013 .

[15]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[16]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.