ISL gesture recognition using multiple feature fusion

Communication is an important part of our day to day lives. It is an essential means of conveying our emotions through gestures as well as verbally. However, communication and recognition of gestures becomes a problem for humans with speaking and hearing impairment. For this very purpose sign language is used. Indian Sign Language (ISL) is resourceful in India for helping such people. The paper proposes a novel technique to identify gestures defined for the English alphabets listed in the Indian Sign Language. The proposed techniques relies on multiple representations namely HOG, GIST and BSIF. A random forest classifier is used for classifying different gestures with the aid of the combined feature vector. The results of the proposed techniques were tested on a ISL hand gesture database and were compared with the existing solutions. The technique outperformed the existing solutions resulting in an accuracy of 92.20%.

[1]  Esa Rahtu,et al.  BSIF: Binarized statistical image features , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[2]  An Liu,et al.  Crater detection algorithm with part PHOG features for safe landing , 2012, 2012 International Conference on Systems and Informatics (ICSAI2012).

[3]  Fahad Shahbaz Khan,et al.  Fusing Color and Shape for Bag-of-Words Based Object Recognition , 2013, CCIW.

[4]  Anand Singh Jalal,et al.  Recognition of Indian Sign Language using feature fusion , 2012, 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI).

[5]  Sun Quan,et al.  The Theory of Canonical Correlation Analysis and Its Application to Feature Fusion , 2005 .

[6]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[7]  Ankush Mittal,et al.  K-nearest correlated neighbor classification for Indian sign language gesture recognition using feature fusion , 2016, 2016 International Conference on Computer Communication and Informatics (ICCCI).

[8]  Ankush Mittal,et al.  A DTW and Fourier Descriptor based approach for Indian Sign Language recognition , 2015, 2015 Third International Conference on Image Information Processing (ICIIP).

[9]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[10]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  V. Adithya,et al.  Artificial neural network based method for Indian sign language recognition , 2013, 2013 IEEE CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGIES.

[12]  M. K. Bhuyan,et al.  A Framework forHand GestureRecognition with Applications toSign Language , 2006 .

[13]  Vivek Kumar Verma,et al.  A comprehensive review on automation of Indian sign language , 2015, 2015 International Conference on Advances in Computer Engineering and Applications.

[14]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Garima Joshi,et al.  Static hand gestures recognition system using shape based features , 2014, 2014 Recent Advances in Engineering and Computational Sciences (RAECS).

[16]  S. Majumder,et al.  Shape, texture and local movement hand gesture features for Indian Sign Language recognition , 2011, 3rd International Conference on Trendz in Information Sciences & Computing (TISC2011).