Real-time Bangla Sign Language Detection using Xception Model with Augmented Dataset

Bangla Sign language (BdSL) is the communication language used by the deaf and dumb of Bangladesh. In this paper, we present an optimal approach to recognize BdSL in real-time. First, we have developed BdSLInfinite dataset, which consists of 2,000 images of 37 different signs. Using this dataset, a convolutional neural network (CNN) based model is trained using Xception architecture that achieves 98.93% accuracy over the test-set, with response time of 48.53 ms on average. To the best of our knowledge, our proposed method outperforms all existing BdSL recognition methods in terms of both accuracy and speed.

[1]  Tsung-Jung Liu,et al.  Smart Facial Age Estimation with Stacked Deep Network Fusion , 2018, 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW).

[2]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Robyn A. Owens,et al.  Australian sign language recognition , 2005, Machine Vision and Applications.

[4]  Thad Starner,et al.  American sign language recognition with the kinect , 2011, ICMI '11.

[5]  Dimitris N. Metaxas,et al.  Handshapes and Movements: Multiple-Channel American Sign Language Recognition , 2003, Gesture Workshop.

[6]  Benjamin Schrauwen,et al.  Sign Language Recognition Using Convolutional Neural Networks , 2014, ECCV Workshops.

[7]  Fariha Ahmed,et al.  Bangla sign language interpretation using image processing , 2017 .

[8]  Mahmood Jasim,et al.  Real-time computer vision-based Bengali Sign Language recognition , 2014, 2014 17th International Conference on Computer and Information Technology (ICCIT).

[9]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[10]  Haibo Mi,et al.  Mixup-Based Acoustic Scene Classification Using Multi-Channel Convolutional Neural Network , 2018, PCM.

[11]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Md. Saiful Islam,et al.  Real Time Bangladeshi Sign Language Detection using Faster R-CNN , 2018, 2018 International Conference on Innovation in Engineering and Technology (ICIET).

[13]  Kaushik Deb,et al.  Two-Handed Sign Language Recognition for Bangla Character Using Normalized Cross Correlation , 2012 .

[14]  Samir I. Shaheen,et al.  Sign language recognition using a combination of new vision based features , 2011, Pattern Recognit. Lett..

[15]  Ting Yao,et al.  Trimmed Action Recognition, Dense-Captioning Events in Videos, and Spatio-temporal Action Localization with Focus on ActivityNet Challenge 2019 , 2019, ArXiv.

[16]  Kang-Hyun Jo,et al.  Comparative study of modern convolutional neural networks for smoke detection on image data , 2017, 2017 10th International Conference on Human System Interactions (HSI).

[17]  Mohiuddin Ahmad,et al.  A new approach of sign language recognition system for bilingual users , 2015, 2015 International Conference on Electrical & Electronic Engineering (ICEEE).

[18]  Alauddin Bhuiyan,et al.  Bengali Sign Language Recognition Using Deep Convolutional Neural Network , 2018, 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR).