Recognition of the Hungarian fingerspelling alphabet using Recurrent Neural Network

The aim of this paper is to introduce a Recurrent Convolutional Neural Network based on depth data to recognize the signs of the Hungarian fingerspelling alphabet. The training dataset contains depth data of 27 static and 15 dynamic signs. A 88.6% classification accuracy was measured for during the test with the recommended model in this paper, which is a special type of recurrent network containing LSTM and convolutional layers.

[1]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[2]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[4]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[5]  Dianna Radpour,et al.  Using Deep Convolutional Networks for Gesture Recognition in American Sign Language , 2017, ArXiv.

[6]  Kevin N. Gurney,et al.  An introduction to neural networks , 2018 .

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  Pavlo Molchanov,et al.  Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Truong Q. Nguyen,et al.  Real-time sign language fingerspelling recognition using convolutional neural networks from depth map , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[10]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[11]  Carla E. Brodley,et al.  Machine learning techniques for the computer security domain of anomaly detection , 2000 .

[12]  Bence Dankó,et al.  Recognition of the Hungarian Fingerspelling Alphabet using Convolutional Neural Network based on Depth Data , 2018, 2018 IEEE 18th International Symposium on Computational Intelligence and Informatics (CINTI).

[13]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[14]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[15]  Pavlo Molchanov,et al.  Hand gesture recognition with 3D convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[17]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[18]  Yoshua Bengio,et al.  Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[19]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[20]  Nojun Kwak,et al.  Analysis on the Dropout Effect in Convolutional Neural Networks , 2016, ACCV.

[21]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[22]  Mohan M. Trivedi,et al.  Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations , 2014, IEEE Transactions on Intelligent Transportation Systems.