A comparison between end-to-end approaches and feature extraction based approaches for Sign Language recognition

In this paper we use a new image dataset for Irish Sign Language (ISL) and we compare different approaches for recognition. We perform experiments and report comparative accuracy and timing. We perform tests over blurred images and compare results with non-blurred images. For classification, we use end-to-end approaches, such as Convolutional Neural Networks (CNN) and feature based extraction approaches, such as Principal Component Analysis (PCA) followed by different classifiers, e.g. multilayer perceptron (MLP). We obtain a recognition accuracy over 99% for both approaches. In addition, we report different ways to split the training and testing dataset, one being iterative and the other randomly selected.

[1]  Mohamed Farouk,et al.  Principal Component Pyramids using Image Blurring for Nonlinearity Reduction in Hand Shape Recognition , 2015 .

[2]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[3]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[4]  Alistair Sutherland,et al.  Two-stage PCA with interpolated data for hand shape recognition in sign language , 2016, 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Alistair Sutherland,et al.  Manifold Interpolation for an Efficient Hand Shape Recognition in the Irish Sign Language , 2016, ISVC.

[7]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[8]  Lorraine Leeson,et al.  Irish sign language : a cognitive linguistic account , 2012 .

[9]  Alistair Sutherland,et al.  Nonlinearity Reduction of Manifolds using Gaussian Blur for Handshape Recognition based on Multi-Dimensional Grids , 2013, ICPRAM.

[10]  Ashok Kumar Sahoo,et al.  SIGN LANGUAGE RECOGNITION: STATE OF THE ART , 2014 .

[11]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[12]  Noel E. O'Connor,et al.  A Dataset for Irish sign language recognition , 2017 .

[13]  Xiaolong Zhu,et al.  A hand shape recognizer from simple sketches , 2013, 2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013).

[14]  Noel E. O'Connor,et al.  Irish Sign Language Recognition Using Principal Component Analysis and Convolutional Neural Networks , 2017, 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[15]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).