Irish Sign Language Recognition Using Principal Component Analysis and Convolutional Neural Networks

Hand-shape recognition is an important problem in computer vision with significant societal impact. In this work, we introduce a new image dataset for Irish Sign Language (ISL) recognition and we compare between two recognition approaches. The dataset was collected by filming human subjects performing ISL hand-shapes and movements. Then, we extracted frames from the videos. This produced a total of 52,688 images for the 23 common hand- shapes from ISL. Afterwards, we filter the redundant images with an iterative image selection process that selects the images which keep the dataset diverse. For classification, we use Principal Component Analysis (PCA) with with K- Nearest Neighbours (k-NN) and Convolutional Neural Networks (CNN). We obtain a recognition accuracy of 0.95 for our PCA model and 0.99 for our CNN model. We show that image blurring improves PCA results to 0.98. In addition, we compare times for classification.

[1]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[2]  Han Liu,et al.  Scale-Invariant Sparse PCA on High-Dimensional Meta-Elliptical Data , 2014, Journal of the American Statistical Association.

[3]  Noel E. O'Connor,et al.  A Dataset for Irish sign language recognition , 2017 .

[4]  Alistair Sutherland,et al.  Manifold Interpolation for an Efficient Hand Shape Recognition in the Irish Sign Language , 2016, ISVC.

[5]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[6]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[7]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[8]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[9]  Mohamed Hisham Jaward,et al.  Robust ASL Fingerspelling Recognition Using Local Binary Patterns and Geometric Features , 2013, 2013 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[10]  Lorraine Leeson,et al.  Irish sign language : a cognitive linguistic account , 2012 .

[11]  Alistair Sutherland,et al.  Nonlinearity Reduction of Manifolds using Gaussian Blur for Handshape Recognition based on Multi-Dimensional Grids , 2013, ICPRAM.

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  H. S. Nagendraswamy,et al.  Indian sign language recognition: An approach based on fuzzy-symbolic data , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[14]  Vijay John,et al.  Deep Learning-Based Fast Hand Gesture Recognition Using Representative Frames , 2016, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[15]  Mohamed Farouk,et al.  Principal Component Pyramids using Image Blurring for Nonlinearity Reduction in Hand Shape Recognition , 2015 .

[16]  Ashok Kumar Sahoo,et al.  SIGN LANGUAGE RECOGNITION: STATE OF THE ART , 2014 .