Isolated sign language recognition with fast hand descriptors

Recognition of sign language, the main mode of communication of the hearing impaired, has attracted the attention of researchers working in the field of computer vision in recent years. In this study, we propose a fast alternative method to the Improved Dense Trajectories (IDT) method for sign language recognition. In our proposed method, Histogram of Oriented Gradients (HOG), Histogram of Optical Flow (HOF) and Motion Boundary Histograms (MBH) are obtained from the cropped hand regions. Then, Fisher Vectors (FV) are coded and used in classification with Linear Support Vector Machine (SVM) for each descriptor from each sign video. It has been shown that our method can achieve similar performance ten times faster than IDT.

[1]  Benjamin Schrauwen,et al.  Sign Language Recognition Using Convolutional Neural Networks , 2014, ECCV Workshops.

[2]  Cordelia Schmid,et al.  Human Detection Using Oriented Histograms of Flow and Appearance , 2006, ECCV.

[3]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Lale Akarun,et al.  Sign Language Recognition for Assisting the Deaf in Hospitals , 2016, HBU.

[5]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Thomas Mensink,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[9]  Cordelia Schmid,et al.  Action Recognition with Improved Trajectories , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Lale Akarun,et al.  HOSPISIGN: AN INTERACTIVE SIGN LANGUAGE PLATFORM FOR HEARING IMPAIRED , 2015 .

[12]  Lale Akarun,et al.  Isolated sign language recognition using Improved Dense Trajectories , 2016, 2016 24th Signal Processing and Communication Application Conference (SIU).

[13]  Sergio Escalera,et al.  Deep Learning for Action and Gesture Recognition in Image Sequences: A Survey , 2017, Gesture Recognition.