A Robust Hand Gesture Recognition Method via Convolutional Neural Network

Hand gesture plays an important role in nonverbal communication and natural human-computer interaction. However, the complex hand gesture structure and various environment factors lead to low recognition rate. For instance, hand gesture depends on individuals, and different individuals' hands are with different sizes and postures, in addition, unconstrained environmental illumination also influences hand gesture recognition performance. Therefore, hand gesture recognition is still a challenging issue. This paper proposes a robust method for hand gesture recognition based on convolutional neural network, which is utilized to automatically extract the spatial and semantic feature of hand gesture. Our method consists of a modified Convolutional Neural Network structure and data preprocessing, which corporately increase hand gesture recognition performance. The experimental results on both Cambridge Hand Gesture Dataset and self-constructed dataset show that the proposed method is effective and competitive.

[1]  Zhang Fengjun,et al.  Recognition of Complex Dynamic Gesture Based on HMM-FNN Model , 2008 .

[2]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[3]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[5]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[7]  Guozhong Dai,et al.  Recognition of Complex Dynamic Gesture Based on HMM-FNN Model: Recognition of Complex Dynamic Gesture Based on HMM-FNN Model , 2008 .

[8]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Nicolas D. Georganas,et al.  Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques , 2011, IEEE Transactions on Instrumentation and Measurement.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Sansanee Auephanwiriyakul,et al.  Thai sign language translation using Scale Invariant Feature Transform and Hidden Markov Models , 2013, Pattern Recognit. Lett..

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Xiaogang Wang,et al.  Deeply learned face representations are sparse, selective, and robust , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[15]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[16]  Yoshihiko Mochizuki,et al.  A HOG-based hand gesture recognition system on a mobile device , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[17]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..