Unsupervised Learning Based Static Hand Gesture Recognition from RGB-D Sensor

In this paper, we propose a novel, real-time static hand gesture recognition framework based on unsupervised learning. We use unlabelled training data from an RGB-D (RGB, Depth) sensor and use the depth data to detect the hand(s) in the image. We believe that the gesture is dependent on the shape of the hand, which is defined by number of extended/open fingers and which fingers are extended/open. We use unsupervised learning techniques to detect and label the extended fingers in the training data. We also use the same algorithm to detect and label the extended fingers in the test images on-the-fly. Our gesture recognition system is independent of the orientation of the gesturing hand and arm. Moreover, it does not require any markers and is completely unobtrusive, making it suitable for assistive living paradigms.

[1]  Di Xiao,et al.  An efficient and noise resistive selective image encryption scheme for gray images based on chaotic maps and DNA complementary rules , 2014, Multimedia Tools and Applications.

[2]  Paul Modler,et al.  Recognition of separate hand gestures by Time-Delay Neural Networks based on multi-state spectral image patterns from cyclic hand movements , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[3]  Nikos Papamarkos,et al.  Hand gesture recognition using a neural network shape fitting technique , 2009, Eng. Appl. Artif. Intell..

[4]  Lale Akarun,et al.  Real Time Hand Pose Estimation Using Depth Sensors , 2013, Consumer Depth Cameras for Computer Vision.

[5]  Mauro Donadeo,et al.  Combining multiple depth-based descriptors for hand gesture recognition , 2014, Pattern Recognit. Lett..

[6]  Junsong Yuan,et al.  Robust hand gesture recognition with kinect sensor , 2011, ACM Multimedia.

[7]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[8]  Andrew W. Fitzgibbon,et al.  Fits Like a Glove: Rapid and Reliable Hand Shape Personalization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Mohan M. Trivedi,et al.  Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[10]  Yimin Zhou,et al.  A novel finger and hand pose estimation technique for real-time hand gesture recognition , 2016, Pattern Recognit..

[11]  Chong Wang,et al.  Superpixel-Based Hand Gesture Recognition With Kinect Depth Camera , 2015, IEEE Transactions on Multimedia.

[12]  Wenji Yang,et al.  Hand gesture recognition using saliency and histogram intersection kernel based sparse representation , 2015, Multimedia Tools and Applications.

[13]  Hong Cheng,et al.  An image-to-class dynamic time warping approach for both 3D static and trajectory hand gesture recognition , 2016, Pattern Recognit..

[14]  Xiaodong Yang,et al.  Histogram of 3D Facets: A characteristic descriptor for hand gesture recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[15]  S. Oniga,et al.  Hand Postures Recognition System Using Artificial Neural Networks Implemented in FPGA , 2007, 2007 30th International Spring Seminar on Electronics Technology (ISSE).