Data fusion-based real-time hand gesture recognition with Kinect V2

Hand gesture recognition is an important topic in human-computer interaction. However, most of the current methods are complicated and time-consuming, which limits the use of hand gesture recognition in real-time circumstances. In this paper, we propose a data fusion-based hand gesture recognition model by fusing depth information and skeleton data. Because of the accurate segmentation and tracking with Kinect V2, the model can achieve real-time performance, which is 18.7% faster than some of the state-of-the-art methods. Based on the experimental results, the proposed model is accurate and robust to rotation, flip, scale changes, lighting changes, cluttered background, and distortions. This ensures its use in different real-world human-computer interaction tasks.

[1]  Tack-Don Han,et al.  Gesture-based interactive augmented reality content authoring system using HMD , 2016, Virtual Reality.

[2]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[3]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[4]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[5]  Simon Weidert,et al.  Machine learning-based augmented reality for improved surgical scene understanding , 2015, Comput. Medical Imaging Graph..

[6]  Luc Van Gool,et al.  Random Forests for Real Time 3D Face Analysis , 2012, International Journal of Computer Vision.

[7]  Ajmal S. Mian,et al.  Using Kinect for face recognition under varying poses, expressions, illumination and disguise , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[8]  Yingli Tian,et al.  Histogram of 3D Facets: A depth descriptor for human action and hand gesture recognition , 2015, Comput. Vis. Image Underst..

[9]  Ling Shao,et al.  One shot learning gesture recognition from RGBD images , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Hafiz Imtiaz,et al.  A template matching approach of one-shot-learning gesture recognition , 2013, Pattern Recognit. Lett..

[11]  Martin Saerbeck,et al.  Recent methods and databases in vision-based hand gesture recognition: A review , 2015, Comput. Vis. Image Underst..

[12]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[13]  Emil M. Petriu,et al.  Dynamic Sign Language Recognition for Smart Home Interactive Application Using Stochastic Linear Formal Grammar , 2015, IEEE Transactions on Instrumentation and Measurement.

[14]  Yui Man Lui,et al.  Human gesture recognition on product manifolds , 2012, J. Mach. Learn. Res..

[15]  Jack Sklansky,et al.  Finding the convex hull of a simple polygon , 1982, Pattern Recognit. Lett..

[16]  Ana M. Barbancho,et al.  Fast-gesture recognition and classification using Kinect: an application for a virtual reality drumkit , 2015, Multimedia Tools and Applications.

[17]  Yui Man Lui,et al.  A least squares regression framework on manifolds and its application to gesture recognition , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[18]  Mohan M. Trivedi,et al.  Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations , 2014, IEEE Transactions on Intelligent Transportation Systems.