Real-time hand gesture recognition with Kinect for playing racing video games

This paper presents a Kinect based hand gesture recognition system that can effectively recognize both one-hand and two-hand gestures. It is robust against the disturbance of complex background and objects such as the faces and hands of other people by exploiting the depth information and carefully choosing the region of interest (ROI) in the process of tracking. The recognition module is implemented using template matching and other light weight techniques to reduce the computational complexity. In the experiments, this system is tested on real world tasks from controlling the slide show in PowerPoint to playing the highly intense racing video game Need for Speed. The practical performance confirms that our system is both effective in terms of robustness and versatility and efficient for various real-time applications.

[1]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[2]  Keechul Jung,et al.  Recognition-based gesture spotting in video games , 2004, Pattern Recognit. Lett..

[3]  Seong-Whan Lee,et al.  Gesture Spotting and Recognition for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[4]  Bin Liang,et al.  Gesture recognition using depth images , 2013, ICMI '13.

[5]  Bharti Bansal,et al.  Gesture Recognition: A Survey , 2016 .

[6]  Luc Van Gool,et al.  Robust tracking-by-detection using a detector confidence particle filter , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[7]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Luc Van Gool,et al.  Real-time sign language letter and word recognition from depth data , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[9]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Matthew Tang,et al.  Recognizing Hand Gestures with Microsoft ’ s Kinect , 2011 .

[11]  Mario Ciampi,et al.  Controller-free exploration of medical image data: Experiencing the Kinect , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[12]  Lale Akarun,et al.  Real time hand pose estimation using depth sensors , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[13]  Deyou Xu.  A Neural Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[14]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[15]  Antonis A. Argyros,et al.  Tracking the articulated motion of two strongly interacting hands , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Junsong Yuan,et al.  Robust hand gesture recognition with kinect sensor , 2011, ACM Multimedia.

[17]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[18]  Jovan Popovic,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH '09.

[19]  J. Y. Lee,et al.  An Elliptical Boundary Model for Skin Color Detection , 2002 .

[20]  Paolo Dario,et al.  A Survey of Glove-Based Systems and Their Applications , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.