Real time Hand Gesture Recognition using a Range Camera

This paper proposes a real time hand gesture recognition system. The approach uses a range camera to capture the depth data. After some preprocessing procedures, the depth data is used to segment the hand and then locate the hand in 3D space. The hand shape is classified into known categories using a chamfer matching method to measure the similarities between the candidate hand image and the hand templates in the database. The 3D hand trajectory is recognized by a Finite State Machine (FSM) method. Each gesture consists of several states. The 3D hand position determines the state transition of each gesture recognizer. Experiments show that the system performs reliably for recognizing both static hand shapes and spatial-temporal trajectories in real time.

[1]  Reinhard Koch,et al.  Pose estimation and map building with a Time-Of-Flight-camera for robot navigation , 2008, Int. J. Intell. Syst. Technol. Appl..

[2]  Bernd Jähne,et al.  Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras , 2009, Optical Engineering.

[3]  Thomas B. Moeslund,et al.  Fusion of range and intensity information for view invariant gesture recognition , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[4]  Andreas Kolb,et al.  Calibration of the intensity-related distance error of the PMD TOF-camera , 2007, SPIE Optics East.

[5]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Alexander H. Waibel,et al.  Natural human-robot interaction using speech, head pose and gestures , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[7]  Marc Pollefeys,et al.  IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops, 2008 , 2008, CVPR 2008.

[8]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[9]  Lisa Gralewski,et al.  Using a tensor framework for the analysis of facial dynamics , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[10]  Sonia Chernova,et al.  Mobile human-robot teaming with environmental tolerance , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Stefan Müller,et al.  Hand Gesture Recognition with a Novel IR Time-of-Flight Range Camera-A Pilot Study , 2007, MIRAGE.

[12]  Shan Lu,et al.  Color-based hands tracking system for sign language recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[13]  Thomas S. Huang,et al.  Gesture modeling and recognition using finite state machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  David Salesin,et al.  Fast multiresolution image querying , 1995, SIGGRAPH.

[15]  Ralf Reulke,et al.  Combination of distance data with high resolution images , 2006 .

[16]  Rainer Stiefelhagen,et al.  3D-tracking of head and hands for pointing gesture recognition in a human-robot interaction scenario , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[17]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[18]  Reinhard Koch,et al.  Single View Motion Tracking by Depth and Silhouette Information , 2007, SCIA.

[19]  Hilmar Ingensand,et al.  Calibration and improvements of the high-resolution range-imaging camera SwissRanger , 2005 .

[20]  John R. Kender,et al.  Toward the use of gesture in traditional user interfaces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[21]  Gunilla Borgefors,et al.  Hierarchical Chamfer Matching: A Parametric Edge Matching Algorithm , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  J. P. Lewis,et al.  SmartCanvas: a gesture-driven intelligent drawing desk system , 2005, IUI.

[23]  Reinhard Koch,et al.  A Comparison of PMD-Cameras and Stereo-Vision for the Task of Surface Reconstruction using Patchlets , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.