Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

This paper presents an innovative solution based on Time-Of-Flight (TOF) video technology to motion patterns detection for real-time dynamic hand gesture recognition. The resulting system is able to detect motion-based hand gestures getting as input depth images. The recognizable motion patterns are modeled on the basis of the human arm anatomy and its degrees of freedom, generating a collection of synthetic motion patterns that is compared with the captured input patterns in order to finally classify the input gesture. For the evaluation of our system a significant collection of gestures has been compiled, getting results for 3D pattern classification as well as a comparison with the results using only 2D information.

[1]  Dacheng Tao,et al.  Feature fusion for 3D hand gesture recognition by learning a shared hidden space , 2012, Pattern Recognit. Lett..

[2]  Daniel Kelly,et al.  A person independent system for recognition of hand postures used in sign language , 2010, Pattern Recognit. Lett..

[3]  Joachim Hornegger,et al.  Gesture recognition with a Time-Of-Flight camera , 2008, Int. J. Intell. Syst. Technol. Appl..

[4]  Surendra Ranganath,et al.  Sign Language Phoneme Transcription with Rule-based Hand Trajectory Segmentation , 2010, J. Signal Process. Syst..

[5]  Gang Zheng,et al.  Application of Projective Invariants in Hand Geometry Biometrics , 2007, IEEE Transactions on Information Forensics and Security.

[6]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[7]  Yen-Ting Chen,et al.  Developing a Multiple-angle Hand Gesture Recognition System for Human Machine Interactions , 2007, IECON 2007 - 33rd Annual Conference of the IEEE Industrial Electronics Society.

[8]  Joachim Hornegger,et al.  3-D gesture-based scene navigation in medical imaging applications using Time-of-Flight cameras , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[9]  Ying Wu,et al.  Robust 3D Action Recognition with Random Occupancy Patterns , 2012, ECCV.

[10]  Z. Liu,et al.  A real time system for dynamic hand gesture recognition with a depth sensor , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[11]  Stefan Müller,et al.  Hand Gesture Recognition with a Novel IR Time-of-Flight Range Camera-A Pilot Study , 2007, MIRAGE.

[12]  Yiping Yang,et al.  Real-time Hand Gesture Recognition from Depth Images Using Convex Shape Decomposition Method , 2014, J. Signal Process. Syst..

[13]  Kwan H. Lee,et al.  Depth video based human model reconstruction resolving self-occlusion , 2010, IEEE Transactions on Consumer Electronics.

[14]  Rasmus Larsen,et al.  Improved 3D reconstruction in smart-room environments using ToF imaging , 2010, Comput. Vis. Image Underst..

[15]  Robyn A. Owens,et al.  Australian sign language recognition , 2005, Machine Vision and Applications.

[16]  Jesús Bescós,et al.  A natural and synthetic corpus for benchmarking of hand gesture recognition systems , 2013, Machine Vision and Applications.

[17]  Lale Akarun,et al.  STARS: Sign tracking and recognition system using input-output HMMs , 2009, Pattern Recognit. Lett..

[18]  Helena M. Mentis,et al.  Instructing people for training gestural interactive systems , 2012, CHI.

[19]  Rasmus Larsen,et al.  TOF imaging in Smart room environments towards improved people tracking , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Jesús Bescós,et al.  Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models , 2011, Machine Vision and Applications.

[21]  Gary R. Bradski,et al.  Stereo based gesture recognition invariant to 3D pose and lighting , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[22]  Chengdong Wu,et al.  Dynamic hand gesture recognition using motion trajectories and key frames , 2010, 2010 2nd International Conference on Advanced Computer Control.

[23]  Rainer Stiefelhagen,et al.  Visual recognition of pointing gestures for human-robot interaction , 2007, Image Vis. Comput..

[24]  Bian Wu,et al.  A hand gesture recognition system based on local linear embedding , 2005, J. Vis. Lang. Comput..

[25]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[26]  Stan Sclaroff,et al.  Estimating 3D hand pose from a cluttered image , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Jianying Hu,et al.  HMM Based On-Line Handwriting Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Mongi A. Abidi,et al.  Spatial and Temporal Enhancement of Depth Images Captured by a Time-of-Flight Depth Sensor , 2010, 2010 20th International Conference on Pattern Recognition.

[29]  Joseph J. LaViola.  Bringing VR and Spatial 3D Interaction to the Masses through Video Games , 2008, IEEE Computer Graphics and Applications.

[30]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[31]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[32]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[33]  José María Martínez Sanchez,et al.  A synthetic training framework for providing gesture scalability to 2.5D pose-based hand gesture recognition systems , 2014, Machine Vision and Applications.

[34]  George Bebis,et al.  Global Hand Pose Estimation by Multiple Camera Ellipse Tracking , 2006, ISVC.

[35]  Christian Lantuéjoul,et al.  Geodesic methods in quantitative image analysis , 1984, Pattern Recognit..

[36]  Ho-Joon Kim,et al.  Dynamic hand gesture recognition using a CNN model with 3D receptive fields , 2008, 2008 International Conference on Neural Networks and Signal Processing.

[37]  Björn Stenger,et al.  Model-based hand tracking using a hierarchical Bayesian filter , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Michael G. Strintzis,et al.  Real-time hand posture recognition using range data , 2008, Image Vis. Comput..