Towards robust cross-user hand tracking and shape recognition

Low-cost depth cameras create new opportunities for robust and ubiquitous vision-based interfaces. While much research has focused on full-body pose estimation and the interpretation of gross body movement, this work investigates skeleton-free hand detection, tracking, and shape classification. Our goal is to build a rich and reliable gestural interface by developing methods that recognize a broad set of hand shapes and which maintain high accuracy rates across a wide range of users. In this paper, we describe our approach to real-time hand detection and tracking using depth data from the Microsoft Kinect. We present quantitative shape recognition results for eight hand shapes collected from 16 users and then discuss physical configuration and interface design issues that help boost reliability and overall user experience.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Antonis A. Argyros,et al.  Vision-based Hand Gesture Recognition for Human-Computer Interaction , 2008 .

[3]  Manolis I. A. Lourakis,et al.  Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse , 2006, ECCV Workshop on HCI.

[4]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[5]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[6]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[8]  Sebastian Thrun,et al.  Real-time identification and localization of body parts from depth images , 2010, 2010 IEEE International Conference on Robotics and Automation.

[9]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Nassir Navab,et al.  Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow , 2011, Face and Gesture 2011.

[11]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[12]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[13]  Jan Flusser,et al.  Rotation Moment Invariants for Recognition of Symmetric Objects , 2006, IEEE Transactions on Image Processing.

[14]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..