A virtual keyboard system based on Multi-Level Feature Matching

In this paper a multi-level feature matching (MLFM) method is presented for 3D hand posture reconstruction of a virtual keyboard system. The human hand is modeled with a mixture of different levels of detail, from skeletal to polygonal surface representation. Different types of features are extracted and paired with the corresponding model. The matching is performed in a bottom-up order by SCG optimization with respect to the state vector of motion parameters. The low level of matching provide initial guess to the high level of matching, refining the precise position of the hand hierarchically. The matching results show that this method is effective for tracking human hand typing motion, even with noisy 3D depth map reconstruction and roughly detected fingertips. Examples of applications include virtual reality, gaming, 3D design, etc.

[1]  Nicol N. Schraudolph,et al.  3D hand tracking by rapid stochastic gradient descent using a skinning model , 2004 .

[2]  Tosiyasu L. Kunii,et al.  Constraint-Based Hand Animation , 1993 .

[3]  Björn Stenger,et al.  Filtering using a tree-based estimator , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Alex Pentland,et al.  Task-Specific Gesture Analysis in Real-Time Using Interpolated Views , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  M. Møller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1990 .

[6]  Frank Forster,et al.  Real-time range imaging for human-machine interfaces , 2005 .

[7]  Mathias Kölsch,et al.  Keyboards without Keyboards: A Survey of Virtual Keyboards , 2002 .

[8]  Luc Van Gool,et al.  Smart particle filtering for 3D hand tracking , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[9]  Florentin Wörgötter,et al.  A VLSI-Compatible Computer Vision Algorithm for Stereoscopic Depth Analysis in Real-Time , 2002, International Journal of Computer Vision.

[10]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[11]  Rómer Rosales,et al.  3D Hand Pose Reconstruction Using Specialized Mappings , 2001, ICCV.

[12]  Edoardo Charbon,et al.  3D Hand Model Fitting for Virtual Keyboard System , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[13]  Nikos Paragios,et al.  Monocular Hand Pose Estimation Using Variable Metric Gradient-Descent , 2006, BMVC.

[14]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[15]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[16]  Helge Ritter,et al.  Learning to recognize 3D-Hand Postures from Perspective Pixel Images , 1992 .

[17]  Jin Y. Kim,et al.  Development of A Wearable Input Device Recognizing Human Hand and Finger Motions as A New Mobile Input Device , 2001 .

[18]  Paulo R. S. Mendonça,et al.  Model-Based Hand Tracking Using an Unscented Kalman Filter , 2001, BMVC.

[19]  Ankur Agarwal,et al.  3D human pose from silhouettes by relevance vector regression , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[20]  Daniel Thalmann,et al.  Models and Techniques in Computer Animation , 2014, Computer Animation Series.

[21]  David J. Fleet,et al.  3D People Tracking with Gaussian Process Dynamical Models , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Michael Gleicher,et al.  Building efficient, accurate character skins from examples , 2003, ACM Trans. Graph..

[23]  Larry S. Davis,et al.  Towards 3-D model-based tracking and recognition of human movement: a multi-view approach , 1995 .

[24]  Edoardo Charbon,et al.  A Virtual Keyboard Based on True-3D Optical Ranging , 2005, BMVC.