3D Hand Model Fitting for Virtual Keyboard System

In this paper, a 3D hand model fitting method is presented which can recover the accurate finger positions for a virtual keyboard system. The 3D hand model consists of a detailed polygonal skin driven by an underlying skeleton system. The system uses a structured light sensor to generate dense range measurements of user's hand motion. We exploit depth information and match it against the model to estimate the pose of the hand. The parameters for model deformation are optimized with the guide of the applied forces between model points and range measurements. To speed up the optimization, we simplify the physical model and apply hash table-based fast point pair matching. The system can be used in any application requiring zero formfactor and requires no contact with a medium. Examples of applications include virtual reality, gaming, design, etc

[1]  Daniel Thalmann,et al.  Models and Techniques in Computer Animation , 2014, Computer Animation Series.

[2]  Dimitris N. Metaxas,et al.  Optical Flow Constraints on Deformable Models with Applications to Face Tracking , 2000, International Journal of Computer Vision.

[3]  Ying Wu,et al.  View-independent recognition of hand postures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Ying Wu,et al.  Capturing articulated human hand motion: a divide-and-conquer approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Mathias Kölsch,et al.  Keyboards without Keyboards: A Survey of Virtual Keyboards , 2002 .

[7]  Pascal Fua,et al.  Articulated Soft Objects for Multiview Shape and Motion Capture , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Mark W. Spong,et al.  Robot dynamics and control , 1989 .

[9]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[10]  Rómer Rosales,et al.  3D Hand Pose Reconstruction Using Specialized Mappings , 2001, ICCV.

[11]  Nicol N. Schraudolph,et al.  3D hand tracking by rapid stochastic gradient descent using a skinning model , 2004 .

[12]  Tosiyasu L. Kunii,et al.  Constraint-Based Hand Animation , 1993 .

[13]  David J. Fleet,et al.  3D People Tracking with Gaussian Process Dynamical Models , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).