Parametric Hand Tracking and Its Applications

An approach for estimating the 3D absolute position and orientation of the hand in space using parametric modelling of the central region of the hand and some of its applications are p resented. A stereo camera is used to first build a preliminary disparity map of the hand. Then, the best fitting plan e to the disparity points is computed using robust estimation. Next, the 3D hand plane is calculated based on the d isparity plane and the six parameters of position and orientation of the hand are estimated. Tracking the hand reg ion over a sequence of frames and coping with noise using robust modelling of the hand motion enables us to estim ate the trajectory of the hand in space. Experimental results demonstrate the accuracy of this technique . A virtual drawing system and a virtual marble game are presented as two sample applications. In virtual drawing , the 3D position of the hand in space is estimated. Then, by tracking the central region of the hand in 3D and estim ating a virtual plane in space, the intended drawing of the user is recognized. This system can be used to co mmunicate drawings and alphabets to a computer where a classifier can transform the drawn alphabets into interpr etable characters. It is also shown how this system can be used to create multi-stroke drawings. In the virtual marb le game, instantaneous orientation of the hand is simulated to render a graphical scene of the game board. Rea l-tim visual feedback allows the user to navigate a virtual ball in a maze.

[1]  Olivier D. Faugeras,et al.  Finding pose of hand in video images: a stereo-based approach , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[2]  Sergio A. Velastin,et al.  Automatic congestion detection system for underground platforms , 2001, Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. ISIMP 2001 (IEEE Cat. No.01EX489).

[3]  Lars Bretzner,et al.  Use Your Hand as a 3-D Mouse, or, Relative Orientation from Extended Sequences of Sparse Point and Line Correspondences Using the Affine Trifocal Tensor , 1998, ECCV.

[4]  Thomas S. Huang,et al.  Vision based hand modeling and tracking for virtual teleconferencing and telecollaboration , 1995, Proceedings of IEEE International Conference on Computer Vision.

[5]  Stan Sclaroff,et al.  Estimating 3D hand pose from a cluttered image , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[7]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ming Xie,et al.  Hand image segmentation using color and RCE neural network , 2001, Robotics Auton. Syst..

[9]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[10]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[11]  Hideo Saito,et al.  3-D drawing system via hand motion recognition from two cameras , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[12]  Nebojsa Jojic,et al.  Detection and estimation of pointing gestures in dense disparity maps , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[13]  Stuart Geman,et al.  Statistical methods for tomographic image reconstruction , 1987 .

[14]  Petros Maragos,et al.  Optimum design of chamfer distance transforms , 1998, IEEE Trans. Image Process..

[15]  Vladimir Pavlovic,et al.  Gestural interface to a visual computing environment for molecular biologists , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[16]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[17]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Ying Wu,et al.  Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper , 2001, PUI '01.

[19]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Yvonne Freeh,et al.  An R and S–PLUS Companion to Applied Regression , 2004 .

[21]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interaction , 1999, ICVS.

[22]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[23]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  David C. Hogg,et al.  Towards 3D hand tracking using a deformable model , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[25]  Yh Nam Yanghee Nam,et al.  Hidden Markov Model Based Recognition of Space-Time Hand Gestures for Human-Computer Interaction , 1996 .

[26]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[27]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[28]  Stan Sclaroff,et al.  3D Hand Pose Estimation by Finding Appearance-Based Matches in a Large Database of Training Views , 2001 .

[29]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[31]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[32]  Tosiyasu L. Kunii,et al.  Model-based analysis of hand posture , 1995, IEEE Computer Graphics and Applications.

[33]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[34]  Rama Chellappa,et al.  Simultaneous background and foreground modeling for tracking in surveillance video , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[35]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[36]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Takeo Kanade,et al.  DigitEyes: vision-based hand tracking for human-computer interaction , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[38]  A. Murat Tekalp,et al.  Digital Video Processing , 1995 .

[39]  Yuntao Cui,et al.  A Learning-Based Prediction-and-Verification Segmentation Scheme for Hand Sign Image Sequence , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[41]  Bohyung Han,et al.  SEQUENTIAL KERNEL DENSITY APPROXIMATION THROUGH MODE PROPAGATION: APPLICATIONS TO BACKGROUND MODELING , 2004 .