Binocular Hand Tracking and Pose Estimation Stereo baserat handfoljning och pose estimering

Hand gestures are important in human communication. It is difficult to transmit visual or spatial concepts only with oral communication (communication human to human) or by regular user interfaces(communication human to machine): keyboard, mouse, etc. The systems of hand gestures recognition are nowadays mainly oriented to the recognition of predefined sets of gestures. This is useful for applications like virtual mouses guided by the fingertips, sign language recognition, etc. In the context of Programming by Demonstration, these sets of predefined gestures are not enough, since a system should be able to learn any kind of ”regular” gesture. The goal of this thesis is to recover the hand pose from a stereo vision system. Two main problems are faced in this thesis: what features should we extract from the vision system and how can we extract from these features the hand configuration. The feature used in this thesis is the location in 3D of fingertips, and this information is translated into the hand pose configuration by a closed form inverse kinematics solution.

[1]  Stan Sclaroff,et al.  3D hand pose reconstruction using specialized mappings , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[2]  M. Lopes,et al.  Motor Representations for Hand Gesture Recognition and Imitation , 2003 .

[3]  Sharath Pankanti,et al.  Appearance models for occlusion handling , 2006, Image Vis. Comput..

[4]  Björn Stenger,et al.  Filtering using a tree-based estimator , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Dante Augusto Couto Barone,et al.  Performance evaluation of single and multiple-Gaussian models for skin color modeling , 2002, Proceedings. XV Brazilian Symposium on Computer Graphics and Image Processing.

[6]  Stan Sclaroff,et al.  An appearance-based framework for 3D hand shape classification and camera viewpoint estimation , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[7]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[8]  Gordon Cheng,et al.  Unconstrained Real-time Markerless Hand Tracking for Humanoid Interaction , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[9]  Cory S Myers,et al.  A comparative study of several dynamic time warping algorithms for speech recognition , 1980 .

[10]  D J Giurintano,et al.  A virtual five-link model of the thumb. , 1995, Medical engineering & physics.

[11]  José Santos-Victor,et al.  Visual learning by imitation with motor representations , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Tosiyasu L. Kunii,et al.  Model-based analysis of hand posture , 1995, IEEE Computer Graphics and Applications.

[13]  Tieniu Tan,et al.  Real time hand tracking by combining particle filtering and mean shift , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[14]  Helge J. Ritter,et al.  Neural Architectures for Robot Intelligence , 2003, Reviews in the neurosciences.

[15]  Helge J. Ritter,et al.  Detection of Fingertips in Human Hand Movement Sequences , 1997, Gesture Workshop.

[16]  Manolis I. A. Lourakis,et al.  Binocular Hand Tracking and Reconstruction Based on 2D Shape Matching , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[17]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[18]  Mubarak Shah,et al.  Visual gesture recognition , 1994 .

[19]  Cullen Jennings.  Thesis Proposal: 3D Tracking & Recognition of Natural Objects , 1998 .

[20]  Manolis I. A. Lourakis,et al.  Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse , 2006, ECCV Workshop on HCI.

[21]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[22]  Josep Maria,et al.  Intelligent task-level grasp mapping for robot control , 2006 .

[23]  Takeo Kanade,et al.  DigitEyes: vision-based hand tracking for human-computer interaction , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[24]  James L. Crowley,et al.  Finger Tracking as an Input Device for Augmented Reality , 1995 .

[25]  Paulo R. S. Mendonça,et al.  Model-Based Hand Tracking Using an Unscented Kalman Filter , 2001, BMVC.

[26]  Patrick Horain,et al.  Hand tracking by 3D model registration , 1999 .

[27]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[28]  Jianping Fan,et al.  Automatic image segmentation by integrating color-edge extraction and seeded region growing , 2001, IEEE Trans. Image Process..

[29]  Helge J. Ritter,et al.  Visual recognition of continuous hand postures , 2002, IEEE Trans. Neural Networks.

[30]  Carlo Tomasi,et al.  3D tracking = classification + interpolation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[31]  Björn Stenger,et al.  Model-based hand tracking using a hierarchical Bayesian filter , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Yoshiaki Shirai,et al.  Hand gesture estimation and model refinement using monocular camera-ambiguity limitation by inequality constraints , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[33]  Antonis A. Argyros.  Tracking Skin-Colored Objects in Real-time , 2005 .

[34]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[35]  David W. Murray,et al.  Recovering articulated pose: a comparison of two pre and postimposed constraint methods , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Shan Lu,et al.  Using multiple cues for hand tracking and model refinement , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[37]  Michael I. Mandel,et al.  Visual Hand Tracking Using Nonparametric Belief Propagation , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[38]  Y. K. Ho,et al.  Hand posture estimation from 2D monocular image , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[39]  Luc Van Gool,et al.  Smart particle filtering for 3D hand tracking , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[40]  Ying Wu,et al.  Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper , 2001, PUI '01.

[41]  James M. Rehg,et al.  Reconstruction of 3D figure motion from 2D correspondences , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  Eun-Jung Holden,et al.  Visual recognition of hand motion , 1997 .

[43]  Paulo R. S. Mendonça,et al.  Model-based 3D tracking of an articulated hand , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[44]  M. Worring,et al.  Occlusion robust adaptive template tracking , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[45]  José Santos-Victor,et al.  Visual transformations in gesture imitation: what you see is what you do , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[46]  Dmitry O. Gorodnichy,et al.  Detection and tracking of pianist hands and fingers , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[47]  Manolis I. A. Lourakis,et al.  Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera , 2004, ECCV.

[48]  Olivier D. Faugeras,et al.  Finding pose of hand in video images: a stereo-based approach , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[49]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[51]  Larry S. Davis,et al.  A Robust Background Subtraction and Shadow Detection , 1999 .

[52]  Yoichi Sato,et al.  Real-Time Fingertip Tracking and Gesture Recognition , 2002, IEEE Computer Graphics and Applications.

[53]  Stan Sclaroff,et al.  Estimating 3D hand pose from a cluttered image , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[54]  Ramesh Raskar,et al.  Discontinuity preserving stereo with small baseline multi-flash illumination , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[55]  Ying Wu,et al.  3D model-based hand tracking using stochastic direct search method , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[56]  S. Ahmad,et al.  A usable real-time 3D hand tracker , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[57]  Ivan Laptev,et al.  Tracking of Multi-state Hand Models Using Particle Filtering and a Hierarchy of Multi-scale Image Features , 2001, Scale-Space.

[58]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[59]  Hai Tao,et al.  A background layer model for object tracking through occlusion , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[60]  Ying Wu,et al.  Capturing articulated human hand motion: a divide-and-conquer approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[61]  Isabel Serrano Vicente.  Human Action Recognition Based on Linear and Non-linear Dimensionality Reduction using PCA and ISOMAP , 2006 .

[62]  Michael Girard,et al.  Computer animation of knowledge-based human grasping , 1991, SIGGRAPH.