Finding pose of hand in video images: a stereo-based approach

We propose a method to estimate the pose of a hand in a sequence of stereo images. This is a difficult problem since a hand is a complex object with a high number of degrees of freedom, and automatically segmenting the hand in the images is not easy. Our method is intended to solve these problems. Two video cameras feed two images to a stereo correlation algorithm, allowing the 3D reconstruction of the scene. Then a 3D articulated model of the hand, made of truncated cones and spheres, is fitted to this reconstruction in order to estimate the pose of the palm and fingers. We are dealing with model-based tracking of hand movement, in which we suppose that the pose of the hand is known in the first images.

[1]  Quentin Delamarre.  Modélisation de la main pour sa localisation dans une séquence d'images , 1996 .

[2]  Jean-Philippe Thirion,et al.  Fast Non-Rigid Matching of 3D Medical Images , 1995 .

[3]  W. Rulka,et al.  Aspects of Efficient and Reliable Multibody System Simulation , 1990 .

[4]  Frederic Devernay,et al.  Système de miroirs pour la stéréoscopie , 1995 .

[5]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[6]  Roderic C. Deyo,et al.  Real-Time Integration Methods for Mechanical System Simulation , 1991 .

[7]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[8]  Ahmed A. Shabana,et al.  Dynamics of Multibody Systems , 2020 .

[9]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[10]  Frederic Devernay.  Vision stéréoscopique et propriétés différentielles des surfaces , 1997 .

[11]  Olivier D. Faugeras,et al.  Computing differential properties of 3-D shapes from stereoscopic images without 3-D models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jean-Philippe Thirion,et al.  Non-rigid matching using demons , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Olivier D. Faugeras,et al.  From projective to Euclidean reconstruction , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Pascal Fua,et al.  Combining Stereo and Monocular Information to Compute Dense Depth Maps that Preserve Depth Discontinuities , 1991, IJCAI.

[15]  S. Sarkar,et al.  Human skin and hand motion analysis from range image sequences using nonlinear FEM , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[16]  Jake K. Aggarwal,et al.  Human motion analysis: a review , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.