Real-time 3d arm pose estimation from monocular video for enhanced HCI

In this paper an approach for 3D arm pose estimation from a monocular video is presented. Our proposal has been designed to provide real-time and realistic reconstruction of the user motion, as required by advanced Human Computer Interaction (HCI) applications. Both a 2D arm tracking and a 3D arm pose estimation algorithm are introduced and discussed. Tracking exploits fast and robust segmentation of the arm silhouette together with detection and tracking of skin colored regions. 3D pose estimation relies on a stick-figure arm model and the Analysis-by-Synthesis approach, but achieves real-time performance using geometrical constraints on tracking results to reduce the search space cardinality. Experiments on the animation of 3D avatars using off-the-shelf hardware demonstrate the effectiveness and real-time performance of our proposal.

[1]  Cordelia Schmid,et al.  Human Detection Based on a Probabilistic Assembly of Robust Part Detectors , 2004, ECCV.

[2]  Thomas B. Moeslund,et al.  Modelling and estimating the pose of a human arm , 2003, Machine Vision and Applications.

[3]  Matheen Siddiqui,et al.  Robust real-time upper body limb detection and tracking , 2006, VSSN '06.

[4]  John S. Zelek,et al.  Towards real-time 3-D monocular visual tracking of human limbs in unconstrained environments , 2005, Real Time Imaging.

[5]  Oliver Schreer,et al.  Vision-based skin-colour segmentation of moving hands for real-time applications , 2004 .

[6]  Cristian Sminchisescu,et al.  Estimating Articulated Human Motion with Covariance Scaled Sampling , 2003, Int. J. Robotics Res..

[7]  Takeo Kanade,et al.  Ambiguities in Visual Tracking of Articulated Objects Using Two- and Three-Dimensional Models , 2003, Int. J. Robotics Res..

[8]  Rajeev Sharma,et al.  Reliable tracking of human arm dynamics by multiple cue integration and constraint fusion , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[9]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[10]  Pietro Perona,et al.  Monocular tracking of the human arm in 3D , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  Peter Eisert,et al.  Real-Time Avatar Animation Steered by Live Body Motion , 2005, ICIAP.

[12]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Richard Bowden,et al.  Real-Time Upper Body Detection and 3D Pose Estimation in Monoscopic Images , 2006, ECCV.

[14]  Sidharth Bhatia,et al.  Tracking loose-limbed people , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[15]  Peter Eisert,et al.  Towards Robust Intuitive Vision-Based User Interfaces , 2006, 2006 IEEE International Conference on Multimedia and Expo.