Real-Time and Markerless 3D Human Motion Capture Using Multiple Views

We present a fully automated system for real-time markerless 3D human motion capture. Our approach, based on fast algorithms, uses simple techniques and requires low-cost devices. Using input from multiple calibrated webcams, an extended Shape-From-Silhouette algorithm reconstructs the person in real-time. Fast 3D shape and 3D skin parts analysis provide a robust and real-time system for human full-body tracking. Animation skeleton and simple morphological constraints make easier the motion capture process. Thanks to fast and simple algorithms and low-cost cameras, our system is perfectly apt for home entertainment device. Results on real video sequences with complicated motions demonstrate the robustness of the approach.

[1]  Mohan M. Trivedi,et al.  Human Body Model Acquisition and Tracking Using Voxel Data , 2003, International Journal of Computer Vision.

[2]  Hans-Peter Seidel,et al.  M3: Marker-free Model Reconstruction and Motion Tracking from 3D Voxel Data , 2004 .

[3]  Ankur Agarwal,et al.  Monocular Human Motion Capture with a Mixture of Regressors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[4]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Bruno Raffin,et al.  3D Skeleton-Based Body Pose Recovery , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[6]  Alvin R. Tilley,et al.  The Measure of Man and Woman: Human Factors in Design , 2001 .

[7]  Saïda Bouakaz,et al.  Shape From Silhouette: Towards a Solution for Partial Visibility Problem , 2006, Eurographics.

[8]  Richard Bowden,et al.  Real-Time Upper Body Detection and 3D Pose Estimation in Monoscopic Images , 2006, ECCV.

[9]  Wei Sun,et al.  Layered animation of captured data , 2001, The Visual Computer.

[10]  David Suter,et al.  Real-Time Human Pose Inference using Kernel Principal Component Pre-image Approximations , 2006, BMVC.

[11]  Toby Howard,et al.  Real-Time 3-D Human Body Tracking using Variable Length Markov Models , 2005, BMVC.

[12]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Thomas Malzbender,et al.  Generalized Voxel Coloring , 1999, Workshop on Vision Algorithms.

[14]  Björn Stenger,et al.  Virtual Fashion Show Using Real-Time Markerless Motion Capture , 2006, ACCV.

[15]  Henry Dreyfuss,et al.  Measure of Man and Woman: Human Factors in Design , 1993 .

[16]  Raghu Machiraju,et al.  Markerless monocular motion capture using image features and physical constraints , 2005, International 2005 Computer Graphics.

[17]  Toby Howard,et al.  Real-time markerless human body tracking using colored voxels and 3D blobs , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[18]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[20]  François X. Sillion,et al.  A Real-Time System for Full Body Interaction with Virtual Worlds , 2004, EGVE.

[21]  Edmond Boyer,et al.  Exact polyhedral visual hulls , 2003, BMVC.

[22]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[24]  Hans-Peter Seidel,et al.  M/sup 3/: marker-free model reconstruction and motion tracking from 3D voxel data , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[25]  Mark Segal,et al.  Fast shadows and lighting effects using texture mapping , 1992, SIGGRAPH.

[26]  Pascal Fua,et al.  Markerless Full Body Shape and Motion Capture from Video Sequences , 2002 .