A Real-Time 3D Human Body Tracking and Modeling System

In this paper a real-time system for 3D human upper body tracking and modeling is proposed. The system uses multiple cameras to recover the depth maps in real-time, then integrates both color and depth information to track the human body, head, and hands, and finally recovers the 3D upper body model parameters from the tracking results. Extensive experiments demonstrate that the system can track and rebuild human model in complicated situations. The system makes good tradeoff between the accuracy and system simplicity, and can be widely used in many applications, such as desktop interaction and digital entertainments.

[1]  Ioannis A. Kakadiaris,et al.  Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Rajeev Sharma,et al.  Articulated models from video , 2004, CVPR 2004.

[3]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[4]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  I-Cheng Chang,et al.  The model-based human body motion analysis system , 2000, Image Vis. Comput..

[6]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[7]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[9]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[10]  B. Kröse,et al.  An EM-like algorithm for color-histogram-based object tracking , 2004, CVPR 2004.

[11]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[12]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[13]  Patrick Pérez,et al.  Data fusion for visual tracking with particles , 2004, Proceedings of the IEEE.

[14]  Thomas W. Calvert,et al.  Goal-directed, dynamic animation of human walking , 1989, SIGGRAPH.

[15]  Andrew Blake,et al.  Probabilistic tracking in a metric space , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  R. Collins,et al.  Representation and matching of articulated shapes , 2004, CVPR 2004.

[17]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Alex Pentland,et al.  Understanding purposeful human motion , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[19]  Wang Liang,et al.  A Survey of Visual Analysis of Human Motion , 2002 .

[20]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[21]  Yunde Jia,et al.  A Miniature Stereo Vision Machine for Real-Time Dense Depth Mapping , 2003, ICVS.