Real-time markerless human body tracking with multi-view 3-d voxel reconstruction.

We present an approach to full human-body tracking, using markerless multiview images as input, performing acquisition, reconstruction and tracking in real-time on a single PC. Our approach employs a hierarchical visual-hull algorithm which segments only the most interesting regions of the images and includes colour information. The tracking step uses blobs attached to a kinematic model to recover joint angles in an expectation-maximization framework. We demonstrate the robustness of the approach on video sequences of various body configurations in an unaugmented office environment. We also show that tracking challenging poses with self-occlusions is possible without the processing cost of stochastic sampling schemes.

[1]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[2]  Larry S. Davis,et al.  Towards 3-D model-based tracking and recognition of human movement: a multi-view approach , 1995 .

[3]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[4]  Charles Q. Little,et al.  Real-Time Tracking of Articulated Human Models Using a 3D Shape-from-Silhouette Method , 2001, RobVis.

[5]  Olivier D. Faugeras,et al.  3D articulated models and multi-view tracking with silhouettes , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Richard Szeliski,et al.  Real-Time Octree Generation from Rotating Objects , 1990 .

[7]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Hans-Peter Seidel,et al.  Combining 2D feature tracking and volume reconstruction for online video-based human motion capture , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[11]  Chris Welman,et al.  INVERSE KINEMATICS AND GEOMETRIC CONSTRAINTS FOR ARTICULATED FIGURE MANIPULATION , 1993 .

[12]  Adrian Hilton,et al.  Simultaneous Pose Estimation of Multiple People using Multiple-View Cues with Hierarchical Sampling , 2003, BMVC.

[13]  Simon Baker,et al.  Visual hull construction, alignment and refinement for human kinematic modeling, motion tracking and rendering , 2003 .

[14]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[16]  Mohan M. Trivedi,et al.  Human Body Model Acquisition and Tracking Using Voxel Data , 2003, International Journal of Computer Vision.