Automated body modeling from video sequences

Synthetic modeling of human bodies and the simulation of motion is a long-standing problem in animation and much work is involved before a near-realistic performance can be achieved. At present, it takes an experienced designer a very long time to build a complete and realistic model that closely resembles a specific person. Our ultimate goal is to automate the process and to produce realistic animation models given a set of video sequences. In this paper we show that, given video sequences of a person moving in front of the camera, we can recover shape information and joint locations. Both of which are essential to instantiate a complete and realistic model that closely resembles a specific person and without knowledge about the position of the articulations a character cannot be animated. This is achieved with minimal human intervention. The recovered shape and motion parameters can be used to reconstruct the original movement or to allow other animation models to mimic the subject's actions.

[1]  Michel Dhome,et al.  Human Body Tracking by Monocular Vision , 1996, ECCV.

[2]  Ioannis A. Kakadiaris,et al.  Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Wei Sun,et al.  Virtual people: capturing human models to populate virtual worlds , 1999, Proceedings Computer Animation 1999.

[4]  James F. Blinn,et al.  A generalization of algebraic surface drawing , 1982, SIGGRAPH.

[5]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[6]  Hans-Gerd Maas.  Image sequence based automatic multi-camera system calibration techniques 1 Revised version of a pap , 1999 .

[7]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Daniel Thalmann,et al.  Interactive Shape Design Using Metaballs and Splines , 1995 .

[9]  Baba C. Vemuri,et al.  Shape Modeling with Front Propagation: A Level Set Approach , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Pascal Fua,et al.  From Image Synthesis to Image Analysis: Using Human Animation Models to Guide Feature Extraction , 1998 .

[11]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[12]  A. Gruen.  ADAPTIVE LEAST SQUARES CORRELATION: A POWERFUL IMAGE MATCHING TECHNIQUE , 1985 .

[13]  Ioannis A. Kakadiaris,et al.  Active part-decomposition, shape and motion estimation of articulated objects: a physics-based approach , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Daniel Thalmann,et al.  Fast realistic human body deformations for animation and VR applications , 1996, Proceedings of CG International '96.

[15]  James W. Davis,et al.  A Robust Human-Silhouette Extraction Technique for Interactive Virtual Environments , 1998, CAPTECH.

[16]  Pascal Fua,et al.  Human Body Modeling and Motion Analysis From Video Sequences , 1998 .