3-D studio production of animated actor models

A framework for construction of detailed animated models of an actor's shape and appearance from multiple view images is presented. Multiple views of an actor are captured in a studio with controlled illumination and background. An initial low-resolution approximation of the person's shape is reconstructed by deformation of a generic humanoid model to fit the visual hull using shape constrained optimisation to preserve the surface parameterisation for animation. Stereo reconstruction with multiple view constraints is then used to reconstruct the detailed surface shape. High-resolution shape detail from stereo is represented in a structured format for animation by displacement mapping from the low-resolution model surface. A novel integration algorithm using displacement maps is introduced to combine overlapping stereo surface measurements from multiple views into a single displacement map representation of the high-resolution surface detail. Results of 3-D actor modelling in a 14 camera studio demonstrate improved representation of detailed surface shape such as creases in clothing compared to previous model fitting approaches. Actor models can be animated and rendered from arbitrary views under different illumination to produce free-viewpoint video sequences. The proposed framework enables rapid transformation of captured multiple view images into a structured representation suitable for realistic animation.

[1]  Zoran Popovic,et al.  Articulated body deformation from range scan data , 2002, SIGGRAPH.

[2]  Saied Moezzi,et al.  Virtual View Generation for 3D Digital Video , 1997, IEEE Multim..

[3]  Wei Sun,et al.  Layered animation using displacement maps , 2000, Proceedings Computer Animation 2000.

[4]  Pascal Fua,et al.  Articulated Soft Objects for Video-based Body Modeling , 2001, ICCV.

[5]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[6]  Wei Sun,et al.  Layered animation of captured data , 2001, The Visual Computer.

[7]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[8]  Denis Laurendeau,et al.  A General Surface Approach to the Integration of a Set of Range Views , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Takeo Kanade,et al.  A locally adaptive window for signal matching , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[10]  Wei Sun,et al.  Whole-body modelling of people from multiview images to populate virtual worlds , 2000, The Visual Computer.

[11]  Adrian Hilton,et al.  Model-based multiple view reconstruction of people , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Mark S. Drew,et al.  A spectrally based framework for realistic image synthesis , 2001, The Visual Computer.

[13]  Hugues Hoppe,et al.  Displaced subdivision surfaces , 2000, SIGGRAPH.

[14]  Stan Sclaroff,et al.  Stochastic refinement of the visual hull to satisfy photometric and silhouette consistency constraints , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  Marc Levoy,et al.  The digital Michelangelo project , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[16]  Demetri Terzopoulos.  From Physics-Based Representation to Functional Modeling of Highly Complex Objects , 1994, Object Representation in Computer Vision.

[17]  Adrian Hilton,et al.  Animated statues , 2003, Machine Vision and Applications.

[18]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[19]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[20]  Hans-Peter Seidel,et al.  Interactive multi-resolution modeling on arbitrary meshes , 1998, SIGGRAPH.

[21]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[22]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[23]  Richard Szeliski,et al.  3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Marc Levoy,et al.  Fitting smooth surfaces to dense polygon meshes , 1996, SIGGRAPH.

[25]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[26]  Takeo Kanade,et al.  Visual hull alignment and refinement across time: a 3D reconstruction algorithm combining shape-from-silhouette with stereo , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Adrian Hilton,et al.  Implicit Surface-Based Geometric Fusion , 1998, Comput. Vis. Image Underst..

[28]  Adrian Hilton,et al.  Reconstruction of Animated Models from Images Using Constrained Deformable Surfaces , 2002, DGCI.

[29]  Jan Paul Siebert,et al.  Conformation from generic animatable models to 3D scanned data , 2001 .

[30]  Takeo Kanade,et al.  Spatio-Temporal View Interpolation , 2002, Rendering Techniques.