Whole-body modelling of people from multiview images to populate virtual worlds

In this paper a new technique is introduced for automatically building recognisable, moving 3D models of individual people. A set of multiview colour images of a person is captured from the front, sides and back by one or more cameras. Model-based reconstruction of shape from silhouettes is used to transform a standard 3D generic humanoid model to approximate a person's shape and anatomical structure. Realistic appearance is achieved by colour texture mapping from the multiview images. The results show the reconstruction of a realistic 3D facsimile of the person suitable for animation in a virtual world. The system is inexpensive and is reliable for large variations in shape, size and clothing. This is the first approach to achieve realistic model capture for clothed people and automatic reconstruction of animated models. A commercial system based on this approach has recently been used to capture thousands of models of the general public.

[1]  R. Plankers,et al.  Automated body modeling from video sequences , 1999, Proceedings IEEE International Workshop on Modelling People. MPeople'99.

[2]  Jochen Wingbermühle,et al.  Automatic reconstruction of 3D objects using a mobile monoscopic camera , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[3]  Demetri Terzopoulos.  From Physics-Based Representation to Functional Modeling of Highly Complex Objects , 1994, Object Representation in Computer Vision.

[4]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[5]  Wei Sun,et al.  Virtual people: capturing human models to populate virtual worlds , 1999, Proceedings Computer Animation 1999.

[6]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[7]  Akshay K. Singh,et al.  Deformable models in medical image analysis , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[8]  Nadia Magnenat-Thalmann,et al.  Head Modeling from Pictures and Morphing in 3D with Image Metamorphosis Based on Triangulation , 1998, CAPTECH.

[9]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.

[10]  Jennifer J. Whitestone,et al.  Automatic Detection, Identification, and Registration of Anatomical Landmarks , 1995 .

[11]  S. Weik,et al.  Highly realistic modeling of persons for 3D videoconferencing systems , 1997, Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing.

[12]  S. McConnell,et al.  Virtual conferencing , 1997 .

[13]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[14]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[15]  Tsuneya Kurihara,et al.  A Transformation Method for Modeling and Animation of the Human Face from Photographs , 1991 .

[16]  Saied Moezzi,et al.  Virtual View Generation for 3D Digital Video , 1997, IEEE Multim..

[17]  Frank P. Ferrie,et al.  Finding the Parts of Objects in Range Images , 1996, Comput. Vis. Image Underst..

[18]  Dimitris N. Metaxas,et al.  Combining information using hard constraints , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[19]  Pascal Fua,et al.  Animated Heads from Ordinary Images: A Least-Squares Approach , 1999, Comput. Vis. Image Underst..

[20]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[21]  Matthew Stone,et al.  An anthropometric face model using variational techniques , 1998, SIGGRAPH.

[22]  Steven Paquette,et al.  3D scanning in apparel design and human engineering , 1996, IEEE Computer Graphics and Applications.

[23]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[24]  Helen C. Shen,et al.  A 3D Reconstruction System for Human Body Modeling , 1998, CAPTECH.

[25]  Nadia Magnenat-Thalmann,et al.  Automatic 3D cloning and real-time animation of a human face , 1997, Proceedings. Computer Animation '97 (Cat. No.97TB100120).

[26]  Dimitris N. Metaxas,et al.  The integration of optical flow and deformable models with applications to human face shape and motion estimation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[28]  Pascal Fua,et al.  Face Models from Uncalibrated Video Sequences , 1998, CAPTECH.

[29]  Demetri Terzopoulos,et al.  Constraints on Deformable Models: Recovering 3D Shape and Nonrigid Motion , 1988, Artif. Intell..

[30]  Adrian Hilton,et al.  3D scene representation using a deformable surface , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[31]  Bf Buxton,et al.  Models for understanding the 3D human body form , 1998, ICCV 1998.

[32]  Takaaki Akimoto,et al.  Automatic creation of 3D facial models , 1993, IEEE Computer Graphics and Applications.

[33]  Nadia Magnenat-Thalmann,et al.  A user-friendly texture-fitting methodology for virtual humans , 1997, Proceedings Computer Graphics International.

[34]  Ioannis A. Kakadiaris,et al.  Vision-based animation of digital humans , 1998, Proceedings Computer Animation '98 (Cat. No.98EX169).