Modeling and Animating Virtual Humans for Real-Time Applications

We report on the workflow for the creation of realistic virtual anthropomorphic characters. 3D-models of human heads have been reconstructed from real people by following a structured light approach to 3D-reconstruction. We describe how these high-resolution models have been simplified and articulated with blend shape and mesh skinning techniques to ensure real-time animation. The full-body models have been created manually based on photographs. We present a system for capturing whole body motions, including the fingers, based on an optical motion capture system with 6 DOF rigid bodies and cybergloves. The motion capture data was processed in one system, mapped to a virtual character and visualized in real-time. We developed tools and methods for quick post processing. To demonstrate the viability of our system, we captured a library consisting of more than 90 gestures.

[1]  Pat Hanrahan,et al.  Reflection from layered surfaces due to subsurface scattering , 1993, SIGGRAPH.

[2]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[3]  Frederick I. Parke,et al.  Computer gernerated animation of faces , 1998 .

[4]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[5]  Steve Marschner,et al.  Filling holes in complex surfaces using volumetric diffusion , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[6]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[7]  José Remo Ferreira Brega,et al.  A motion capture system using passive markers , 2004, VRCAI '04.

[8]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[9]  Jason Osipa.  Stop Staring: Facial Modeling and Animation Done Right , 2003 .

[10]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[11]  Daniel Thalmann,et al.  Handbook of Virtual Humans , 2004 .

[12]  Leif Kobbelt,et al.  Self-calibrating optical motion tracking for articulated bodies , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[13]  Klaus Dorfmüller-Ulhaas.  Robust Optical User Motion Tracking Using a Kalman Filter , 2005 .

[14]  Jessica K. Hodgins,et al.  Automatic Joint Parameter Estimation from Magnetic Motion Capture Data , 2023, Graphics Interface.

[15]  Frederick I. Parke,et al.  Computer generated animation of faces , 1972, ACM Annual Conference.

[16]  Robert van Liere,et al.  Search Space Reduction in Optical Tracking , 2003, IPT/EGVE.

[17]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.

[18]  H.J.C.M. Sterenborg,et al.  Skin optics , 1989, IEEE Transactions on Biomedical Engineering.

[19]  Chengzhi Jiang,et al.  Optical properties of skin , 2002, SPIE/COS Photonics Asia.

[20]  Pascal Fua,et al.  Skeleton-based motion capture for robust reconstruction of human motion , 2000, Proceedings Computer Animation 2000.

[21]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[22]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Alberto Menache,et al.  Understanding Motion Capture for Computer Animation and Video Games , 1999 .

[24]  Gladimir V. G. Baranoski,et al.  A Study on Skin Optics , 2004 .

[25]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.