Capturing and animating skin deformation in human motion

During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture system and approximately 350 markers. Although the number of markers is significantly larger than that used in conventional motion capture, it is only a sparse representation of the true shape of the body. We supplement this sparse sample with a detailed, actor-specific surface model. The motion of the skin can then be computed by segmenting the markers into the motion of a set of rigid parts and a residual deformation (approximated first as a quadratic transformation and then with radial basis functions). We demonstrate the power of this approach by capturing flexing muscles, high frequency motions, and abrupt decelerations on several actors. We compare these results both to conventional motion capture and skinning and to synchronized video of the actors.

[1]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[2]  Richard E. Parent,et al.  Layered construction for deformable animated characters , 1989, SIGGRAPH.

[3]  Kenichi Kanatani,et al.  Analysis of 3-D Rotation Fitting , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Harry Shum,et al.  Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jane Wilhelms,et al.  Anatomically based modeling , 1997, SIGGRAPH.

[6]  Wayne E. Carlson,et al.  Anatomy-based modeling of the human musculature , 1997, SIGGRAPH.

[7]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[8]  Daniel Thalmann,et al.  Anatomic modeling of deformable human bodies , 2000, The Visual Computer.

[9]  Evangelos Kokkevis,et al.  Skinning Characters using Surface Oriented Free-Form Deformations , 2000, Graphics Interface.

[10]  John P. Lewis,et al.  Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation , 2000, SIGGRAPH.

[11]  Michael J. Black,et al.  Dynamic coupled component analysis , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Nadia Magnenat-Thalmann,et al.  Principal components of expressive speech animation , 2001, Proceedings. Computer Graphics International 2001.

[13]  Jeng-Sheng Yeh.  Extracting 3D facial animation parameters from multiview video clips - Computer Graphics and Applications, IEEE , 2001 .

[14]  Hyeong-Seok Ko,et al.  Performance-driven muscle-based facial animation , 2001, Comput. Animat. Virtual Worlds.

[15]  Wei Sun,et al.  Layered animation of captured data , 2001, The Visual Computer.

[16]  Peter-Pike J. Sloan,et al.  Shape by example , 2001, I3D '01.

[17]  Cary B. Phillips,et al.  Multi-weight enveloping: least-squares approximation techniques for skin animation , 2002, SCA '02.

[18]  Gordon Clapworthy,et al.  An Anatomy-Based Approach to Human Muscle Modeling and Deformation , 2002, IEEE Trans. Vis. Comput. Graph..

[19]  Zoran Popovic,et al.  Articulated body deformation from range scan data , 2002, SIGGRAPH.

[20]  Seong-Whan Lee,et al.  Reconstruction of Partially Damaged Face Images Based on a Morphable Face Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Nadia Magnenat-Thalmann,et al.  Synthesizing animatable body models with parameterized shape modifications , 2003, SCA '03.

[22]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[23]  Victor B. Zordan,et al.  Mapping optical motion capture data to skeletal motion using a physical model , 2003, SCA '03.

[24]  Hans-Peter Seidel,et al.  Construction and animation of anatomically based human hand models , 2003, SCA '03.

[25]  Nadia Magnenat-Thalmann,et al.  An automatic modeling of human bodies from sizing parameters , 2003, I3D '03.

[26]  Jing Xiao,et al.  Vision-based control of 3D facial animation , 2003, SCA '03.

[27]  Jovan Popovic,et al.  Continuous capture of skin deformation , 2003, ACM Trans. Graph..

[28]  Michael Gleicher,et al.  Building efficient, accurate character skins from examples , 2003, ACM Trans. Graph..

[29]  Tsuneya Kurihara,et al.  Modeling deformable human hands from medical images , 2004, SCA '04.

[30]  Breathe easy: model and control of simulated respiration for animation , 2004, SCA '04.

[31]  Ahmed M. Elgammal,et al.  High Resolution Acquisition, Learning and Transfer of Dynamic 3‐D Facial Expressions , 2004, Comput. Graph. Forum.

[32]  Towards perceptually realistic talking heads: models, methods and McGurk , 2004, APGV '04.

[33]  Ulrich Neumann,et al.  Analysis of co‐articulation regions for performance‐driven facial animation , 2004, Comput. Animat. Virtual Worlds.

[34]  Li Zhang,et al.  Spacetime faces: high resolution capture for modeling and animation , 2004, SIGGRAPH 2004.

[35]  Sebastian Thrun,et al.  Recovering Articulated Object Models from 3D Range Data , 2004, UAI.

[36]  Ronald Fedkiw,et al.  Creating and simulating skeletal muscle from the visible human data set , 2005, IEEE Transactions on Visualization and Computer Graphics.

[37]  Ronald Fedkiw,et al.  Automatic determination of facial muscle activations from sparse motion capture marker data , 2005, SIGGRAPH '05.

[38]  Ronald Fedkiw,et al.  Robust quasistatic finite elements and flesh simulation , 2005, SCA '05.

[39]  Ming Ouhyoung,et al.  Mirror MoCap: Automatic and efficient capture of dense 3D facial motion parameters from video , 2005, The Visual Computer.

[40]  Chun-Fa Chang,et al.  Key Probe: a technique for animation keyframe extraction , 2005, The Visual Computer.

[41]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, SIGGRAPH '05.

[42]  Matthias Zwicker,et al.  Mesh-based inverse kinematics , 2005, ACM Trans. Graph..

[43]  Shaogang Gong,et al.  2D statistical models of facial expressions for realistic 3D avatar animation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  D. Levin,et al.  Linear rotation-invariant coordinates for meshes , 2005, ACM Trans. Graph..

[45]  Markus H. Gross,et al.  Meshless deformations based on shape matching , 2005, ACM Trans. Graph..

[46]  Zheng Guo,et al.  Skinning With Deformable Chunks , 2005, Comput. Graph. Forum.

[47]  Bruno Arnaldi,et al.  Dynamic skinning: adding real-time dynamic effects to an existing character animation , 2005, SCCG '05.

[48]  Jon G. Rokne,et al.  Modeling and simulating the deformation of human skeletal muscle based on anatomy and physiology , 2005, Comput. Animat. Virtual Worlds.

[49]  Georges Dumont,et al.  Morphological and stance interpolations in database for simulating bipedalism of virtual humans , 2005, The Visual Computer.

[50]  Heinrich H. Bülthoff,et al.  Psychophysical evaluation of animated facial expressions , 2005, APGV '05.

[51]  Jirí Zára,et al.  Spherical blend skinning: a real-time deformation of articulated models , 2005, I3D '05.

[52]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, ACM Trans. Graph..

[53]  Karan Singh,et al.  Outside-in anatomy based character rigging , 2005, SCA '05.

[54]  Dragomir Anguelov,et al.  SCAPE: shape completion and animation of people , 2005, ACM Trans. Graph..

[55]  Doug L. James,et al.  Skinning mesh animations , 2005, ACM Trans. Graph..

[56]  Christoph Bregler,et al.  Mood swings: expressive speech animation , 2005, TOGS.

[57]  Bert Jüttler,et al.  Sweep-based human deformation , 2005, The Visual Computer.

[58]  Daniel Thalmann,et al.  Virtual humans: thirty years of research, what next? , 2005, The Visual Computer.