Continuous capture of skin deformation

We describe a method for the acquisition of deformable human geometry from silhouettes. Our technique uses a commercial tracking system to determine the motion of the skeleton, then estimates geometry for each bone using constraints provided by the silhouettes from one or more cameras. These silhouettes do not give a complete characterization of the geometry for a particular point in time, but when the subject moves, many observations of the same local geometries allow the construction of a complete model. Our reconstruction algorithm provides a simple mechanism for solving the problems of view aggregation, occlusion handling, hole filling, noise removal, and deformation modeling. The resulting model is parameterized to synthesize geometry for new poses of the skeleton. We demonstrate this capability by rendering the geometry for motion sequences that were not included in the original datasets.

[1]  Ramakant Nevatia,et al.  Description and Recognition of Curved Objects , 1977, Artif. Intell..

[2]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[4]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[5]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[6]  Peter-Pike J. Sloan,et al.  Shape by example , 2001, I3D '01.

[7]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[8]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[9]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[10]  Ioannis A. Kakadiaris,et al.  3D human body model acquisition from multiple views , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  Takeo Kanade,et al.  Spatio-Temporal View Interpolation , 2002, Rendering Techniques.

[12]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Lorenzo Torresani,et al.  Space-Time Tracking , 2002, ECCV.

[14]  Hans-Peter Seidel,et al.  Combining 2d Feature Tracking And Volume Reconstruction For Online Video-Based Human Motion Capture , 2004, Int. J. Image Graph..

[15]  Steven J. Gortler,et al.  Silhouette Mapping , 1999 .

[16]  Zoran Popovic,et al.  Articulated body deformation from range scan data , 2002, SIGGRAPH.

[17]  Jean-Christophe Nebel,et al.  Stroboscopic stereo rangefinder , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[18]  R. Plankers,et al.  Articulated soft objects for video-based body modeling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[19]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[20]  Markus H. Gross,et al.  3D video recorder , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[21]  John P. Lewis,et al.  Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation , 2000, SIGGRAPH.

[22]  Mohan M. Trivedi,et al.  Human Body Model Acquisition and Tracking Using Voxel Data , 2003, International Journal of Computer Vision.

[23]  Cary B. Phillips,et al.  Multi-weight enveloping: least-squares approximation techniques for skin animation , 2002, SCA '02.

[24]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[25]  Pascal Fua,et al.  Articulated Soft Objects for Video-based Body Modeling , 2001, ICCV.

[26]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[27]  Richard Szeliski,et al.  Surface modeling with oriented particle systems , 1992, SIGGRAPH.

[28]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.