Model-based 3D face capture with shape-from-silhouettes

We present a method for 3D face acquisition using a set or sequence of 2D binary silhouettes. Since silhouette images depend only on the shape and pose of an object, they are immune to lighting and/or texture variations (unlike feature or texture-based shape-from-correspondence). Our prior 3D face model is a linear combination of "eigenheads" obtained by applying PCA to a training set of laser-scanned 3D faces. These shape coefficients are the parameters for a near-automatic system for capturing the 3D shape as well as the 2D texture-map of a novel input face. Specifically, we use back-projection and a boundary-weighted XOR-based cost function for binary silhouette matching, coupled with a probabilistic "downhill-simplex" optimization for shape estimation and refinement. Experiments with a multicamera rig as well as monocular video sequences demonstrate the advantages of our 3D modeling framework and ultimately, its utility for robust face recognition with built-in invariance to pose and illumination.

[1]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[2]  J AtickJoseph,et al.  Statistical approach to shape from shading , 1996 .

[3]  Gregory M. Nielson,et al.  Scattered data modeling , 1993, IEEE Computer Graphics and Applications.

[4]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[5]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[7]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[8]  Aldo Grattarola.  Volumetric reconstruction from object silhouettes: A regularization procedure , 1992, Signal Process..

[9]  Bernd Girod,et al.  Silhouette-Based Multiple-View Camera Calibration , 2000, VMV.

[10]  Paolo Cignoni,et al.  Multiple Texture Stitching and Blending on 3D Objects , 1999, Rendering Techniques.

[11]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[12]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[13]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[14]  Richard Szeliski,et al.  Recovering 3D shape and motion from image streams using nonlinear least squares , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  J. Atick,et al.  Statistical Approach to Shape from Shading : Reconstruction of 3 D Face Surfaces from Single 2 D , 1997 .

[16]  Hans-Peter Seidel,et al.  Texturing Faces , 2002, Graphics Interface.

[17]  Hans-Peter Seidel,et al.  Automated texture registration and stitching for real world models , 2000, Proceedings the Eighth Pacific Conference on Computer Graphics and Applications.

[18]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[19]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[20]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[21]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[22]  Thomas Vetter,et al.  Estimating Coloured 3D Face Models from Single Images: An Example Based Approach , 1998, ECCV.