Using differential constraints to generate a 3D face model from stereo

We propose a way to incorporate a priori information in a 3D stereo reconstruction process from a pair of calibrated face images. A 3D mesh modeling the surface is iteratively deformed in order to minimize an energy function. Differential information about the object shape is used to generate an adaptive mesh that can fulfil the compacity and the accuracy requirements. Moreover in areas where the stereo information is not reliable enough to accurately recover the surface shape, because of inappropriate texture or bad lighting conditions, we incorporate geometric constraints related to the differential properties of the surface, that can be intuitive or refer to predefined geometric properties of the object to be reconstructed. They can be applied to scalar fields, such as curvature values, or structural features, such as crest lines. Therefore, we generate a 3D face model using computer vision techniques that is compact, accurate and consistent with the a priori knowledge about the underlying surface.

[1]  Alexis Gourdon,et al.  Computing the Differential Characteristics of Isointensity Surfaces , 1995, Comput. Vis. Image Underst..

[2]  Pascal Fua,et al.  Imposing Hard Constraints on Soft Snakes , 1996, ECCV.

[3]  Pascal Fua,et al.  Using crest lines to guide surface reconstruction from stereo , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[4]  Frédéric Hecht,et al.  Anisotropic adaptive mesh generation in two dimensions for CFD , 1996 .

[5]  Rachid Deriche,et al.  Extraction of the zero-crossings of the curvature derivatives in volumic 3D medical images: a multi-scale approach , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jean-Philippe Tarel,et al.  From 2D images to 3D face geometry , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[7]  Olivier Monga,et al.  Using Partial Derivatives of 3D Images to Extract Typical Surface Features , 1995, Comput. Vis. Image Underst..

[8]  Walter M. Neuenschwander.  Elastic deformable contour and surface models for 2-D and 3-D image segmentation , 1995 .

[9]  Manfredo P. do Carmo,et al.  Differential geometry of curves and surfaces , 1976 .

[10]  J. Z. Zhu,et al.  The finite element method , 1977 .

[11]  Pascal Fua,et al.  Using differential constraints to reconstruct complex surfaces from stereo , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Jian L. Zhou,et al.  User's Guide for CFSQP Version 2.0: A C Code for Solving (Large Scale) Constrained Nonlinear (Minimax) Optimization Problems, Generating Iterates Satisfying All Inequality Constraints , 1994 .

[13]  Olivier D. Faugeras,et al.  Computing differential properties of 3-D shapes from stereoscopic images without 3-D models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.