3D stereo reconstruction of human faces driven by differential constraints

Abstract Conventional stereo algorithms often fail in accurately reconstructing a 3D object because the image data do not provide enough information about the geometry of the object. We propose a way to incorporate a priori information in a reconstruction process from a sequence of calibrated face images. A 3D mesh modeling the face is iteratively deformed in order to minimize an energy function. Differential information extracted from the object shape is used to generate an adaptive mesh. We also propose to explicitly incorporate a priori constraints related to the differential properties of the surface where the image information cannot yield an accurate shape recovery.

[1]  Frédéric Hecht,et al.  Anisotropic adaptive mesh generation in two dimensions for CFD , 1996 .

[2]  Manfredo P. do Carmo,et al.  Differential geometry of curves and surfaces , 1976 .

[3]  H. Saunders.  Book Reviews : The Finite Element Method (Revised): O.C. Zienkiewicz McGraw-Hill Book Co., New York, New York , 1980 .

[4]  Alexis Gourdon,et al.  Computing the Differential Characteristics of Isointensity Surfaces , 1995, Comput. Vis. Image Underst..

[5]  Pascal Fua,et al.  Animated Heads from Ordinary Images: A Least-Squares Approach , 1999, Comput. Vis. Image Underst..

[6]  Pascal Fua,et al.  Imposing Hard Constraints on Soft Snakes , 1996, ECCV.

[7]  Pascal Fua,et al.  Using differential constraints to reconstruct complex surfaces from stereo , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Pascal Fua,et al.  Using crest lines to guide surface reconstruction from stereo , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[9]  Nicholas Ayache,et al.  From voxel to curvature , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Olivier D. Faugeras,et al.  Computing differential properties of 3-D shapes from stereoscopic images without 3-D models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Pascal Fua,et al.  From Regular Images to Animated Heads: A Least Squares Approach , 1998, ECCV.

[12]  Walter M. Neuenschwander.  Elastic deformable contour and surface models for 2-D and 3-D image segmentation , 1995 .

[13]  J. Z. Zhu,et al.  The finite element method , 1977 .

[14]  O. Monga,et al.  Using partial Derivatives of 3D images to extract typical surface features , 1992, Proceedings of the Third Annual Conference of AI, Simulation, and Planning in High Autonomy Systems 'Integrating Perception, Planning and Action'..