Facial Geometry Estimation Using Photometric Stereo and Profile Views

This paper presents a novel method for estimating the three-dimensional shape of faces, facilitating the possibility of enhanced face recognition. The method involves a combined use of photometric stereo and profile view information. It can be divided into three principal stages: (1) An initial estimate of the face is obtained using four-source high-speed photometric stereo. (2) The profile is determined from a side-view camera. (3) The facial shape estimation is iteratively refined using the profile until an energy functional is minimised. This final stage, which is the most important contribution of the paper, works by continually deforming the shape estimate so that its profile is exact. An energy is then calculated based on the difference between the raw images and synthetic images generated using the new shape estimate. The surface normals are then adjusted according to energy until convergence. Several real face reconstructions are presented and compared to ground truth. The results clearly demonstrate a significant improvement in accuracy compared to standard photometric stereo.

[1]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[2]  Daniel Rueckert,et al.  Evaluation of automatic 4D face recognition using surface and texture registration , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[3]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[7]  Rama Chellappa,et al.  Face Processing: Advanced Modeling and Methods , 2006, J. Electronic Imaging.

[8]  Maja Pantic,et al.  Facial action recognition for facial expression analysis from static face images , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[10]  Yanxi Liu,et al.  Facial asymmetry quantification for expression invariant human identification , 2003, Comput. Vis. Image Underst..

[11]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[12]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[13]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Roberto Cipolla,et al.  Multiview Photometric Stereo , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[16]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Rama Chellappa,et al.  Symmetric Shape-from-Shading Using Self-ratio Image , 2001, International Journal of Computer Vision.

[18]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.