Depth Estimation of Face Images Using the Nonlinear Least-Squares Model

In this paper, we propose an efficient algorithm to reconstruct the 3D structure of a human face from one or more of its 2D images with different poses. In our algorithm, the nonlinear least-squares model is first employed to estimate the depth values of facial feature points and the pose of the 2D face image concerned by means of the similarity transform. Furthermore, different optimization schemes are presented with regard to the accuracy levels and the training time required. Our algorithm also embeds the symmetrical property of the human face into the optimization procedure, in order to alleviate the sensitivities arising from changes in pose. In addition, the regularization term, based on linear correlation, is added in the objective function to improve the estimation accuracy of the 3D structure. Further, a model-integration method is proposed to improve the depth-estimation accuracy when multiple nonfrontal-view face images are available. Experimental results on the 2D and 3D databases demonstrate the feasibility and efficiency of the proposed methods.

[1]  Edwin R. Hancock,et al.  A Coupled Statistical Model for Face Shape Recovery From Brightness Images , 2007, IEEE Transactions on Image Processing.

[2]  Edwin R. Hancock,et al.  Acquiring height data from a single image of a face using local shape indicators , 2006, Comput. Vis. Image Underst..

[3]  Julia Kastner,et al.  Introduction to Robust Estimation and Hypothesis Testing , 2005 .

[4]  NairPrathap,et al.  3-D face detection, landmark localization, and registration using a point distribution model , 2009 .

[5]  Hong Yan,et al.  Locating and extracting the eye in human face images , 1996, Pattern Recognit..

[6]  Kin-Man Lam,et al.  Recovering the 3D shape and poses of face images based on the similarity transform , 2008, Pattern Recognit. Lett..

[7]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[8]  Fernand S. Cohen,et al.  3-D face structure extraction and recognition from images using 3-D morphing and distance mapping , 2002, IEEE Trans. Image Process..

[9]  Sami Romdhani,et al.  Efficient, robust and accurate fitting of a 3D morphable model , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Adrian G. Bors,et al.  Robust Processing of Optical Flow of Fluids , 2010, IEEE Transactions on Image Processing.

[11]  Jose Gonzalez-Mora,et al.  Efficient image alignment using linear appearance models , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Jörgen Ahlberg,et al.  CANDIDE-3 - An Updated Parameterised Face , 2001 .

[14]  Aleix M. Martínez,et al.  Rigid Structure from Motion from a Blind Source Separation Perspective , 2009, International Journal of Computer Vision.

[15]  Andrea Thelen,et al.  Improvements in Shape-From-Focus for Holographic Reconstructions With Regard to Focus Operators, Neighborhood-Size, and Height Value Interpolation , 2009, IEEE Transactions on Image Processing.

[16]  Rama Chellappa,et al.  Statistical bias in 3-D reconstruction from a monocular video , 2005, IEEE Transactions on Image Processing.

[17]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Li Shang,et al.  An improved constrained ICA with reference based unmixing matrix initialization , 2010, Neurocomputing.

[19]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Fernando De la Torre,et al.  Learning a generic 3D face model from 2D image databases using incremental Structure-from-Motion , 2010, Image Vis. Comput..

[21]  Andrea Cavallaro,et al.  3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model , 2009, IEEE Transactions on Multimedia.

[22]  Zhan-Li Sun.  An Extension of MISEP for Post–Nonlinear–Linear Mixture Separation , 2009, IEEE Transactions on Circuits and Systems II: Express Briefs.

[23]  Fernando De la Torre,et al.  Bilinear Active Appearance Models , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[24]  Thomas F. Coleman,et al.  An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds , 1993, SIAM J. Optim..

[25]  Aaron Hertzmann,et al.  Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Martin D. Levine,et al.  State-of-the-art of 3D facial reconstruction methods for face recognition based on a single 2D training image per person , 2009, Pattern Recognit. Lett..

[27]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[28]  Kin-Man Lam,et al.  Elastic shape-texture matching for human face recognition , 2008, Pattern Recognit..

[29]  Philip H. Ramsey.  Nonparametric Statistical Methods , 1974, Technometrics.

[30]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Wen Gao,et al.  Efficient 3D reconstruction for face recognition , 2005, Pattern Recognit..

[32]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[33]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[34]  Adrian G. Bors,et al.  Smoothing of optical flow using robustified diffusion kernels , 2010, Image Vis. Comput..

[35]  S. Stigler.  Do Robust Estimators Work with Real Data , 1977 .

[36]  Takeo Kanade,et al.  Trajectory Space: A Dual Representation for Nonrigid Structure from Motion , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.