Relating intensities with three-dimensional facial shape using partial least squares

The authors apply partial least squares regression to predict three-dimensional (3D) face shape from a single image. PLS describes the relationship between independent (intensity images) and dependent (3D shape) variables by seeking directions in the space of independent variables that are associated with large variations in the space of dependent variables. We use this idea to construct statistical models of intensity and 3D shape that capture strongly linked variations in both spaces. This decomposition leads to the construction of two different models that capture common variations in 3D shape and intensity. Using the intensity model, a set of parameters is obtained from out-of-training intensity examples. These intensity parameters can then be used directly in the 3D shape model to approximate facial shape. Experiments show that prediction is achieved with reasonable accuracy, improving results obtained through canonical correlation analysis.

[1]  H. Abdi.  Partial Least Square Regression PLS-Regression , 2007 .

[2]  Dehua Li,et al.  A structured light-based system for human heads , 2004 .

[3]  Abhijeet Ghosh,et al.  Practical modeling and acquisition of layered facial reflectance , 2008, SIGGRAPH 2008.

[4]  Edwin R. Hancock,et al.  Recovering Facial Shape Using a Statistical Model of Surface Normal Direction , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[7]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[8]  A. Johnston,et al.  Recognising Faces: Effects of Lighting Direction, Inversion, and Brightness Reversal , 1992, Perception.

[9]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[11]  Edwin R. Hancock,et al.  New Constraints on Data-Closeness and Needle Map Consistency for Shape-from-Shading , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[13]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[14]  H. Knutsson,et al.  A Unified Approach to PCA, PLS, MLR and CCA , 1997 .

[15]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[16]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[17]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[18]  Edwin R. Hancock,et al.  A Coupled Statistical Model for Face Shape Recovery From Brightness Images , 2007, IEEE Transactions on Image Processing.

[19]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Johan A. K. Suykens,et al.  Subset based least squares subspace regression in RKHS , 2005, Neurocomputing.