3D and Infrared Face Reconstruction from RGB data using Canonical Correlation Analysis

In this paper, we apply a multiple regression method based on canonical correlation analysis (CCA) to face data modelling. CCA is a factor analysis method which exploits the correlation between two high dimensional signals. We first use CCA to perform 3D face reconstruction and in a separate application we predict near-infrared (NIR) face texture. In both cases, the input data are color (RGB) face images. Experiments show, that due to the correlation between input and output signal, only a small number of canonical factors are needed to describe the functional relation of RGB images to the respective output (NIR images and 3D depth maps) with reasonable accuracy

[1]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Daniel Snow,et al.  Determining Generative Models of Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability , 1999, International Journal of Computer Vision.

[3]  J. Friedman,et al.  Predicting Multivariate Responses in Multiple Linear Regression , 1997 .

[4]  Sami Romdhani,et al.  Face identification across different poses and illuminations with a 3D morphable model , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[5]  Hans Knutsson,et al.  Learning multidimensional signal processing , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[6]  A. Höskuldsson.  PLS regression methods , 1988 .

[7]  Horst Bischof,et al.  Appearance models based on kernel canonical correlation analysis , 2003, Pattern Recognit..

[8]  John A. Robinson,et al.  Estimation of Face Depths by Conditional Densities , 2005, BMVC.

[9]  Petre Stoica,et al.  Optimal reduced-rank estimation and filtering , 2001, IEEE Trans. Signal Process..

[10]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[12]  M.R. Azimi-Sadjadi,et al.  Underwater target classification using canonical correlations , 2003, Oceans 2003. Celebrating the Past ... Teaming Toward the Future (IEEE Cat. No.03CH37492).

[13]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[14]  Rama Chellappa,et al.  Illumination-insensitive face recognition using symmetric shape-from-shading , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  R. Basri,et al.  Statistical Symmetric Shape from Shading for 3D Structure Recovery of Faces , 2004, eccv 2004.

[16]  Sun-Yuan Kung,et al.  Principal Component Neural Networks: Theory and Applications , 1996 .

[17]  Juha Karhunen,et al.  Principal component neural networks — Theory and applications , 1998, Pattern Analysis and Applications.