3D human face reconstruction using principal components spaces

In this work we propose a new method of 3D face computational photography based on a facial expressions training dataset composed of both facial range images (3D geometry) and facial texture (2D photo). The method allows to obtain a 3D representation of facial geometry given only a 2D photo and a set of facial landmarks, which undergoes a series of transformations through the estimated texture and geometry spaces. In the training stage, principal component analysis is used to represent the face dataset, thus defining an orthonormal basis of texture and another of geometry. In the reconstruction stage, an input is given by a 2D face image and their corresponding landmarks. This data is fed to the PCA basis transform, and a 3D version of the 2D input is built. Several tests using a 3D faces dataset, together with the adoption of a metric, show good results in the 3D facial reconstruction. Additionally, we explored two applications related to the facial expressions transferring and caricaturization. The results of these applications show a rapid and simple synthesis of new 3D models with new expressions and exaggerated facial proportions, useful for 3D facial animation. Results and demos are available at www.vision.ime.usp.br/~jmena/projects/3Dface Keywords-3D face reconstruction; principal components analysis; computer graphics