3D Face Model Fitting Method based on Active Appearance Model with 3D Depth Estimation

Special cameras (e.g. 3D scanners or depth cameras) are needed to recognize 3D shape information from input faces. In this paper, we propose an efficient face fitting method which can fit various 3D faces including various poses (the rotation of X, Y axies) and facial expressions. Our method takes advantage of 2D Active Appearance Models (AAM) from 2D face images rather than the depth information measured by special cameras. The proposed method combines 2D face model with depth information, thereby the poses of the input faces are not depend on the training data. We construct an AAM for the variation of facial expressions. Then, we estimate depth information of each land mark from frontal, side view images. By combining the estimated depth information with AAM, we can fit various 3D faces. Self-occlusions due to the 3D pose variation are also processed by the region weighting function on the normalized face at each frame. Our proposed method can fit on various face poses which are not trained. Our experimental results show that the proposed method can efficiently fit various faces better than the typical AAM and View-based AAM.