Enhanced 3D geometric-model-based face tracking in low resolution with appearance model

3D geometric-model-based face tracking utilizes 3D geometric face model to extract facial motion information from video. It has the advantages of dealing with oblique views of faces and it is robust to changes in people, expressions and lighting. Although most existing 3D model-based methods work well with high resolution faces (e.g. face regions around 200 /spl times/ 200 pixels), the face image in real environments (e.g. surveillance video) is often in low resolution. In low resolution, the performance of 3D model-based face tracking can be degraded because the underlying low-level image motion estimation is less reliable. In this paper, we present an appearance-based enhancement to 3D geometric-model-based face tracking in low resolution. First, we estimate the motion parameters of the 3D face model. Then we extract the stabilized texture map image of the face model. The appearance variations in the face texture images are caused by the errors in motion estimation or lighting changes. Therefore these appearance variations are used as additional constraints for refining the motion estimations. Experiments show that the proposed enhancement improves the robustness of 3D model-based tracking in low resolution.

[1]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[2]  Irfan Essa,et al.  Visual Coding and Tracking of Speech Related Facial Motion , 2001, CVPR 2001.

[3]  Kiyoharu Aizawa,et al.  MODEL-BASED VIDEO CODING , 1996 .

[4]  N. Magnenat-Thalmann,et al.  Automatic face cloning and animation using real-time facial feature tracking and speech acquisition , 2001, IEEE Signal Process. Mag..

[5]  David Salesin,et al.  Resynthesizing facial animation through 3D model-based tracking , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Kiyoharu Aizawa,et al.  Model-based image coding , 1994, Other Conferences.

[9]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[10]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Dimitris N. Metaxas,et al.  Generation, estimation and tracking of faces , 1998 .

[12]  Ying-li Tian,et al.  Evaluation of Face Resolution for Expression Analysis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[13]  平山亮.  会議報告－Speechreading by Humans and Machines; Models Systems and Applications , 1997 .

[14]  Thomas S. Huang,et al.  Capturing subtle facial motions in 3D face tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  Peter Eisert,et al.  Model-aided coding: a new approach to incorporate facial animation into motion-compensated video coding , 2000, IEEE Trans. Circuits Syst. Video Technol..