Automatic Face Modeling and Synthesis Based on Image Pairs

Unlike traditional 3D model based or image based animation methods, in this paper a novel approach is presented to generate both facial actions and head rotations for photo-realistic facial animation based on one frontal and one half-profile facial image taken with an uncalibrated camera. We represent faces with 2D wire-frame models and use MPEG4 FAPs to encode basic facial actions. Hierarchical Direct Appearance Model is employed for facial feature localization. 3D deformable model is applied for pose estimation. By affine projection 3D deformable model and facial actions are mapped to 2D facial models and actions at various head poses. Coarse 2D models are refined with extracted facial features by RBF interpolation. Pose-variable facial animation is generated by synthesizing facial actions on 2D models and morphing facial textures between frontal and halfprofile views. Experimental results demonstrate the effectiveness of our approach.

[1]  Lijun Yin,et al.  Generating Realistic Facial Expressions with Wrinkles for Model-Based Coding , 2001, Comput. Vis. Image Underst..

[2]  Stan Z. Li,et al.  Direct appearance models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Jerry L. Prince,et al.  Snakes, shapes, and gradient vector flow , 1998, IEEE Trans. Image Process..

[4]  Fabio Lavagetto,et al.  The facial animation engine: toward a high-level interface for the design of MPEG-4 compliant animated faces , 1999, IEEE Trans. Circuits Syst. Video Technol..

[5]  Pascal Fua,et al.  Animated Heads from Ordinary Images: A Least-Squares Approach , 1999, Comput. Vis. Image Underst..

[6]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.

[7]  Hans Peter Graf,et al.  Sample-based synthesis of photo-realistic talking heads , 1998, Proceedings Computer Animation '98 (Cat. No.98EX169).

[8]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[9]  Ulrich Neumann,et al.  CoArt: coarticulation region analysis for control of 2D characters , 2002, Proceedings of Computer Animation 2002 (CA 2002).

[10]  Fadi Dornaika,et al.  Face model adaptation using robust matching and active appearance models , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[11]  Tony Ezzat,et al.  Trainable videorealistic speech animation , 2002, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[12]  Yu Zhang,et al.  A new physical model with multilayer architecture for facial expression animation using dynamic adaptive mesh , 2004, IEEE Transactions on Visualization and Computer Graphics.

[13]  Won-Sook Lee,et al.  Generating a population of animated faces from pictures , 1999, Proceedings IEEE International Workshop on Modelling People. MPeople'99.

[14]  David Salesin,et al.  Resynthesizing facial animation through 3D model-based tracking , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  A. Murat Tekalp,et al.  Face and 2-D mesh animation in MPEG-4 , 2000, Signal Process. Image Commun..

[16]  Tony Ezzat,et al.  Facial analysis and synthesis using image-based models , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[17]  Xiaozhou Wei,et al.  A Real Time Face Tracking And Animation System , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  TerzopoulosDemetri,et al.  Geometry-Driven Photorealistic Facial Expression Synthesis , 2006 .

[19]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[20]  Tony Ezzat,et al.  Visual Speech Synthesis by Morphing Visemes , 2000, International Journal of Computer Vision.

[21]  Gang Song,et al.  Hierarchical direct appearance model for elastic labeled graph localization , 2003, International Symposium on Multispectral Image Processing and Pattern Recognition.

[22]  Thomas Vetter,et al.  Synthesis of Novel Views from a Single Face Image , 1998, International Journal of Computer Vision.