AUTOMATIC ADAPTATION OF A HUMAN FACE MODEL FOR MODEL-BASED CODING

For coding of videophone sequences at very low bit rates, model-based coding is investigated. In a model-based coder, the human face in the videophone sequence is described by a three-dimensional (3D) face model. At the beginning of the videophone sequence, the model has to be adapted automatically to the shape, position and orientation of the real face present in the scene. In this paper, a new face model adaptation algorithm is presented. Facial features such as eye corners, mouth corners, chin and cheek borders, and nose corners are supposed to be known or previously estimated. First, the orientation, shape and position of the model are adapted using the positions of the mouth corners, outer corners of the eyes and top pixels of the cheek borders. Then, the shape of the model is locally adapted using the chin and cheek borders as well as the nose corners. The algorithm was implemented and applied to the first 40 frames of the videophone sequence Claire (CIF, 10Hz) with very encouraging results, even in cases when the person’s eyes are closed or the person is facing downward.

[1]  Kiyoharu Aizawa,et al.  Model-based analysis synthesis image coding (MBASIC) system for a person's face , 1989, Signal Process. Image Commun..

[2]  Kiyoharu Aizawa,et al.  Analysis and synthesis of facial image sequences in model-based image coding , 1994, IEEE Trans. Circuits Syst. Video Technol..

[3]  Bülent Sankur,et al.  Facial feature localization and adaptation of a generic face model for model-based coding , 1995, Signal Process. Image Commun..

[4]  Jörn Ostermann,et al.  Automatic adaptation of a face model in a layered coder with an object-based analysis-synthesis layer and a knowledge-based layer , 1997, Signal Process. Image Commun..

[5]  Hans Georg Musmann.  A layered coding system for very low bit rate video coding , 1995, Signal Process. Image Commun..

[6]  Kiyoharu Aizawa,et al.  Model-based image coding advanced video coding techniques for very low bit-rate applications , 1995, Proc. IEEE.

[7]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Markus Kampmann.  Automatic 3-D face model adaptation for model-based coding of videophone sequences , 2002, IEEE Trans. Circuits Syst. Video Technol..

[9]  G. Matrinez.  Maximum-likelihood motion estimation of a human face , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[10]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Liang Zhang,et al.  Automatic adaptation of a face model using action units for semantic coding of videophone sequences , 1998, IEEE Trans. Circuits Syst. Video Technol..

[12]  D. E. Pearson,et al.  Developments in model-based video coding , 1995, Proc. IEEE.

[13]  Geovanni Martinez,et al.  FACIAL FEATURE EXTRACTION BASED ON THE SMALLEST UNIVALUE SEGMENT ASSIMILATING NUCLEUS ( SUSAN ) ALGORITHM , 2004 .

[14]  Markus Kampmann,et al.  Content-based Coding of Videophone Sequences Using Automatic Face Detection , 1997 .

[15]  Bill Welsh,et al.  Model-based image coding , 1990 .

[16]  Geovanni Martinez.  Improving the speed of convergence of a maximum-likelihood motion estimation algorithm of a human face , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[17]  Markus Kampmann,et al.  Precise Face Model Adaptation for Semantic Coding of Videophone Sequences , 1997 .

[18]  Haibo Li,et al.  Image sequence coding at very low bit rates: a review , 1994, IEEE Trans. Image Process..