Generation of optimized facial animation parameters

The MPEG-4 FBA (Face and Body Animation) standard provides a set of FAPs (Facial Animation Parameters) for animating a talking face with its moods and expressions. A face model driven by this set of FAPs can produce high quality animation at a bitrate as low as 2 kb/s. The applications for the face model range from as diverse areas as video phone and video conferencing for wireless, portable units like PDAs (Personal Digital Assistants) and cell phones, to game development and movie production. The success of facial animation relies on being able to track facial features and to generate FAPs accurately and reliably. This thesis presents a method for extracting FAPs from a person's face in a video sequence. The goal is to generate FAPs that make the animated face resemble the original face in the video sequence. The proposed method is based on feedback from the render unit during the FAP generation process. This ensures that the animated face is as close to the original face as possible. A penalty function is derived to measure the resemblance between the animated and the original face. The FAP generation process includes optimizing the penalty function, which consists of a match function and some barrier functions. The match function compares how well an animated face matches the original face in the video sequence. Knowledge about the appearance of a normal looking face is contained in the barrier functions. Each barrier functions indicates the level of distortion from a normal looking face for a certain part of a face, and advises the optimizer. Unnecessary FAPs are eliminated and the search space is partitioned into smaller, independent subspaces to speed up the optimization process. Steepest Descent Method, Cyclic Coordinate Method, Linear Search as well as Golden Section Line Search are applied to obtain an optimal solution. The results show that the generated FAPs are accurate and the proposed method is very robust. The generated FAPs can drive animations that are lifelike and truthful to the original sequence, making them suitable for very high quality applications.

[1]  M. Yachida,et al.  Facial expression recognition and its degree estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Nam Ling,et al.  Optimizing facial animation parameters for MPEG-4 , 2003, IEEE Trans. Consumer Electron..

[3]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Jun Ohya,et al.  Recognizing abruptly changing facial expressions from time-sequential face images , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[5]  Hazem M. El-Bakry,et al.  Automatic face recognition system using neural networks , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[6]  Nam Ling,et al.  On speed optimization of MPEG-4 decoder for real-time multimedia applications , 1999, Proceedings Third International Conference on Computational Intelligence and Multimedia Applications. ICCIMA'99 (Cat. No.PR00300).

[7]  Marco La Cascia,et al.  Fast, reliable head tracking under varying illumination , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Andrew Blake,et al.  Determining facial expressions in real time , 1995, Proceedings of IEEE International Conference on Computer Vision.

[9]  Nam Ling,et al.  MPEG-4 FAP generation as an optimization problem , 2003, 2003 IEEE International Conference on Consumer Electronics, 2003. ICCE..

[10]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[11]  Nam Ling,et al.  An MPEG-4 real-time video decoder software , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[12]  김형곤,et al.  Automatic FDP (Facial Definition Parameters) generation for a video conferencing system , 1999 .

[13]  Lap-Pui Chau,et al.  A real-time realization of MPEG-4 video decoder , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[14]  Takeo Kanade,et al.  Recognizing lower face action units for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[15]  Lance Williams,et al.  Performance-driven facial animation , 1990, SIGGRAPH.

[16]  M.G. Strintzis,et al.  Automatic extraction of facial feature points for MPEG4 videophone applications , 2000, 2000 Digest of Technical Papers. International Conference on Consumer Electronics. Nineteenth in the Series (Cat. No.00CH37102).

[17]  R. E. Kalman,et al.  Optimum Seeking Methods. , 1964 .

[18]  Jörn Ostermann,et al.  Very low bit rate coding of virtual human animation in MPEG-4 , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[19]  Marco La Cascia,et al.  Head tracking via robust registration in texture map images , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[20]  Yong-Moo Kwon,et al.  Automatic FDP/FAP generation from an image sequence , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[21]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[22]  Keith Waters,et al.  Computer facial animation , 1996 .

[23]  Fabio Lavagetto,et al.  The facial animation engine: toward a high-level interface for the design of MPEG-4 compliant animated faces , 1999, IEEE Trans. Circuits Syst. Video Technol..

[24]  Luke S. Zettlemoyer,et al.  Explanatory lifelike avatars: performing user-centered tasks in 3D learning environments , 1999, AGENTS '99.

[25]  Uday B. Desai,et al.  Finding faces in photographs , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  Lijun Yin,et al.  Eye tracking and animation for MPEG-4 coding , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[27]  Ju-Jang Lee,et al.  Face detection and facial feature extraction using color snake , 2002 .

[28]  Nam Ling,et al.  An MPEG-4 facial animation parameters generation system , 2003, 2003 IEEE Workshop on Signal Processing Systems (IEEE Cat. No.03TH8682).

[29]  Tom Davis,et al.  Opengl programming guide: the official guide to learning opengl , 1993 .

[30]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.