Facial Animation and Analysis Using 2D+3D Facial Motion Tracking

This paper presents a facial animation system using real-time tracking of 3D facial motions from a depth camera. We first applied 2D facial motion tracking based on extended Active Shape Models (ASMs) from 2D texture image corresponding to captured 3D depth information. Based on the estimated feature point tracking from extended 2D facial motion tracking, 3D facial motions are estimated. From the estimated 3D facial motion using extended ASMs, we extract MPEG-4 facial animation parameters (FAPs) from the 3D facial motion tracking, which provides more accurate estimation of FAPs invariant to view variations. Facial animations can be achieved by any facial animation tools supporting facial animation based on FAPs.

[1]  Massimo Tistarelli,et al.  Biometrics and Identity Management , 2008, Lecture Notes in Computer Science.

[2]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[3]  Zhigang Deng,et al.  Data-Driven 3D Facial Animation , 2007 .

[4]  A. Murat Tekalp,et al.  Estimation and Analysis of Facial Animation Parameter Patterns , 2007, 2007 IEEE International Conference on Image Processing.

[5]  Qiang Ji,et al.  Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, SIGGRAPH 2005.

[7]  Sotiris Malassiotis,et al.  Real-time 2D+3D facial action and expression recognition , 2010, Pattern Recognit..

[8]  Arman Savran,et al.  Comparative evaluation of 3D vs. 2D modality for automatic detection of facial action units , 2012, Pattern Recognit..

[9]  Algirdas Pakstas,et al.  MPEG-4 Facial Animation: The Standard,Implementation and Applications , 2002 .

[10]  Tieniu Tan,et al.  Affective Computing: A Review , 2005, ACII.

[11]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[12]  Nikolaos Grammalidis,et al.  FAP extraction using three-dimensional motion estimation , 2002, IEEE Trans. Circuits Syst. Video Technol..

[13]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..