Real-time avatar animation from a single image

A real time facial puppetry system is presented. Compared with existing systems, the proposed method requires no special hardware, runs in real time (23 frames-per-second), and requires only a single image of the avatar and user. The user's facial expression is captured through a real-time 3D non-rigid tracking system. Expression transfer is achieved by combining a generic expression model with synthetically generated examples that better capture person specific characteristics. Performance of the system is evaluated on avatars of real people as well as masks and cartoon characters.

[1]  Ira Kemelmacher-Shlizerman,et al.  Being John Malkovich , 2010, ECCV.

[2]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[3]  Minh Hoai.  Local Minima Free Parameterized Appearance Models , 2008 .

[4]  Simon Lucey,et al.  Face alignment through subspace constrained mean-shifts , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Jing Xiao,et al.  A Closed-Form Solution to Non-rigid Shape and Motion Recovery , 2004, ECCV.

[6]  Timothy F. Cootes,et al.  Feature Detection and Tracking with Constrained Local Models , 2006, BMVC.

[7]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[8]  Fabio Lavagetto,et al.  The facial animation engine: toward a high-level interface for the design of MPEG-4 compliant animated faces , 1999, IEEE Trans. Circuits Syst. Video Technol..

[9]  Roland Göcke,et al.  Monocular and Stereo Methods for AAM Learning from Video , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Jeffrey R. Spies,et al.  Mapping and Manipulating Facial Expression , 2009, Language and speech.

[11]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[12]  HansenPer Christian.  The truncated SVD as a method for regularization , 1987 .

[13]  Fernando De la Torre,et al.  Bilinear Kernel Reduced Rank Regression for Facial Expression Synthesis , 2010, ECCV.

[14]  Roland Göcke,et al.  A Nonlinear Discriminative Approach to AAM Fitting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Luc Van Gool,et al.  Face/Off: live facial puppetry , 2009, SCA '09.

[16]  C. Taylor,et al.  Active shape models - 'Smart Snakes'. , 1992 .

[17]  Roland Göcke,et al.  Automatic frontal face annotation and AAM building for arbitrary expressions from a single frontal image only , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).