2D/3D Model-Based Facial Video Coding/Decoding at Ultra-Low Bit-Rate

For facial video coding/decoding in mobile communication, a 2D/3D model-based system was proposed: (1)Online appearance model and cylinder head model were combined to track 3D facial motion in particle filter; (2)3D facial animation was produced combining parameterized model and muscular model; (3)3D hair was synthesized by hair detection and 3D hair model; (4)3D coding/decoding of foreground and 2D coding/decoding of background were stitched. At ultra-low bit-rate, the object experiment confirmed the advantage between coding efficiency and decoding quality of it, and the between subjects experiment indicated the suitability of subjective face identification by it.

[1]  Dani Lischinski,et al.  Spectral Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  CaniMarie-Paule,et al.  A Survey on Hair Modeling , 2007 .

[3]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[4]  Meng Wang,et al.  Movie2Comics: Towards a Lively Video Content Presentation , 2012, IEEE Transactions on Multimedia.

[5]  Jing Xiao,et al.  Robust full‐motion recovery of head by dynamic templates and re‐registration techniques , 2003 .

[6]  Qiang Wang,et al.  Real Time Feature Based 3-D Deformable Face Tracking , 2008, ECCV.

[7]  Steve Marschner,et al.  A Survey on Hair Modeling: Styling, Simulation, and Rendering , 2007, IEEE Transactions on Visualization and Computer Graphics.

[8]  Jörgen Ahlberg.  Model-based coding : extraction, coding, and evaluation of face model parameters , 2002 .

[9]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[10]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Larry S. Davis,et al.  Detection and analysis of hair , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Peter Eisert,et al.  Model-aided coding: a new approach to incorporate facial animation into motion-compensated video coding , 2000, IEEE Trans. Circuits Syst. Video Technol..

[13]  David J. Fleet,et al.  Robust Online Appearance Models for Visual Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  P. Grassberger.  Pruned-enriched Rosenbluth method: Simulations of θ polymers of chain length up to 1 000 000 , 1997 .

[15]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[16]  Vincent Lepetit,et al.  An Investigation of Model Bias in 3D Face Tracking , 2005, AMFG.

[17]  Pheng-Ann Heng,et al.  A Physically-Based Modeling and Simulation Framework for Facial Animation , 2009, 2009 Fifth International Conference on Image and Graphics.

[18]  Keith Waters,et al.  Computer facial animation , 1996 .

[19]  Harry Shum,et al.  Real-Time Bayesian 3-D Pose Tracking , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Thomas S. Huang,et al.  Capturing subtle facial motions in 3D face tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  Simon Baker,et al.  2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting , 2007, International Journal of Computer Vision.

[22]  Sheng-De Wang,et al.  A virtual teleconferencing system based on face detection and 3D animation in a low‐bandwidth environment , 2010, Int. J. Imaging Syst. Technol..

[23]  Fabio Pianesi,et al.  Xface open source project and smil-agent scripting language for creating and animating embodied conversational agents , 2007, ACM Multimedia.

[24]  L. Darrell Whitley,et al.  Adaptive Appearance Model and Condensation Algorithm for Robust Face Tracking , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[25]  Fadi Dornaika,et al.  Simultaneous Facial Action Tracking and Expression Recognition in the Presence of Head Motion , 2008, International Journal of Computer Vision.

[26]  Eduardo Zalama Casanova,et al.  A realistic facial animation suitable for human-robot interfacing , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Shaogang Gong,et al.  Analysis and Modelling of Faces and Gestures , 2008 .

[28]  Tomaso A. Poggio,et al.  Reanimating Faces in Images and Video , 2003, Comput. Graph. Forum.