Human-Centered Face Computing in Multimedia Interaction and Communication

Facial image computing has been an extensively studied topic since its wide applications in human-centered computing, human-avatar interaction, virtual reality, and multimedia communication. Successful systems have been equipped with realistic face models, efficient compression algorithms, reliable animation techniques and user friendly interaction schemes. In this chapter, we will mainly focus on techniques, algorithms, models, applications and real-world systems. Comprehensive summarization of the state-of-the-art works will be presented as well as our experiences and contributions in the field, especially several real prototype systems developed in our group, such as the online interactive gaming system hMouse, humanoid emotive audio-visual avatar, and 3D face/head tracking based video compression. Performances of these three systems are also illustrated based on standard evaluations.

[1]  Yun Fu,et al.  EAVA: A 3D Emotive Audio-Visual Avatar , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[2]  Kikuo Fujimura,et al.  A robust elliptical head tracker , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[3]  Itu-T.  Video coding for low bitrate communication , 1996 .

[4]  Vincent Lepetit,et al.  Fusing online and offline information for stable 3D tracking in real-time , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Keith Waters,et al.  Computer facial animation , 1996 .

[6]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.

[7]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Gregory D. Hager,et al.  A Particle Filter without Dynamics for Robust 3D Face Tracking , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[9]  Thomas S. Huang,et al.  iFACE: A 3D Synthetic Talking Face , 2001, Int. J. Image Graph..

[10]  Alex Pentland,et al.  Recursive estimation of structure and motion using relative orientation constraints , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Bill Welsh,et al.  Model-based image coding , 1990 .

[12]  C. G. Fisher,et al.  Confusions among visually perceived consonants. , 1968, Journal of speech and hearing research.

[13]  Gerhard Roth,et al.  Nouse 'Use Your Nose as a Mouse' - a New Technology for Hands-free Games and Interfaces , 2002 .

[14]  Li Zhiguo,et al.  Animating 3D facial models with MPEG-4 FaceDefTables , 2002, Proceedings 35th Annual Simulation Symposium. SS 2002.

[15]  Yu-Luen Chen,et al.  Application of tilt sensors in human-computer mouse interface for people with disabilities. , 2001, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[16]  Markus Kampmann.  Automatic 3-D face model adaptation for model-based coding of videophone sequences , 2002, IEEE Trans. Circuits Syst. Video Technol..

[17]  Wen-Hsiang Tsai,et al.  Determination of Head Pose and Facial Expression from a Single Perspective View by Successive Scaled Orthographic Approximations , 2002, International Journal of Computer Vision.

[18]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[19]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[20]  Dimitris N. Metaxas,et al.  Optical Flow Constraints on Deformable Models with Applications to Face Tracking , 2000, International Journal of Computer Vision.

[21]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[22]  Thomas S. Huang,et al.  Visual face tracking and its applications , 2007 .

[23]  Keith Waters,et al.  A muscle model for animation three-dimensional facial expression , 1987, SIGGRAPH.

[24]  Margrit Betke,et al.  Communication via eye blinks - detection and duration analysis in real time , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[25]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[26]  Tim Morris,et al.  Facial feature tracking for cursor control , 2006, J. Netw. Comput. Appl..

[27]  Thomas S. Huang,et al.  Face as mouse through visual face tracking , 2007, Comput. Vis. Image Underst..

[28]  Seong G. Kong,et al.  A Survey on 3D Modeling of Human Faces for Face Recognition , 2007 .

[29]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[30]  Thomas S. Huang,et al.  Face localization via hierarchical CONDENSATION with Fisher boosting feature selection , 2004, CVPR 2004.

[31]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[32]  Thomas S. Huang,et al.  3D Face Processing: Modeling, Analysis and Synthesis , 2004 .

[33]  Peter Eisert,et al.  Model-aided coding: a new approach to incorporate facial animation into motion-compensated video coding , 2000, IEEE Trans. Circuits Syst. Video Technol..

[34]  Hans Georg Musmann.  A layered coding system for very low bit rate video coding , 1995, Signal Process. Image Commun..

[35]  P Blenkhorn,et al.  Controlling mouse pointer position using an infrared head-operated joystick. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[36]  Qiang Zhang,et al.  A NURBS Facial Model Based on MPEG-4 , 2006, 16th International Conference on Artificial Reality and Telexistence--Workshops (ICAT'06).

[37]  Kentaro Toyama,et al.  “Look, Ma – No Hands!” Hands-Free Cursor Control with Real-Time 3D Face Tracking , 1998 .

[38]  Matthew Brand,et al.  Flexible flow for 3D nonrigid tracking and shape recovery , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[40]  M. Betke,et al.  The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[41]  Timothy F. Cootes,et al.  Active Shape Model Search using Local Grey-Level Models: A Quantitative Evaluation , 1993, BMVC.

[42]  Thomas S. Huang,et al.  Natural Mouse-a novel human computer interface , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[43]  Yun Fu,et al.  Humanoid Audio–Visual Avatar With Emotive Text-to-Speech Synthesis , 2008, IEEE Transactions on Multimedia.

[44]  A. Mehrabian.  Communication without words , 1968 .

[45]  Yuxiao Hu,et al.  Real-time conversion from a single 2D face image to a 3D text-driven emotive audio-visual avatar , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[46]  Wolfram Schiffmann,et al.  Head pose estimation of partially occluded faces , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[47]  Quan Pan,et al.  Reliable and fast tracking of faces under varying pose , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[48]  Yun Fu,et al.  hMouse: Head Tracking Driven Virtual Computer Mouse , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[49]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[50]  Itu-T and Iso Iec Jtc.  Advanced video coding for generic audiovisual services , 2010 .

[51]  Thomas S. Huang,et al.  Two-stage prosody prediction for emotional text-to-speech synthesis , 2008, INTERSPEECH.

[52]  Jiayan Jiang,et al.  Building a 3D Morphable Face Model by Using Thin Plate Splines for Face Reconstruction , 2004, SINOBIOMETRICS.

[53]  Sharath Pankanti,et al.  Advances in Biometric Person Authentication, International Workshop on Biometric Recognition Systems, IWBRS2005, Beijing, China, October 22-23, 2005, Proceedings , 2005, IWBRS.

[54]  Thomas S. Huang,et al.  Real-time speech-driven face animation with expressions using neural networks , 2002, IEEE Trans. Neural Networks.

[55]  Thomas S. Huang,et al.  Face as mouse through visual face tracking , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[56]  D. Massaro.  Perceiving talking faces: from speech perception to a behavioral principle , 1999 .

[57]  Larry S. Davis,et al.  Model-Based Object Pose in 25 Lines of Code , 1992, ECCV.

[58]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[59]  Kiyoharu Aizawa,et al.  Model-based image coding advanced video coding techniques for very low bit-rate applications , 1995, Proc. IEEE.

[60]  Alfredo Gardel Vicente,et al.  Commands generation by face movements applied to the guidance of a wheelchair for handicapped people , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[61]  Gary R. Bradski,et al.  Real time face and object tracking as a component of a perceptual user interface , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).