Instant Messenger with Personalized 3D Avatar

Instant Messengers (IM) have become popular online chat tools in cyber worlds. The existing IMs mainly rely on text, audio, and video for user conversation and communication. The pure text chat is monotonous, while the video chat only occurs when users have their webcoms installed and wish to see each other. In this paper, we propose a new IM with personalized 3D Avatars, and present a prototype of this system. Through our IM, the user can synthesize a personalized 3D avatar by just inputting a 2D self face image. Expressions of the avatar are synthesized by a group of predefined phonemes and visemes, and driven by a text-to-visual speech engine. Moreover, our system supports 3D avatar decoration. During online chat, text is input through the text-to-visual speech engine to drive the generation of voice and animation for the 3D avatar. Tests have demonstrated that our IM with personalized 3D avatars can enhance the fun and vividness of online chat experience.

[1]  Wen Gao,et al.  Learning and synthesizing MPEG-4 compatible 3-D face animation from video sequence , 2003, IEEE Trans. Circuits Syst. Video Technol..

[2]  Ming-Hui Wen,et al.  Body and mind: a study of avatar personalization in three virtual worlds , 2009, CHI.

[3]  Ahmet M. Kondoz,et al.  Automatic Single View-Based 3-D Face Synthesis for Unsupervised Multimedia Applications , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Jian-Huang Lai,et al.  Virtual view face image synthesis using 3D spring-based face model from a single image , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[5]  Mitsuru Ishizuka,et al.  User Study of AffectIM, an Emotionally Intelligent Instant Messaging System , 2008, IVA.

[6]  Thomas S. Huang,et al.  iFACE: A 3D Synthetic Talking Face , 2001, Int. J. Image Graph..

[7]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[8]  Shinn-Ying Ho,et al.  Facial modeling from an uncalibrated face image using a coarse-to-fine genetic algorithm , 2001, Pattern Recognit..

[9]  Engin Erzin,et al.  Comparison of Phoneme and Viseme Based Acoustic Units for Speech Driven Realistic Lip Animation , 2007 .

[10]  Peter Eisert,et al.  Analyzing Facial Expressions for Virtual Conferencing , 1998, IEEE Computer Graphics and Applications.

[11]  Zhigang Deng,et al.  A Text-Driven Conversational Avatar Interface for Instant Messaging on Mobile Devices , 2013, IEEE Transactions on Human-Machine Systems.

[12]  Sin-Hwa Kang,et al.  Communicators' Perceptions of Social Presence as a Function of Avatar Realism in Small Display Mobile Communication Devices , 2008, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[13]  Jörn Ostermann,et al.  3D talking head customization by adapting a generic model to one uncalibrated picture , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[14]  Nadia Magnenat-Thalmann,et al.  Facial feature extraction for quick 3D face modeling , 2002, Signal Process. Image Commun..

[15]  Pascal Fua,et al.  3D stereo reconstruction of human faces driven by differential constraints , 2000, Image Vis. Comput..

[16]  Ruey-Song Huang,et al.  3-D facial model estimation from single front-view facial image , 2002, IEEE Trans. Circuits Syst. Video Technol..