CuteChat: a lightweight tele-immersive video chat system

This paper presents a lightweight tele-immersive video chat system named CuteChat. Based on our recently developed video object cutout technology, the CuteChat system is designed and optimized to provide a radically new video chat experience by merging each participant in the same shared space, allowing them to interact more naturally in an integrated manner. With the goal to make the system easily accessible by massive consumers, we address the challenges in the whole pipeline of video processing, coding, communication, composition, and playback. Extensive experiments have shown that the proposed CuteChat system runs reliably and comfortably in real time on one's laptop or desktop PC, and it needs only a commodity webcam for video acquisition and just public Internet for tele-immersive video conferencing. With such a really minimal deployment requirement, we present a variety of interesting applications and user experiences created by the CuteChat system.

[1]  Ming-Ting Sun,et al.  A Standard-Compliant Virtual Meeting System with Active Video Object Tracking , 2002, EURASIP J. Adv. Signal Process..

[2]  Ketan Mayer-Patel,et al.  Coordinated multi-streaming for 3D tele-immersion , 2004, MULTIMEDIA '04.

[3]  Harry Shum,et al.  Background Cut , 2006, ECCV.

[4]  Andrew Blake,et al.  Bi-layer segmentation of binocular stereo video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Ruzena Bajcsy,et al.  TEEVE: the next generation architecture for tele-immersive environments , 2005, Seventh IEEE International Symposium on Multimedia (ISM'05).

[6]  Ruzena Bajcsy,et al.  New digital options in geographically distributed dance collaborations with TEEVE: tele-immersive environments for everybody , 2007, ACM Multimedia.

[7]  Klara Nahrstedt,et al.  MobileTI: a portable tele-immersive system , 2009, ACM Multimedia.

[8]  A. Criminisi,et al.  Bilayer Segmentation of Live Video , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).