Immersive 3D video conferencing: challenges, concepts, and implementation

In this paper, a next generation 3-D video conferencing system is presented that provides immersive tele-presence and natural representation of all participants in a shared virtual meeting space. The system is based on the principle of a shared virtual table environment which guarantees correct eye contact and gesture reproduction and enhances the quality of human-centered communication. The virtual environment is modeled in MPEG-4 which also allows the seamless integration of explicit 3-D head models for a low-bandwidth connection to mobile users. In this case, facial expression and motion information is transmitted instead of video streams resulting in bit-rates of a few kbit/s per participant. Beside low bit-rates, the model-based approach enables new possibilities for image enhancements like digital make-up, digital dressing, or modification of scene lighting.

[1]  Olov Ståhl,et al.  Meetings for real—experiences from a series of VR-based project meetings , 1999, VRST '99.

[2]  Peter Eisert,et al.  Model-based enhancement of lighting conditions in image sequences , 2002, IS&T/SPIE Electronic Imaging.

[3]  Milton Chen,et al.  Design of a virtual auditorium , 2001, MULTIMEDIA '01.

[4]  Terumasa Aoki,et al.  MONJU no CHIE SYSTEM : VIDEOCONFERENCE SYSTEM WITH EYE CONTACT FOR DECISION MAKING , 1999 .

[5]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[6]  Amnon Shashua,et al.  Novel view synthesis in tensor space , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Greg Welch,et al.  Toward a compelling sensation of telepresence: demonstrating a portal to a distant (static) office , 2000 .

[8]  Bang Jun Lei,et al.  Multi-step View Synthesis with Occlusion Handling , 2001, VMV.

[9]  Oliver Schreer,et al.  Virtual team user environments - a step from tele-cubicles towards distributed tele-collaboration in mediated workspaces , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[10]  D. E. Pearson,et al.  Developments in model-based video coding , 1995, Proc. IEEE.

[11]  Peter Eisert,et al.  Very low bit rate video coding using 3-D models , 2001 .

[12]  Oliver Schreer,et al.  Real-time disparity analysis for applications in immersive teleconference scenarios-a comparative study , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[13]  Christian Breiteneder,et al.  TELEPORT – Towards immersive copresence , 1999, Multimedia Systems.

[14]  Oliver Schreer,et al.  An immersive 3D video-conferencing system using shared virtual team user environments , 2002, CVE '02.

[15]  Steve Benford,et al.  Crowded collaborative virtual environments , 1997, CHI.

[16]  Peter Eisert,et al.  Analyzing Facial Expressions for Virtual Conferencing , 1998, IEEE Computer Graphics and Applications.

[17]  Oliver Schreer,et al.  Fast and robust shadow detection in videoconference applications , 2002, International Symposium on VIPromCom Video/Image Processing and Multimedia Communications.

[18]  Bill Welsh,et al.  Model-based image coding , 1990 .

[19]  V. Rajan.  A Realistic Video Avatar System for Networked Virtual Environments , 2002 .