Data streaming in telepresence environments

In this paper, we discuss data transmission in telepresence environments for collaborative virtual reality applications. We analyze data streams in the context of networked virtual environments and classify them according to their traffic characteristics. Special emphasis is put on geometry-enhanced (3D) video. We review architectures for real-time 3D video pipelines and derive theoretical bounds on the minimal system latency as a function of the transmission and processing delays. Furthermore, we discuss bandwidth issues of differential update coding for 3D video. In our telepresence system - the blue-c - we use a point-based 3D video technology which allows for differentially encoded 3D representations of human users. While we discuss the considerations which lead to the design of our three-stage 3D video pipeline, we also elucidate some critical implementation details regarding decoupling of acquisition, processing and rendering frame rates, and audio/video synchronization. Finally, we demonstrate the communication and networking features of the blue-c system in its full deployment. We show how the system can possibly be controlled to face processing or networking bottlenecks by adapting the multiple system components like audio, application data, and 3D video.

[1]  Steve Benford,et al.  Coping with inconsistency due to network delays in collaborative virtual environments , 1999, VRST '99.

[2]  Frederick P. Brooks.  What's Real About Virtual Reality? , 1999, IEEE Computer Graphics and Applications.

[3]  Hirokazu Kato,et al.  3-D live: real time interaction for mixed reality , 2002, CSCW '02.

[4]  Christian Breiteneder,et al.  TELEPORT – Towards immersive copresence , 1999, Multimedia Systems.

[5]  Michael Zyda,et al.  NPSNET:A Network Software Architecture for LargeScale Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[6]  Markus H. Gross,et al.  3D video fragments: dynamic point samples for real-time free-viewpoint video , 2004, Comput. Graph..

[7]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[8]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[9]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[10]  Steve Benford,et al.  MASSIVE: a collaborative virtual environment for teleconferencing , 1995, TCHI.

[11]  Luc Van Gool,et al.  Blue-c: a spatially immersive display and 3D video portal for telepresence , 2003, IPT/EGVE.

[12]  Markus H. Gross,et al.  The blue-c distributed scene graph , 2003, IEEE Virtual Reality, 2003. Proceedings..

[13]  Oliver Schreer,et al.  An immersive 3D video-conferencing system using shared virtual team user environments , 2002, CVE '02.

[14]  C. M. Greenhalgh,et al.  Benford: MASSIVE: A Virtual Reality System for Tele-conferencing , 1995 .

[15]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[16]  Nobuhiko Kitawaki,et al.  Pure Delay Effects on Speech Quality in Telecommunications , 1991, IEEE J. Sel. Areas Commun..

[17]  Kyoung Shin Park,et al.  Effects of network characteristics on human performance in a collaborative virtual environment , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[18]  Markus H. Gross,et al.  Real-time streaming of point-based 3D video , 2004, IEEE Virtual Reality 2004.

[19]  Rashid Ansari,et al.  Adaptive Networking for Tele-Immersion , 2001, EGVE/IPT.