Blue-c: a spatially immersive display and 3D video portal for telepresence

We present blue-c, a new generation immersive projection and 3D video acquisition environment for virtual design and collaboration. It combines simultaneous acquisition of multiple live video streams with advanced 3D projection technology in a CAVETMlike environment, creating the impression of total immersion. The blue-c portal currently consists of three rectangular projection screens that are built from glass panels containing liquid crystal layers. These screens can be switched from a whitish opaque state (for projection) to a transparent state (for acquisition), which allows the video cameras to “look through” the walls. Our projection technology is based on active stereo using two LCD projectors per screen. The projectors are synchronously shuttered along with the screens, the stereo glasses, active illumination devices, and the acquisition hardware. From multiple video streams, we compute a 3D video representation of the user in real time. The resulting video inlays are integrated into a networked virtual environment. Our design is highly scalable, enabling blue-c to connect to portals with less sophisticated hardware.

[1]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[2]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[4]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[5]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[6]  Markus H. Gross,et al.  3D video fragments: dynamic point samples for real-time free-viewpoint video , 2004, Comput. Graph..

[7]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[8]  Bernd Fröhlich,et al.  The Responsive Workbench: A Virtual Work Environment , 1995, Computer.

[9]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[10]  John Rohlf,et al.  IRIS performer: a high performance multiprocessing toolkit for real-time 3D graphics , 1994, SIGGRAPH.

[11]  Andrew Gardner,et al.  A lighting reproduction approach to live-action compositing , 2002, SIGGRAPH.

[12]  Sean Hayes,et al.  View synthesis by edge transfer with application to the generation of immersive video objects , 1998, VRST '98.

[13]  V. Rajan.  A Realistic Video Avatar System for Networked Virtual Environments , 2002 .

[14]  Markus H. Gross,et al.  The blue-c distributed scene graph , 2003, IEEE Virtual Reality, 2003. Proceedings..

[15]  Takeo Kanade,et al.  Spatio-Temporal View Interpolation , 2002, Rendering Techniques.

[16]  Kostas Daniilidis,et al.  View-independent scene acquisition for tele-presence , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[17]  Carl M. Lampert,et al.  The world of large-area glazing and displays , 1999, Optics & Photonics.

[18]  Ken Perlin,et al.  An autostereoscopic display , 2000, SIGGRAPH.

[19]  Oliver Schreer,et al.  An immersive 3D video-conferencing system using shared virtual team user environments , 2002, CVE '02.

[20]  Markus H. Gross,et al.  3D video recorder , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[21]  Markus H. Gross,et al.  Spatialized audio rendering for immersive virtual environments , 2002, VRST '02.

[22]  Ramesh C. Jain,et al.  Immersive video , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[23]  Christian Breiteneder,et al.  TELEPORT – Towards immersive copresence , 1999, Multimedia Systems.

[24]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[25]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  Matthias Zwicker,et al.  Surface splatting , 2001, SIGGRAPH.