High-Quality Visualization for Geographically Distributed 3-D Teleimmersive Applications

The growing popularity of 3-D movies has led to the rapid development of numerous affordable consumer 3-D displays. In contrast, the development of technology to generate 3-D content has lagged behind considerably. In spite of significant improvements to the quality of imaging devices, the accuracy of the algorithms that generate 3-D data, and the hardware available to render such data, the algorithms available to calibrate, reconstruct, and then visualize such data remain difficult to use, extremely noise sensitive, and unreasonably slow. In this paper, we present a multi-camera system that creates a highly accurate (on the order of a centimeter), 3-D reconstruction of an environment in real-time (under 30 ms) that allows for remote interaction between users. This paper focuses on addressing the aforementioned deficiencies by describing algorithms to calibrate, reconstruct, and render objects in the system. We demonstrate the accuracy and speed of our results on a variety of benchmarks and data collected from our own system.

[1]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  J. Bailenson,et al.  The Effect of Interactivity on Learning Physical Actions in Virtual Reality , 2008 .

[3]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[4]  Thomas S. Huang,et al.  Technologies for virtual reality/tele—immersion applications: issues of research in image display and global networking , 2001 .

[5]  Thomas Malzbender,et al.  The Coliseum Immersive Teleconferencing System , 2002 .

[6]  Peter Eisert,et al.  3DPresence -A System Concept for Multi-User and Multi-Party Immersive 3D Videoconferencing , 2008 .

[7]  Zhi-Gang Zheng,et al.  A region based stereo matching algorithm using cooperative optimization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[9]  Zeyu Li,et al.  Wide-area external multi-camera calibration using vision graphs and virtual calibration object , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[10]  John F. Canny,et al.  MultiView: spatially faithful group video conferencing , 2005, CHI.

[11]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[12]  Søren Hauberg,et al.  Gaussian-Like Spatial Priors for Articulated Tracking , 2010, ECCV.

[13]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision , 2004 .

[14]  BajcsyRuzena,et al.  Enabling multi-party 3D tele-immersive environments with ViewCast , 2010 .

[15]  Joseph M. Maubach,et al.  Local bisection refinement for $n$-simplicial grids generated by reflection , 2017 .

[16]  Ruzena Bajcsy,et al.  Real-Time 3D Avatars for Tele-rehabilitation in Virtual Reality , 2011, MMVR.

[17]  Margrit Gelautz,et al.  A layered stereo matching algorithm using image segmentation and global visibility constraints , 2005 .

[18]  Luc Van Gool,et al.  Blue-c: a spatially immersive display and 3D video portal for telepresence , 2003, IPT/EGVE.

[19]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[21]  Philipp Slusallek,et al.  Wide area camera calibration using virtual calibration objects , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[22]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Tomás Svoboda,et al.  A Convenient Multicamera Self-Calibration for Virtual Environments , 2005, Presence: Teleoperators & Virtual Environments.

[24]  Ruzena Bajcsy,et al.  Immersive 3D Environment for Remote Collaboration and Training of Physical Activities , 2008, 2008 IEEE Virtual Reality Conference.

[25]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[26]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[27]  Kostas Daniilidis,et al.  Real time trinocular stereo for tele-immersion , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[28]  Yoshihiko Nomura,et al.  Error analysis and optimization of camera calibration , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[29]  Gregorij Kurillo,et al.  Cyberarchaeology: Experimenting with teleimmersive archaeology , 2010, 2010 16th International Conference on Virtual Systems and Multimedia.

[30]  François X. Sillion,et al.  A Real-Time System for Full Body Interaction with Virtual Worlds , 2004, EGVE.

[31]  Takeo Kanade,et al.  Virtualized Reality : Digitizing a 3D Time-Varying Event As Is and in Real Time , 1999 .

[32]  Ruzena Bajcsy,et al.  A Framework for Constructing Real-time Immersive Environments for Training Physical Activities , 2006, J. Multim..

[33]  Marcus A. Magnor,et al.  External Camera Calibration for Synchronized Multi-video Systems , 2004, WSCG.