Reconstruction for 3D immersive virtual environments

The future of tele-conferencing is towards multi-party 3D Tele-Immersion (TI) and TI environments that can support realistic inter-personal communications and virtual interaction among participants. In this paper, we address two important issues, pertinent to TI environments. The paper focuses on techniques for the real-time, 3D reconstruction of moving humans from multiple Kinect devices. The off-line generation of real-life 3D scenes from visual data, captured by non-professional users is also addressed. Experimental results are provided that demonstrate the efficiency of the methods, along with an example of mixing real with virtual in a shared space.

[1]  Noel E. O'Connor,et al.  Evaluating a dancer's performance using kinect-based skeleton tracking , 2011, ACM Multimedia.

[2]  Roberto Toldo,et al.  Global registration of multiple point clouds embedding the Generalized Procrustes Analysis into an ICP framework , 2010 .

[3]  Roberto Cipolla,et al.  Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[5]  Ruzena Bajcsy,et al.  High-Quality Visualization for Geographically Distributed 3-D Teleimmersive Applications , 2011, IEEE Transactions on Multimedia.

[6]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Zeyu Li,et al.  Wide-area external multi-camera calibration using vision graphs and virtual calibration object , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[8]  Aljoscha Smolic,et al.  Increasing the Accuracy of the Space-Sweeping Approach to Stereo Reconstruction, using Spherical Backprojection Surfaces , 2006, 2006 International Conference on Image Processing.

[9]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[10]  Edmond Boyer,et al.  Efficient Polyhedral Modeling from Silhouettes , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[12]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[13]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time dynamic 3D surface reconstruction and interaction , 2011, SIGGRAPH '11.

[14]  Tomás Svoboda,et al.  A Convenient Multicamera Self-Calibration for Virtual Environments , 2005, Presence: Teleoperators & Virtual Environments.