Modeling, Combining, and Rendering Dynamic Real-World Events From Image Sequences

Virtualized Reality creates time-varying three-dimensional models and virtual images from image sequences. In this paper, we present two recent enhancements to Virtualized Reality. We present Model Enhanced Stereo (MES) as a method to use widely separated images to iteratively improve the quality of each local stereo output in a multi-camera system. We then show, using an example, how Virtualized Reality models of two different events are integrated with each other, and with a synthetic virtual model. In addition, we also develop a new calibration method that allows simultaneous calibration of a large number of cameras without visibility problems. The method goes from capturing real image sequences, integrating two or more events with a static or time-varying virtual model, to virtual image sequence generation.

[1]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[2]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[3]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Reinhard Koch,et al.  Shape adaptation for modelling of 3D objects in natural scenes , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[6]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[7]  Václav Hlavác,et al.  Rendering real-world objects using view interpolation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[8]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[9]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[10]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[11]  Adrian Hilton,et al.  Reliable Surface Reconstructiuon from Multiple Range Images , 1996, ECCV.

[12]  Takeo Kanade,et al.  Recovery of dynamic scene structure from multiple image sequences , 1996, 1996 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems (Cat. No.96TH8242).

[13]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[14]  Takeo Kanade,et al.  Virtual ized reality: constructing time-varying virtual worlds from real world events , 1997 .

[15]  Takeo Kanade,et al.  Virtualized reality: constructing time-varying virtual worlds from real world events , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[16]  Katsushi Ikeuchi,et al.  Consensus surfaces for modeling 3D objects from multiple range images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.