Real-Time Rendering of Real World Environments

One of the most important goals of interactive computer graphics is to allow a user to freely walk around a virtual recreation of a real environment that looks as real as the world around us. But hand-modeling such a virtual environment is inherently limited and acquiring the scene model using devices also presents challenges. Interactively rendering such a detailed model is beyond the limits of current graphics hardware, but image-based approaches can significantly improve the status quo. We present an end-to-end system for acquiring highly detailed scans of large real world spaces, consisting of forty to eighty million range and color samples, using a digital camera and laser rangefinder. We explain successful techniques to represent these large data sets as image-based models and present contributions to image-based rendering that allow these models to be rendered in real time on existing graphics hardware without sacrificing the high resolution at which the data sets were acquired.

[1]  Lee Westover,et al.  Footprint evaluation for volume rendering , 1990, SIGGRAPH.

[2]  Marc Levoy,et al.  The Use of Points as a Display Primitive , 2000 .

[3]  Nelson L. Max,et al.  Hierarchical Rendering of Trees from Precomputed Multi-Layer Z-Buffers , 1996, Rendering Techniques.

[4]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[5]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[6]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[7]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  T WhitakerRoss.  A Level-Set Approach to 3D Reconstruction from Range Data , 1998 .

[9]  Bob Francis,et al.  Silicon Graphics Inc. , 1993 .

[10]  Steven Molnar.  Efficient Supersampling Antialiasing for High-Performance Architectures , 1991 .

[11]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[12]  R. Bajcsy,et al.  3-D data acquisition and interpretation for virtual reality and telepresence , 1998, Proceedings 1998 IEEE and ATR Workshop on Computer Vision for Virtual Reality Based Human Communications.

[13]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[14]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[15]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[16]  Gregory Ward Larson.  The Holodeck: A Parallel Ray-caching Rendering System , 1998 .

[17]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[18]  Takeo Kanade,et al.  Virtual ized reality: constructing time-varying virtual worlds from real world events , 1997 .

[19]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[20]  Anselmo Lastra,et al.  PixelFlow: the realization , 1997, HWWS '97.

[21]  John S. Montrym,et al.  InfiniteReality: a real-time graphics system , 1997, SIGGRAPH.

[22]  Paul Rademacher,et al.  Multiple-center-of-projection images , 1998, SIGGRAPH.

[23]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[24]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[25]  Anselmo Lastra,et al.  A shading language on graphics hardware: the pixelflow shading system , 1998, SIGGRAPH.

[26]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[27]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[28]  Kiriakos N. Kutulakos,et al.  Plenoptic Image Editing , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  James M. Coggins,et al.  Using Statistical Pattern Recognition Techniques to Control Variable Conductance Diffusion , 1993, IPMI.

[30]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[31]  P. Boulanger,et al.  Sensor based creation of indoor virtual environment models , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[32]  William J. Dally,et al.  Point Sample Rendering , 1998, Rendering Techniques.

[33]  Takeo Kanade,et al.  Virtualized reality: constructing time-varying virtual worlds from real world events , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[34]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[35]  Gabriel Taubin,et al.  Acquiring Input for Rendering at Appropriate Levels of Detail: Digitizing a Tietà , 1998, Rendering Techniques.

[36]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[37]  Linda G. Shapiro,et al.  View-base Rendering: Visualizing Real Objects from Scanned Range and Color Data , 1997, Rendering Techniques.

[38]  Linda G. Shapiro,et al.  Robust meshes from multiple range maps , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).