Interactive Rendering with Real-World Illumination

We propose solutions for seamlessly integrating synthetic objects into background photographs at interactive rates. Recently developed image-based methods are used to capture real-world illumination, and sphere-mapping is used illuminate and render the synthetic objects. We present a new procedure for approximating shadows cast by the real-world illumination using standard hardware-based shadow mapping, and a novel image composition algorithm that uses frame-buffer hardware to correctly overlay the synthetic objects and their shadows onto the background image. We show results of an OpenGL implementation of the algorithm that is capable of rendering complex synthetic objects and their shadows at rates of up to 10 frames per second on an SGI Onyx2.

[1]  Pierre Poulin,et al.  Painting Surface Characteristics , 1995, Rendering Techniques.

[2]  Gregory J. Ward,et al.  The RADIANCE lighting simulation and rendering system , 1994, SIGGRAPH.

[3]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[4]  James T. Kajiya,et al.  The rendering equation , 1998 .

[5]  Tomoyuki Nishita,et al.  A montage method: the overlaying of the computer generated images onto a background photograph , 1986, SIGGRAPH.

[6]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[7]  Pierre Poulin,et al.  Sketching shadows and highlights to position lights , 1997, Proceedings Computer Graphics International.

[8]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.

[9]  Marc Olano,et al.  Reflection space image based rendering , 1999, SIGGRAPH.

[10]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[11]  Pierre Poulin,et al.  Interactive Virtual Relighting and Remodeling of Real Scenes , 1999, Rendering Techniques.

[12]  George Drettakis,et al.  Interactive Common Illumination for Computer Augmented Reality , 1997, Rendering Techniques.

[13]  Ned Greene,et al.  Environment Mapping and Other Applications of World Projections , 1986, IEEE Computer Graphics and Applications.

[14]  Peter Shirley,et al.  Visual Glue , 1998 .

[15]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[16]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.

[17]  Paul S. Heckbert,et al.  Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter , 1986, IEEE Computer Graphics and Applications.

[18]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[19]  Kurt Akeley,et al.  The accumulation buffer: hardware support for high-quality rendering , 1990, SIGGRAPH.

[20]  Robert R. Lewis,et al.  Making Shaders More Physically Plausible , 1994, Comput. Graph. Forum.

[21]  Mark Segal,et al.  Fast shadows and lighting effects using texture mapping , 1992, SIGGRAPH.

[22]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[23]  Hans-Peter Seidel,et al.  Realistic, hardware-accelerated shading and lighting , 1999, SIGGRAPH.

[24]  Norman I. Badler,et al.  Multi-pass pipeline rendering: realism for dynamic environments , 1997, SI3D.

[25]  David Salesin,et al.  Rendering antialiased shadows with depth maps , 1987, SIGGRAPH.

[26]  Hans-Peter Seidel,et al.  View-independent environment maps , 1998, Workshop on Graphics Hardware.

[27]  Donald P. Greenberg,et al.  Fitting virtual lights for non-diffuse walkthroughs , 1997, SIGGRAPH.

[28]  A. Fournier,et al.  Common Illumination between Real and Computer Generated Scenes , 1992 .

[29]  James F. Blinn,et al.  Texture and reflection in computer generated images , 1998 .

[30]  Donald P. Greenberg,et al.  Modeling the interaction of light between diffuse surfaces , 1984, SIGGRAPH.