Interactive system for dynamic scene lighting using captured video environment maps

We present an interactive system for fully dynamic scene lighting using captured high dynamic range (HDR) video environment maps. The key component of our system is an algorithm for efficient decomposition of HDR video environment map captured over hemisphere into a set of representative directional light sources, which can be used for the direct lighting computation with shadows using graphics hardware. The resulting lights exhibit good temporal coherence and their number can be adaptively changed to keep a constant framerate while good spatial distribution (stratification) properties are maintained. We can handle a large number of light sources with shadows using a novel technique which reduces the cost of BRDF-based shading and visibility computations. We demonstrate the use of our system in a mixed reality application in which real and synthetic objects are illuminated by consistent lighting at interactive framerates.

[1]  Harry Shum,et al.  Eurographics Symposium on Rendering (2004) All-frequency Precomputed Radiance Transfer for Glossy Objects , 2022 .

[2]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[3]  Hans-Peter Seidel,et al.  Realistic, hardware-accelerated shading and lighting , 1999, SIGGRAPH.

[4]  Roland W Fleming,et al.  Real-world illumination and the perception of surface reflectance properties. , 2003, Journal of vision.

[5]  Alexander Keller,et al.  Tiled Blue Noise Samples , 2001, VMV.

[6]  Andrew S. Glassner,et al.  Principles of Digital Image Synthesis , 1995 .

[7]  Ronald L. Wasserstein,et al.  Monte Carlo: Concepts, Algorithms, and Applications , 1997 .

[8]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.

[9]  Stefan Winkler,et al.  Digital Video Quality: Vision Models and Metrics , 2005 .

[10]  Jan Kautz,et al.  Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments , 2002 .

[11]  Leonidas J. Guibas,et al.  Optimally combining sampling techniques for Monte Carlo rendering , 1995, SIGGRAPH.

[12]  Alexander Keller,et al.  Efficient Illumination by High Dynamic Range Images , 2003, Rendering Techniques.

[13]  Wolfgang Heidrich,et al.  Fast Primitive Distribution for Illustration , 2002, Rendering Techniques.

[14]  Tomas Akenine-Möller,et al.  A geometry-based soft shadow volume algorithm using graphics hardware , 2003, ACM Trans. Graph..

[15]  Richard Szeliski,et al.  High dynamic range video , 2003, ACM Trans. Graph..

[16]  V. Ostromoukhov,et al.  Fast hierarchical importance sampling with blue noise properties , 2004, SIGGRAPH 2004.

[17]  Jan Kautz,et al.  Approximation of Glossy Reflection with Prefiltered Environment Maps , 2000, Graphics Interface.

[18]  Ned Greene,et al.  Environment Mapping and Other Applications of World Projections , 1986, IEEE Computer Graphics and Applications.

[19]  Mateu Sbert,et al.  Combined Correlated and Importance Sampling in Direct Light Source Computation and Environment Mapping , 2004, Comput. Graph. Forum.

[20]  Wolfgang Heidrich,et al.  Bidirectional importance sampling for illumination from environment maps , 2004, SIGGRAPH '04.

[21]  Peter Shirley,et al.  A Low Distortion Map Between Disk and Square , 1997, J. Graphics, GPU, & Game Tools.

[22]  Philippe Bekaert,et al.  Advanced global illumination , 2006 .

[23]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[24]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[25]  Serge J. Belongie,et al.  Structured importance sampling of environment maps , 2003, ACM Trans. Graph..

[26]  P. Hanrahan,et al.  Triple product wavelet integrals for all-frequency relighting , 2004, SIGGRAPH 2004.

[27]  Jaakko Lehtinen,et al.  Hemispherical Rasterization for Self-Shadowing of Dynamic Objects , 2004, Rendering Techniques.

[28]  Simon Gibson,et al.  Interactive Rendering with Real-World Illumination , 2000, Rendering Techniques.

[29]  Andrew Jones,et al.  Direct HDR capture of the sun and sky , 2006, SIGGRAPH Courses.

[30]  Erik Reinhard,et al.  Photographic tone reproduction for digital images , 2002, ACM Trans. Graph..

[31]  Juho Kannala,et al.  A generic camera calibration method for fish-eye lenses , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[32]  Pat Hanrahan,et al.  Triple product wavelet integrals for all-frequency relighting , 2004, ACM Trans. Graph..

[33]  Pierre Poulin,et al.  A Light Hierarchy for Fast Rendering of Scenes with Many Lights , 1998, Comput. Graph. Forum.

[34]  Hans-Peter Seidel,et al.  Goniometric Diagram Mapping for Hemisphere , 2003, Eurographics.