Scene surface light field representation for realistic lighting

Abstract. Existing light field-based lighting follows the representation on plane, such as the commonly used light slab. A novel light field representation on scene surface is proposed. The illumination is captured by a panorama camera on translation stage with a nonuniform sampling and calibration strategy. Sampled light rays are reprojected on the reconstructed scene to generate the scene surface light field representations for both direct light source and indirect light. This representation is more consistent with the actual emission and reflection of light transport than the existing ones. An accurate light source estimation and a practical indirect light resampling approach are presented based on the representation. The rendering results illustrate the realistic lighting effect, especially in spatially varying illumination.

[1]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[2]  Marc Levoy,et al.  Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[4]  Da-Hai Li,et al.  High-optical-efficiency integral imaging display based on gradient-aperture pinhole array , 2013 .

[5]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[6]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[7]  Yan Sun,et al.  Homogeneous light field model for interactive control of viewing parameters of integral imaging displays. , 2012, Optics express.

[8]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[9]  C. R. Hoffman,et al.  Illumination and Reflection Maps : Simulated Objects in Simulated and Real Environments Gene , 1984 .

[10]  Edmund Y Lam,et al.  High-resolution lightfield photography using two masks. , 2012, Optics express.

[11]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[12]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[13]  Lin Wang,et al.  Non-uniform Illumination Representation based on HDR Light Probe Sequences , 2012, 2012 International Conference on Virtual Reality and Visualization.

[14]  Anders Ynnerman,et al.  Spatially varying image based lighting by light probe sequences , 2007, The Visual Computer.

[15]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[16]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[17]  Anders Ynnerman,et al.  Free Form Incident Light Fields , 2008, Comput. Graph. Forum.

[18]  Kwanghoon Sohn,et al.  Face relighting based on virtual irradiance sphere and reflection coefficient , 2008 .

[19]  T M Lehmann,et al.  Color line search for illuminant estimation in real-world scenes. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[20]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[21]  Jae-Hyeung Park.  Comparative analysis on light field reconstruction characteristics of autostereoscopic three-dimensional display technologies , 2012, Photonics Asia.

[22]  Alexander A. Mury,et al.  Light field constancy within natural scenes. , 2007, Applied optics.

[23]  Andrew Gardner,et al.  Capturing and Rendering with Incident Light Fields , 2003, Rendering Techniques.

[24]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[25]  Hyunjung Shim,et al.  Faces as light probes for relighting , 2012 .

[26]  Hans-Peter Seidel,et al.  Accurate light source acquisition and rendering , 2003, ACM Trans. Graph..

[27]  Alexander A. Mury,et al.  Representing the light field in finite three-dimensional spaces from sparse discrete samples. , 2009, Applied optics.

[28]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.

[29]  Stefan Gustavson,et al.  Next generation image based lighting using HDR video , 2011, SIGGRAPH '11.

[30]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[31]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[32]  Paolo Cignoni,et al.  Stereo Light Probe , 2008, Comput. Graph. Forum.

[33]  D. Shepard.  A two-dimensional interpolation function for irregularly-spaced data , 1968, ACM National Conference.