A Synthesis-and-Analysis Approach to Image Based Lighting

Given a set of sample images of a scene with variable illumination and the corresponding parameters of the light sources, namely position and/or intensity, we propose an interpolation based approach to model the variation of illumination in images. Once our interpolation based model is constructed from the sample images, we are able to synthesize images under any possible lighting configuration defined in the parametric space. Moreover, given a query image of a scene with a known reference object, our method is able to estimate the lighting parameters of the image. Therefore, our approach allows for both synthesis and analysis of images in different lighting conditions. Our model is ultimately a compact representation of the set of all images with the lighting conditions defined within a parametric space. The interpolation model can generate the rendering of different objects in unknown environments and also perceive unknown environments provided a known object exists in the scene. In this paper, we show robust image synthesis and analysis with two different datasets: an object image dataset with varying lighting intensity and a face image dataset with varying light source position.

[1]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[2]  Václav Hlavác,et al.  Rendering real-world objects using view interpolation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[3]  Paul E. Debevec,et al.  Practical Image-Based Relighting and Editing with Spherical-Harmonics and Local Lights , 2011, 2011 Conference for Visual Media Production.

[4]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[5]  Stephen Lin,et al.  Lighting interpolation by shadow morphing using intrinsic lumigraphs , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[6]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[7]  Allen R. Hanson,et al.  Computer Vision Systems , 1978 .

[8]  Harry Shum,et al.  Review of image-based rendering techniques , 2000, Visual Communications and Image Processing.

[9]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[10]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[11]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[12]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[13]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[14]  Julie Dorsey,et al.  Effic ient Re-rendering of Naturally Illuminated Environments , 1994 .

[15]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Yacov Hel-Or,et al.  Robust estimation of surface properties and interpolation of shadow/specularity components , 2012, Image Vis. Comput..

[17]  Sylvain Paris,et al.  User-assisted intrinsic images , 2009, ACM Trans. Graph..

[18]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[19]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[20]  Sharat Chandran,et al.  A Survey of Image-based Relighting Techniques , 2007, J. Virtual Real. Broadcast..

[21]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.