Illumination decomposition for material recoloring with consistent interreflections

Changing the color of an object is a basic image editing operation, but a high quality result must also preserve natural shading. A common approach is to first compute reflectance and illumination intrinsic images. Reflectances can then be edited independently, and recomposed with the illumination. However, manipulating only the reflectance color does not account for diffuse interreflections, and can result in inconsistent shading in the edited image. We propose an approach for further decomposing illumination into direct lighting, and indirect diffuse illumination from each material. This decomposition allows us to change indirect illumination from an individual material independently, so it matches the modified reflectance color. To address the underconstrained problem of decomposing illumination into multiple components, we take advantage of its smooth nature, as well as user-provided constraints. We demonstrate our approach on a number of examples, where we consistently edit material colors and the associated interreflections.

[1]  Richard Szeliski,et al.  Image deblurring and denoising using color priors , 2009, CVPR.

[2]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[3]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[4]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[5]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[7]  Erik Reinhard,et al.  Depicting procedural caustics in single images , 2008, SIGGRAPH 2008.

[8]  Kiriakos N. Kutulakos,et al.  A theory of inverse light transport , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  John Hart,et al.  Textureshop: texture synthesis as a photograph editing tool , 2004, SIGGRAPH 2004.

[10]  Jack Tumblin,et al.  Editing Soft Shadows in a Digital Photograph , 2007, IEEE Computer Graphics and Applications.

[11]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Stephen Lin,et al.  Intrinsic image decomposition with non-local texture cues , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Frédo Durand,et al.  Light mixture estimation for spatially varying white balance , 2008, SIGGRAPH 2008.

[14]  Edward H. Adelson,et al.  Recovering intrinsic images from a single image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[16]  Raanan Fattal.  Single image dehazing , 2008, SIGGRAPH 2008.

[17]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[18]  Sumanta N. Pattanaik,et al.  Interactive light transport editing for flexible global illumination , 2007, SIGGRAPH '07.

[19]  Dani Lischinski,et al.  The Shadow Meets the Mask: Pyramid‐Based Shadow Removal , 2008, Comput. Graph. Forum.

[20]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[21]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, SIGGRAPH 2006.

[22]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[23]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[24]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[25]  Sylvain Paris,et al.  User-assisted intrinsic images , 2009, ACM Trans. Graph..

[26]  James Arvo,et al.  Painting with light , 1993, SIGGRAPH.

[27]  Miloš Hašan,et al.  Direct-to-indirect transfer for cinematic relighting , 2006, SIGGRAPH 2006.

[28]  Tian-Tsong Ng,et al.  A Dual Theory of Inverse and Forward Light Transport , 2010, ECCV.

[29]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[30]  Maneesh Agrawala,et al.  Edge-based image coarsening , 2009, TOGS.

[31]  K. Bala,et al.  Lightcuts: a scalable approach to illumination , 2005, SIGGRAPH 2005.

[32]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[33]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[34]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[35]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[36]  Edward H. Adelson,et al.  Recovering reflectance and illumination in a world of painted polyhedra , 1993, 1993 (4th) International Conference on Computer Vision.

[37]  Anat Levin,et al.  User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Cheng Lu,et al.  On the removal of shadows from images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[40]  Michael F. Cohen,et al.  Radiosity and realistic image synthesis , 1993 .

[41]  Alexander Keller,et al.  Instant radiosity , 1997, SIGGRAPH.