Single-View Relighting with Normal Map Painting

This paper presents an interactive technique for changing the illumination of objects in a real photograph. 1 Because lighting effects are very sensitive to surface normal perturbations but hard to automatically analyze, we rely on a pen-based interface to quickly draw an approximate normal map over a photo. The photo and the approximate normal map are then given as input to a novel algorithm that refines the map and assigns reflectance to every pixel. We present relighting results for a variety of scenes, and use our technique to match the illumination of multiple pho-

[1]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[2]  Steve Marschner,et al.  Inverse Lighting for Photography , 1997, CIC.

[3]  Jan J. Koenderink,et al.  Pictorial relief , 2019, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[4]  Li Zhang,et al.  Single view modeling of free-form scenes , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[5]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[6]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[7]  Philip K. Robertson,et al.  Spatial transformations for rapid scan-line surface shadowing , 1989, IEEE Computer Graphics and Applications.

[8]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[9]  Ian D. Reid,et al.  Single View Metrology , 2000, International Journal of Computer Vision.

[10]  Pieter Peers,et al.  Relighting with 4D incident light fields , 2003, ACM Trans. Graph..

[11]  André Gagalowicz,et al.  Image-based rendering of diffuse, specular and glossy surfaces from a single image , 2001, SIGGRAPH.

[12]  Scott F. Johnston.  Lumo: illumination for cel animation , 2002, NPAR '02.

[13]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[14]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[15]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[16]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[17]  Sing Bing Kang,et al.  Depth Painting for Image-based Rendering Applications , 1999 .

[18]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[19]  Hans-Peter Seidel,et al.  Bayesian relighting , 2005, EGSR '05.

[20]  Harry Shum,et al.  Relighting with the Reflected Irradiance Field: Representation, Sampling and Reconstruction , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.