An approximate shading model for object relighting

We propose an approximate shading model for image-based object modeling and insertion. Our approach is a hybrid of 3D rendering and image-based composition. It avoids the difficulties of physically accurate shape estimation from a single image, and allows for more flexible image composition than pure image-based methods. The model decomposes the shading field into (a) a rough shape term that can be reshaded, (b) a parametric shading detail that encodes missing features from the first term, and (c) a geometric detail term that captures fine-scale material properties. With this object model, we build an object relighting system that allows an artist to select an object from an image and insert it into a 3D scene. Through simple interactions, the system can adjust illumination on the inserted object so that it appears more naturally in the scene. Our quantitative evaluation and extensive user study suggest our method is a promising alternative to existing methods of object insertion.

[1]  Jian Sun,et al.  Interactive normal reconstruction from a single image , 2008, SIGGRAPH 2008.

[2]  Derek Hoiem,et al.  Recovering the spatial layout of cluttered rooms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  Kalyan Sunkavalli,et al.  Automatic Scene Inference for 3D Object Compositing , 2014, ACM Trans. Graph..

[4]  Jean-Denis Durou,et al.  Numerical methods for shape-from-shading: A new survey with benchmarks , 2008, Comput. Vis. Image Underst..

[5]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[6]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[7]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  David A. Forsyth,et al.  Non-parametric Filtering for Geometric Detail Extraction and Material Representation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  David A. Forsyth,et al.  Variable-Source Shading Analysis , 2011, International Journal of Computer Vision.

[10]  Alexei A. Efros,et al.  Photo clip art , 2007, ACM Trans. Graph..

[11]  Yizhou Yu,et al.  Patch-based image vectorization with automatic curvilinear feature alignment , 2009, ACM Trans. Graph..

[12]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[13]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[14]  Andrew W. Fitzgibbon,et al.  Single View Reconstruction of Curved Surfaces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Patrick Cavanagh,et al.  Perceiving Illumination Inconsistencies in Scenes , 2005, Perception.

[16]  Edward H. Adelson,et al.  Ground truth dataset and baseline evaluations for intrinsic image algorithms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Jitendra Malik,et al.  Color Constancy, Intrinsic Images, and Shape Estimation , 2012, ECCV.

[18]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[20]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[21]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[22]  David A. Forsyth,et al.  Rendering synthetic objects into legacy photographs , 2011, ACM Trans. Graph..

[23]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[24]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[25]  Bevil R. Conway,et al.  Perspectives on science and art , 2007, Current Opinion in Neurobiology.

[26]  Erik Reinhard,et al.  Compositing images through light source detection , 2010, Comput. Graph..

[27]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[28]  Scott F. Johnston.  Lumo: illumination for cel animation , 2002, NPAR '02.

[29]  P. Cavanagh.  The artist as neuroscientist , 2005, Nature.