Removing Shadows for Color Projection Using Sun Position Estimation

The result of an outdoors 3D scanning acquisition campaign is usually an accurate 3D model of the site, but in most of the cases the quality of the color acquired by the scanner is not satisfying. Alternative solutions, like the projection of a photographic dataset acquired in a different stage, are still dependent on the quality of the initial images. The short time for the acquisition campaigns and the weather conditions often force the shooting of images taken under a strong direct sun illumination. This generates colored models of poor quality. In this paper we present a method to estimate the sun position starting from a geo-referenced 3D model and a set of images. The sun position can enhance both the image alignment and color projection phases. Moreover, the detection and removal of the shadows from the images produces a better and more coherent color in the final 3D model. Using this kind of approach, outdoor sites can be acquired producing a high quality color information together with an accurate geometric measurement.

[1]  Cheng Lu,et al.  Intrinsic Images by Entropy Minimization , 2004, ECCV.

[2]  Daniel Cohen-Or,et al.  Seamless Montage for Texturing Models , 2010, Comput. Graph. Forum.

[3]  Paul E. Debevec,et al.  Unlighting the Parthenon , 2004, SIGGRAPH '04.

[4]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[5]  G. Finlayson,et al.  Simple Shadow Removal , 2006 .

[6]  Paolo Cignoni,et al.  Masked photo blending: Mapping dense photographic data set on high-resolution sampled 3D models , 2008, Comput. Graph..

[7]  Stephen Lin,et al.  Diffuse-Specular Separation and Depth Recovery from Image Sequences , 2002, ECCV.

[8]  Paolo Cignoni,et al.  Stereo Light Probe , 2008, Comput. Graph. Forum.

[9]  Paolo Cignoni,et al.  Improved color acquisition and mapping on 3D models via flash-based photography , 2010, JOCCH.

[10]  Honggang Zhang,et al.  Chromaticity-based separation of reflection components in a single image , 2008, Pattern Recognit..

[11]  Richard Szeliski,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[12]  F. Ortiz,et al.  Automatic detection and elimination of specular reflectance in color images by means of MS diagram and vector connected filters , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[14]  Katsushi Ikeuchi,et al.  Interactive Shadow Removal from a Single Image Using Hierarchical Graph Cut , 2009, ACCV.

[15]  Dani Lischinski,et al.  The Shadow Meets the Mask: Pyramid‐Based Shadow Removal , 2008, Comput. Graph. Forum.

[16]  Stephen Lin,et al.  Highlight removal by illumination-constrained inpainting , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Robert B. Fisher,et al.  Fusing multiple color images for texturing models , 2004 .

[18]  Roberto Scopigno,et al.  Image‐to‐Geometry Registration: a Mutual Information Method exploiting Illumination‐related Geometric Properties , 2009, Comput. Graph. Forum.

[19]  Paolo Cignoni,et al.  Reconstructing Textured Meshes from Multiple Range RGB Maps , 2002, VMV.

[20]  Cheng Lu,et al.  Shadow Removal via Flash/Noflash Illumination , 2006, 2006 IEEE Workshop on Multimedia Signal Processing.

[21]  Hagit Hel-Or,et al.  Texture-Preserving Shadow Removal in Color Images Containing Curved Surfaces , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  F. Durand,et al.  Flash photography enhancement via intrinsic relighting , 2004, ACM Trans. Graph..

[23]  Katsushi Ikeuchi,et al.  Temporal-color space analysis of reflection , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Adam Baumberg,et al.  Blending Images for Texturing 3D Models , 2002, BMVC.

[25]  Robert B. Fisher,et al.  Fusing multiple color images for texturing models , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[26]  Holly E. Rushmeier,et al.  High-Quality Texture Reconstruction from Multiple Scans , 2001, IEEE Trans. Vis. Comput. Graph..

[27]  Lawrence B. Wolff,et al.  Using polarization to separate reflection components , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Kentaro Toyama,et al.  Continuous Flash , 2003 .

[29]  Hans-Peter Seidel,et al.  Automated texture registration and stitching for real world models , 2000, Proceedings the Eighth Pacific Conference on Computer Graphics and Applications.

[30]  Borivoj Vojnovic,et al.  An Algorithm for image stitching and blending , 2005, SPIE BiOS.

[31]  Cheng Lu,et al.  Entropy Minimization for Shadow Removal , 2009, International Journal of Computer Vision.

[32]  Graham D. Finlayson,et al.  Simple Shadow Remova , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[33]  Paolo Cignoni,et al.  Minimizing user intervention in registering 2D images to 3D models , 2005, The Visual Computer.

[34]  Alexei A. Efros,et al.  Estimating natural illumination from a single outdoor image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  Linda G. Shapiro,et al.  Acquisition and visualization of colored 3D objects , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[36]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.

[37]  Frédo Durand,et al.  Flash photography enhancement via intrinsic relighting , 2004, SIGGRAPH 2004.