RGB2AO: Ambient Occlusion Generation from RGB Images

We present RGB2AO, a novel task to generate ambient occlusion (AO) from a single RGB image instead of screen space buffers such as depth and normal. RGB2AO produces a new image filter that creates a non‐directional shading effect that darkens enclosed and sheltered areas. RGB2AO aims to enhance two 2D image editing applications: image composition and geometry‐aware contrast enhancement. We first collect a synthetic dataset consisting of pairs of RGB images and AO maps. Subsequently, we propose a model for RGB2AO by supervised learning of a convolutional neural network (CNN), considering 3D geometry of the input image. Experimental results quantitatively and qualitatively demonstrate the effectiveness of our model.

[1]  Julie Dorsey,et al.  Understanding and improving the realism of image composites , 2012, ACM Trans. Graph..

[2]  Yannick Hold-Geoffroy,et al.  Deep Outdoor Illumination Estimation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[4]  Nassir Navab,et al.  Deeper Depth Prediction with Fully Convolutional Residual Networks , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[5]  Kalyan Sunkavalli,et al.  Fast Spatially-Varying Indoor Lighting Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Jitendra Malik,et al.  Shape, Illumination, and Reflectance from Shading , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[8]  Yannick Hold-Geoffroy,et al.  Deep Sky Modeling for Single Image Outdoor Lighting Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Sergey Zhukov,et al.  An Ambient Light Illumination Model , 1998, Rendering Techniques.

[11]  Ashutosh Saxena,et al.  Learning Depth from Single Monocular Images , 2005, NIPS.

[12]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  LalondeJean-François,et al.  Learning to predict indoor illumination from a single image , 2017 .

[14]  Okan Arikan,et al.  Hardware accelerated ambient occlusion techniques on GPUs , 2007, SI3D.

[15]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[16]  Ian D. Reid,et al.  Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Nancy Argüelles,et al.  Author ' s , 2008 .

[18]  Shijian Lu,et al.  Spatial Fusion GAN for Image Synthesis , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[20]  Morgan McGuire,et al.  The alchemy screen-space ambient obscurance algorithm , 2011, HPG '11.

[21]  Hans-Peter Seidel,et al.  Deep Shading: Convolutional Neural Networks for Screen Space Shading , 2016, Comput. Graph. Forum.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Alexei A. Efros,et al.  Estimating the Natural Illumination Conditions from a Single Outdoor Image , 2012, International Journal of Computer Vision.

[25]  Sylvain Paris,et al.  Error-Tolerant Image Compositing , 2010, International Journal of Computer Vision.

[26]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Weifeng Chen,et al.  Single-Image Depth Perception in the Wild , 2016, NIPS.

[28]  Zhengqi Li,et al.  MegaDepth: Learning Single-View Depth Prediction from Internet Photos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Micah K. Johnson,et al.  Multi-scale image harmonization , 2010, ACM Trans. Graph..

[30]  Oliver Deussen,et al.  Image enhancement by unsharp masking the depth buffer , 2006, ACM Trans. Graph..

[31]  Taku Komura,et al.  Neural network ambient occlusion , 2016, SIGGRAPH Asia Technical Briefs.

[32]  Noah Snavely,et al.  Intrinsic images in the wild , 2014, ACM Trans. Graph..

[33]  Alexei A. Efros,et al.  Learning a Discriminative Model for the Perception of Realism in Composite Images , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[34]  Tim Weyrich,et al.  Decomposing Single Images for Layered Photo Retouching , 2017, Comput. Graph. Forum.

[35]  Louis Bavoil,et al.  Image-space horizon-based ambient occlusion , 2008, SIGGRAPH '08.

[36]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[37]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[38]  Ersin Yumer,et al.  Learning to predict indoor illumination from a single image , 2017, ACM Trans. Graph..

[39]  Alexei A. Efros,et al.  Using Color Compatibility for Assessing Image Realism , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[40]  Hans-Peter Seidel,et al.  Approximating dynamic global illumination in image space , 2009, I3D '09.

[41]  Xiaoyan Sun,et al.  Contrast Enhancement Based on Intrinsic Image Decomposition , 2017, IEEE Transactions on Image Processing.

[42]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Ming-Hsuan Yang,et al.  Deep Image Harmonization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Martin Mittring,et al.  Finding next gen: CryEngine 2 , 2007, SIGGRAPH Courses.

[45]  Noah Snavely,et al.  Photometric Ambient Occlusion for Intrinsic Image Decomposition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Randima Fernando,et al.  GPU Gems: Programming Techniques, Tips and Tricks for Real-Time Graphics , 2004 .