Single Image Reflection Removal with Absorption Effect

In this paper, we consider the absorption effect for the problem of single image reflection removal. We show that the absorption effect can be numerically approximated by the average of refractive amplitude coefficient map. We then reformulate the image formation model and propose a two-step solution that explicitly takes the absorption effect into account. The first step estimates the absorption effect from a reflection-contaminated image, while the second step recovers the transmission image by taking a reflection-contaminated image and the estimated absorption effect as the input. Experimental results on four public datasets show that our two-step solution not only successfully removes reflection artifact, but also faithfully restores the intensity distortion caused by the absorption effect. Our ablation studies further demonstrate that our method achieves superior performance on the recovery of overall intensity and has good model generalization capacity. The code is available at https://github.com/q-zh/absorption.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Ling-Yu Duan,et al.  What Does Plate Glass Reveal About Camera Calibration? , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[4]  Yang Yang,et al.  Fast Single Image Reflection Suppression via Convex Optimization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Frédo Durand,et al.  Reflection removal using ghosting cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[7]  Anat Levin,et al.  User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Delu Zeng,et al.  Removing Rain from Single Images via a Deep Detail Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Ren Ng,et al.  Single Image Reflection Separation with Perceptual Losses , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Kristian Kirsch,et al.  Theory Of Ordinary Differential Equations , 2016 .

[11]  Jiaolong Yang,et al.  Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Ah-Hwee Tan,et al.  Sparsity based reflection removal using external patch search , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[13]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[14]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[15]  Ah-Hwee Tan,et al.  Depth of field guided reflection removal , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Marc Pollefeys,et al.  Reflection Separation using a Pair of Unpolarized and Polarized Images , 2019, NeurIPS.

[19]  Zahra Kadkhodaie,et al.  Robust and interpretable blind image denoising via bias-free convolutional neural networks , 2019, ICLR.

[20]  Sung-eui Yoon,et al.  Single Image Reflection Removal With Physically-Based Training Images , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  M. Schmid.  Principles Of Optics Electromagnetic Theory Of Propagation Interference And Diffraction Of Light , 2016 .

[22]  Ling-Yu Duan,et al.  CoRRN: Cooperative Reflection Removal Network , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[24]  A. Krizhevsky.  Convolutional Deep Belief Networks on CIFAR-10 , 2010 .

[25]  Jan Palczewski,et al.  Monte Carlo Simulation , 2008, Encyclopedia of GIS.

[26]  Sabine Süsstrunk,et al.  Single Image Reflection Suppression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Qiong Yan,et al.  Polarized Reflection Removal With Perfect Alignment in the Wild , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ling-Yu Duan,et al.  Benchmarking Single-Image Reflection Removal Algorithms , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Patrick T. Komiske,et al.  Deep learning in color: towards automated quark/gluon jet discrimination , 2016, Journal of High Energy Physics.

[30]  C. Peirce.  An unpublished manuscript) , 2016 .

[31]  Yu-Chiang Frank Wang,et al.  Exploiting image structural similarity for single image rain removal , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[32]  Huchuan Lu,et al.  Learning Dual Convolutional Neural Networks for Low-Level Vision , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Mohammed Ghanbari,et al.  Scope of validity of PSNR in image/video quality assessment , 2008 .

[34]  Guoqiang Han,et al.  Single Image Reflection Removal Beyond Linearity , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  G. Sigel.  Optical Absorption of Glasses , 1977 .

[36]  John E. Hopcroft,et al.  Single Image Reflection Removal Through Cascaded Refinement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Jiaolong Yang,et al.  A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing (Supplementary Material) , 2017 .

[38]  Jan Kautz,et al.  Separating Reflection and Transmission Images in the Wild , 2017, ECCV.

[39]  Yu-Wing Tai,et al.  A Physically-Based Approach to Reflection Separation: From Physical Modeling to Constrained Optimization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Wen Gao,et al.  Region-Aware Reflection Removal With Unified Content and Gradient Priors , 2018, IEEE Transactions on Image Processing.

[42]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[43]  Michael S. Brown,et al.  Exploiting Reflection Change for Automatic Reflection Removal , 2013, 2013 IEEE International Conference on Computer Vision.

[44]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[45]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[46]  Jie Yang,et al.  Seeing Deeply and Bidirectionally: A Deep Learning Approach for Single Image Reflection Removal , 2018, ECCV.

[47]  Ling-Yu Duan,et al.  CRRN: Multi-scale Guided Concurrent Reflection Removal Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[48]  Michael S. Brown,et al.  Reflection Removal Using a Dual-Pixel Sensor , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  R. Parkin.  A note on the extinction coefficient and absorptivity of glass , 2015 .