Iterative Gradient Encoding Network with Feature Co-Occurrence Loss for Single Image Reflection Removal

Removing undesired reflections from a photo taken in front of a glass is of great importance for enhancing visual computing systems' efficiency. Previous learning-based approaches have produced visually plausible results for some reflections type, however, failed to generalize against other reflection types. There is a dearth of literature for efficient methods concerning single image reflection removal, which can generalize well in large-scale reflection type. In this study, we proposed an iterative gradient encoding network for single image reflection removal. Next, to further supervise the network in learning the correlation between the transmission layer features, we proposed a feature co-occurrence loss. Extensive experiments on the public benchmark dataset of SIR2 demonstrated that our method can remove reflection favorably against the existing state of the art method on all imaging settings, including diverse backgrounds. Moreover, as the reflection strength increases, our method can still remove reflection even where other state of the art methods failed.

[1]  Anat Levin,et al.  User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Marc Pollefeys,et al.  Reflection Separation using a Pair of Unpolarized and Polarized Images , 2019, NeurIPS.

[3]  Frédo Durand,et al.  Reflection removal using ghosting cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Xiaochun Cao,et al.  Robust Separation of Reflection from Multiple Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Michal Irani,et al.  Separating Transparent Layers through Layer Information Exchange , 2004, ECCV.

[6]  Cheolkon Jung,et al.  Multi-Modal Reflection Removal Using Convolutional Neural Networks , 2019, IEEE Signal Processing Letters.

[7]  John E. Hopcroft,et al.  Single Image Reflection Removal Through Cascaded Refinement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jiaolong Yang,et al.  A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing (Supplementary Material) , 2017 .

[9]  Leon A. Gatys,et al.  A Neural Algorithm of Artistic Style , 2015, ArXiv.

[10]  Ah-Hwee Tan,et al.  Depth of field guided reflection removal , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[11]  Thomas S. Huang,et al.  Free-Form Image Inpainting With Gated Convolution , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Ling-Yu Duan,et al.  Benchmarking Single-Image Reflection Removal Algorithms , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Masaaki Ikehara,et al.  Single Image Reflection Removal Based on GAN With Gradient Constraint , 2019, IEEE Access.

[15]  Jiaolong Yang,et al.  Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ling-Yu Duan,et al.  CoRRN: Cooperative Reflection Removal Network , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Michael S. Brown,et al.  Single Image Layer Separation Using Relative Smoothness , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ling-Yu Duan,et al.  CRRN: Multi-scale Guided Concurrent Reflection Removal Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.