Deep Fence Estimation using Stereo Guidance and Adversarial Learning

People capture memorable images of events and exhibits that are often occluded by a wire mesh loosely termed as fence. Recent works in removing fence have limited performance due to the difficulty in initial fence segmentation. This work aims to accurately segment fence using a novel fence guidance mask (FM) generated from stereo image pair. This binary guidance mask contains deterministic cues about the structure of fence and is given as additional input to the deep fence estimation model. We also introduce a directional connectivity loss (DCL), which is used alongside adversarial loss to precisely detect thin wires. Experimental results obtained on real world scenarios demonstrate the superiority of proposed method over state-of-the-art techniques.

[1]  Truong Q. Nguyen,et al.  Accurate and Efficient Video De-Fencing Using Convolutional Neural Networks and Temporal Information , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[2]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Wei Liu,et al.  Video De-Fencing , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[7]  James F. Peters,et al.  Fence-like Quasi-periodic Texture Detection in Images , 2014 .

[8]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[9]  Peter Bajcsy,et al.  Cell Image Segmentation Using Generative Adversarial Networks, Transfer Learning, and Augmentations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[11]  Ping Tan,et al.  Automatic Fence Segmentation in Videos of Dynamic Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  William T. Freeman,et al.  A computational approach for obstruction-free photography , 2015, ACM Trans. Graph..

[13]  Rajiv R. Sahay,et al.  Stereo image de-fencing using smartphones , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Thomas S. Huang,et al.  Generative Image Inpainting with Contextual Attention , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Yanxi Liu,et al.  Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Carola-Bibiane Schönlieb,et al.  Combined First and Second Order Total Variation Inpainting using Split Bregman , 2013, Image Process. Line.

[17]  Rajiv R. Sahay,et al.  Deep Learning Based Fence Segmentation and Removal from an Image Using a Video Sequence , 2016, ECCV Workshops.

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[20]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.