Image de-fencing framework with hybrid inpainting algorithm

Detection and removal of fences from digital images become essential when an important part of the scene turns to be occluded by such unwanted structures. Image de-fencing is challenging because manually marking fence boundaries is tedious and time-consuming. In this paper, a novel image de-fencing algorithm that effectively detects and removes fences with minimal user input is presented. The user is only requested to mark few fence pixels; then, color models are estimated and used to train Bayes classifier to segment the fence and the background. Finally, the fence mask is refined exploiting connected component analysis and morphological operators. To restore the occluded region, a hybrid inpainting algorithm is proposed that integrates exemplar-based technique with a pyramid-based interpolation approach. In contrast to previous solutions which work only for regular pattern fences, the proposed technique is able to remove both regular and irregular fences. A large number of experiments are carried out on a wide variety of images containing different types of fences demonstrating the effectiveness of the proposed approach. The proposed approach is also compared with state-of-the-art image de-fencing and inpainting techniques and showed convincing results.

[1]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[2]  M. Shahid Farid,et al.  Image inpainting using dynamic weighted kernels , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[3]  P. Nagabhushan,et al.  Object removal by depth-wise image inpainting , 2015, Signal Image Video Process..

[4]  Kai-Kuang Ma,et al.  A survey on super-resolution imaging , 2011, Signal Image Video Process..

[5]  Guillermo Sapiro,et al.  A Variational Framework for Exemplar-Based Image Inpainting , 2011, International Journal of Computer Vision.

[6]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[7]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .

[8]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[9]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[10]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[11]  Yanxi Liu,et al.  Image De-fencing Revisited , 2010, ACCV.

[12]  Arif Mahmood,et al.  Image inpainting based on pyramids , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[13]  Qiuqi Ruan,et al.  Object Removal By Cross Isophotes Exemplar-based Inpainting , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[14]  Luc Van Gool,et al.  Noncombinatorial Detection of Regular Repetitions under Perspective Skew , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[16]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Carola-Bibiane Schönlieb,et al.  A Combined First and Second Order Variational Approach for Image Reconstruction , 2012, Journal of Mathematical Imaging and Vision.

[18]  Manuel Menezes de Oliveira Neto,et al.  Fast Digital Image Inpainting , 2001, VIIP.

[19]  Yanxi Liu,et al.  Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  David A. Forsyth,et al.  Shape from Texture without Boundaries , 2002, International Journal of Computer Vision.

[21]  Chuan-Kai Yang,et al.  Stain removal in 2D images with globally varying textures , 2014, Signal Image Video Process..

[22]  Yanxi Liu,et al.  Image de-fencing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Folkmar Bornemann,et al.  Fast Image Inpainting Based on Coherence Transport , 2007, Journal of Mathematical Imaging and Vision.

[24]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[26]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Alexei A. Efros,et al.  Discovering Texture Regularity as a Higher-Order Correspondence Problem , 2006, ECCV.

[28]  Yanxi Liu,et al.  A computational model for periodic pattern perception based on frieze and wallpaper groups , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Anil C. Kokaram,et al.  Semi-transparent blotches removal from sepia images exploiting visibility laws , 2013, Signal Image Video Process..

[30]  Jitendra Malik,et al.  Detecting, localizing and grouping repeated scene elements from an image , 1996, ECCV.

[31]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[32]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.