Towards an Automated Image De-fencing Algorithm Using Sparsity

Conventional approaches to image de-fencing suffer from non-robust fence detection and are limited to processing images of static scenes. In this position paper, we propose an automatic de-fencing algorithm for images of dynamic scenes. We divide the problem of image de-fencing into the tasks of automated fence detection, motion estimation and fusion of data from multiple frames of a captured video of the dynamic scene. Fences are detected automatically using two approaches, namely, employing Gabor filter and a machine learning method. We cast the fence removal problem in an optimization framework, by modeling the formation of the degraded observations. The inverse problem is solved using split Bregman technique assuming total variation of the de-fenced image as the regularization constraint.

[1]  Mohan S. Kankanhalli,et al.  Seeing through the fence: Image de-fencing using a video sequence , 2013, 2013 IEEE International Conference on Image Processing.

[2]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[3]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Carola-Bibiane Schönlieb,et al.  Combined First and Second Order Total Variation Inpainting using Split Bregman , 2013, Image Process. Line.

[7]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2008, Commun. ACM.

[8]  Pascal Getreuer,et al.  Total Variation Inpainting using Split Bregman , 2012, Image Process. Line.

[9]  Yanxi Liu,et al.  Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[11]  Yuanjie Zheng,et al.  Learning based digital matting , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Wei Liu,et al.  Video De-Fencing , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Yanxi Liu,et al.  Image De-fencing Revisited , 2010, ACCV.

[14]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[15]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[16]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  J. Daugman Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[18]  Yanxi Liu,et al.  Image de-fencing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.