Non-texture image inpainting using histogram of oriented gradients

Abstract This paper presents a novel and efficient algorithm for non-texture inpainting of images based on using the dominant orientation of local gradients. It first introduces the concept of a new matrix called orientation matrix and then uses it for faster and better inpainting. The process of propagating information is carried out by using a new formulation which leads to much more efficient computations than the previous methods. The gain is both in terms of computational complexity and visual quality. The promising results in contexts of text, scratch, and block loss inpainting demonstrate the effectiveness of the proposed method.

[1]  Christine Guillemot,et al.  Image Inpainting : Overview and Recent Advances , 2014, IEEE Signal Processing Magazine.

[2]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[3]  Farzin Yaghmaee,et al.  Exemplar-based image inpainting using svd-based approximation matrix and multi-scale analysis , 2017, Multimedia Tools and Applications.

[4]  Nizar Bouguila,et al.  Bandlet-based sparsity regularization in video inpainting , 2014, J. Vis. Commun. Image Represent..

[5]  Guillermo Sapiro,et al.  A Comprehensive Framework for Image Inpainting , 2010, IEEE Transactions on Image Processing.

[6]  Loris Nanni,et al.  Descriptors for image-based fingerprint matchers , 2009, Expert Syst. Appl..

[7]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[8]  François Malgouyres,et al.  Edge Direction Preserving Image Zooming: A Mathematical and Numerical Analysis , 2001, SIAM J. Numer. Anal..

[9]  Nizar Bouguila,et al.  Automatic Inpainting Scheme for Video Text Detection and Removal , 2013, IEEE Transactions on Image Processing.

[10]  Nikolay N. Ponomarenko,et al.  METRICS PERFORMANCE COMPARISON FOR COLOR IMAGE DATABASE , 2008 .

[11]  Folkmar Bornemann,et al.  Fast Image Inpainting Based on Coherence Transport , 2007, Journal of Mathematical Imaging and Vision.

[12]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Zhenjun Tang,et al.  Salt and pepper noise removal with image inpainting , 2015 .

[14]  Jian Zhao,et al.  Efficient Object-Based Video Inpainting , 2006, 2006 International Conference on Image Processing.

[15]  Choong-Woong Lee,et al.  Error concealment technique using projection data for block-based image coding , 1994, Other Conferences.

[16]  M. Melgosa.  Testing CIELAB-Based Color-Difference Formulas , 2000 .

[17]  Dong Liu,et al.  Inpainting with image patches for compression , 2016, J. Vis. Commun. Image Represent..

[18]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[19]  Xiaoli Huan,et al.  Image restoration based on the fast marching method and block based sampling , 2010, Comput. Vis. Image Underst..

[20]  Guillermo Sapiro,et al.  Video Inpainting Under Constrained Camera Motion , 2007, IEEE Transactions on Image Processing.

[21]  Farzin Yaghmaee,et al.  Application of GRNN neural network in non-texture image inpainting and restoration , 2015, Pattern Recognit. Lett..

[22]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[23]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[24]  Søren Holdt Jensen,et al.  Algorithms and software for total variation image reconstruction via first-order methods , 2009, Numerical Algorithms.

[25]  Andrew P. Bradley,et al.  Perceptual quality metrics applied to still image compression , 1998, Signal Process..

[26]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[27]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[28]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[29]  Zhiwei Xiong,et al.  Block-Based Image Compression With Parameter-Assistant Inpainting , 2010, IEEE Transactions on Image Processing.

[30]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[31]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[32]  Jayanta Mukhopadhyay,et al.  Modified Exemplar-Based Image Inpainting via Primal-Dual Optimization , 2015, PReMI.

[33]  Chen Wang,et al.  Image compression with structure-aware inpainting , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[34]  Jayanta Mukherjee,et al.  Combinatorial Exemplar-Based Image Inpainting , 2015, IWCIA.

[35]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Paul C. Miller,et al.  Full body image feature representations for gender profiling , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[37]  Jian Sun,et al.  Image Completion Approaches Using the Statistics of Similar Patches , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Byungjin Chung,et al.  Hybrid error concealment method combining exemplar-based image inpainting and spatial interpolation , 2014, Signal Process. Image Commun..

[39]  Nikos Komodakis,et al.  Image Completion Using Efficient Belief Propagation Via Priority Scheduling and Dynamic Pruning , 2007, IEEE Transactions on Image Processing.

[40]  Amit Phadikar,et al.  Roi based error concealment of compressed object based image using QIM data hiding and wavelet transform , 2010, IEEE Transactions on Consumer Electronics.

[41]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[42]  Yao Wang,et al.  Error control and concealment for video communication: a review , 1998, Proc. IEEE.

[43]  Homer H. Chen,et al.  Single Image Realism Assessment and Recoloring by Color Compatibility , 2012, IEEE Transactions on Multimedia.

[44]  Karol Myszkowski,et al.  Image Quality Metrics , 2000 .

[45]  Alexander Wong,et al.  A nonlocal-means approach to exemplar-based inpainting , 2008, 2008 15th IEEE International Conference on Image Processing.

[46]  Jayanta Mukherjee,et al.  Image Inpainting Through Metric Labeling via Guided Patch Mixing , 2016, IEEE Transactions on Image Processing.

[47]  Lucas Pereira,et al.  Image replacement through texture synthesis , 1997, Proceedings of International Conference on Image Processing.

[48]  David Tschumperlé,et al.  Fast Anisotropic Smoothing of Multi-Valued Images using Curvature-Preserving PDE's , 2006, International Journal of Computer Vision.

[49]  Tony F. Chan,et al.  Total Variation Wavelet Inpainting , 2006, Journal of Mathematical Imaging and Vision.

[50]  Takio Kurita,et al.  Selection of Histograms of Oriented Gradients Features for Pedestrian Detection , 2007, ICONIP.

[51]  Thomas Wiegand,et al.  Depth Image-Based Rendering With Advanced Texture Synthesis for 3-D Video , 2010, IEEE Transactions on Multimedia.

[52]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[53]  Massimo Fornasier,et al.  Nonlinear Projection Recovery in Digital Inpainting for Color Image Restoration , 2006, Journal of Mathematical Imaging and Vision.

[54]  Michael T. Orchard,et al.  Novel sequential error-concealment techniques using orientation adaptive interpolation , 2001, IEEE Trans. Circuits Syst. Video Technol..