Low Dimensional Manifold Regularization Based Blind Image Inpainting and Non-Uniform Impulse Noise Recovery

Blind image inpainting is a challenging task in image processing. Motivated by the excellent performance of low dimensional manifold model (LDMM) in image inpainting for large-scale pixels missing, we introduce a novel blind inpainting model to repair images with missing pixels or damaged with impulse noise, in spite of the unknown locations of the corrupted pixels. We applied logarithmic transformation to separate the image and binary mask. LDMM regularization and $l_{0}$ norm regularization were applied respectively to simultaneously estimate the image and the mask. The resulted minimization problem was then solved by the split Bregman algorithm. The simulation results showed that the proposed model, compared with the existing ones, can effectively restore the image with large uniform and non-uniform missing rate.

[1]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[2]  Baining Guo,et al.  Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jiang Hsieh,et al.  CT Image Reconstruction in a Low Dimensional Manifold , 2017, ArXiv.

[4]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[5]  Tony F. Chan,et al.  Euler's Elastica and Curvature-Based Inpainting , 2003, SIAM J. Appl. Math..

[6]  Stanley Osher,et al.  Low dimensional manifold model in hyperspectral image reconstruction , 2016, Hyperspectral Image Analysis.

[7]  Aleksandra Pizurica,et al.  Crack detection and inpainting for virtual restoration of paintings: The case of the Ghent Altarpiece , 2013, Signal Process..

[8]  Lan Tang,et al.  Group-Based Sparse Representation Based on lp-Norm Minimization for Image Inpainting , 2020, IEEE Access.

[9]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[10]  Yi Wang,et al.  Image Inpainting Based on Multi-Patch Match with Adaptive Size , 2020, Applied Sciences.

[11]  Yucai Zhou,et al.  The research of image inpainting algorithm using self-adaptive group structure and sparse representation , 2019, Cluster Computing.

[12]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[13]  Zuowei Shen,et al.  WAVELET FRAME BASED IMAGE RESTORATION WITH MISSING / DAMAGED PIXELS , 2010 .

[14]  Rebecca Willett,et al.  Logarithmic total variation regularization for cross-validation in photon-limited imaging , 2013, 2013 IEEE International Conference on Image Processing.

[15]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[16]  Lei Zhang,et al.  Image Inpainting for Object Removal Based on Adaptive Two-Round Search Strategy , 2020, IEEE Access.

[17]  Xianfeng Zhao,et al.  A deep learning approach to patch-based image inpainting forensics , 2018, Signal Process. Image Commun..

[18]  Thomas S. Huang,et al.  Generative Image Inpainting with Contextual Attention , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Manya V. Afonso,et al.  Blind Inpainting Using $\ell _{0}$ and Total Variation Regularization , 2015, IEEE Transactions on Image Processing.

[20]  Zhen Li,et al.  A Convergent Point Integral Method for Isotropic Elliptic Equations on a Point Cloud , 2015, Multiscale Model. Simul..

[21]  Mohamed-Jalal Fadili,et al.  Multiplicative Noise Removal Using L1 Fidelity on Frame Coefficients , 2008, Journal of Mathematical Imaging and Vision.

[22]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[23]  Ting-Chun Wang,et al.  Image Inpainting for Irregular Holes Using Partial Convolutions , 2018, ECCV.

[24]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[25]  Anthony J. Yezzi,et al.  Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification , 2001, IEEE Trans. Image Process..

[26]  Stanley Osher,et al.  Low Dimensional Manifold Model for Image Processing , 2017, SIAM J. Imaging Sci..

[27]  Abdul Halim,et al.  A linear fourth-order PDE-based gray-scale image inpainting model , 2019, Computational and Applied Mathematics.

[28]  Gilad Lerman,et al.  Robust Locally Linear Analysis with Applications to Image Denoising and Blind Inpainting , 2013, SIAM J. Imaging Sci..

[29]  Sung-Jea Ko,et al.  PEPSI : Fast Image Inpainting With Parallel Decoding Network , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Hui Ji,et al.  Wavelet frame based blind image inpainting , 2012 .

[31]  Fan Yao,et al.  Damaged region filling by improved criminisi image inpainting algorithm for thangka , 2018, Cluster Computing.

[32]  Jun Liu,et al.  Nonlocal Patches based Gaussian Mixture Model for Image Inpainting , 2019, Applied Mathematical Modelling.

[33]  Lei Zhang,et al.  Image Inpainting Based on Exemplar and Sparse Representation , 2016 .

[34]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[35]  Jian Sun,et al.  Convergence of the point integral method for Laplace–Beltrami equation on point cloud , 2017, Research in the Mathematical Sciences.

[36]  Bernhard Schölkopf,et al.  Mask-Specific Inpainting with Deep Neural Networks , 2014, GCPR.

[37]  Jianhong Shen,et al.  Digital inpainting based on the Mumford–Shah–Euler image model , 2002, European Journal of Applied Mathematics.

[38]  G. Sridevi,et al.  Image Inpainting and Enhancement using Fractional Order Variational Model , 2017 .

[39]  Ming Yan,et al.  Restoration of Images Corrupted by Impulse Noise and Mixed Gaussian Impulse Noise using Blind Inpainting , 2013, SIAM J. Imaging Sci..

[40]  Abderrahim Elmoataz,et al.  Nonlocal video denoising, simplification and inpainting using discrete regularization on graphs , 2010, Signal Process..

[41]  Weilan Wang,et al.  Image Inpainting Based on Generative Adversarial Networks , 2018, 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[42]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..