Accurate imagery recovery using a multi-observation patch model

Abstract A patch-based non-local self-similarity prior underpins many of the current state-of-the-art results in image-recovery problems. The associated methods tend to exploit such priors either heuristically in terms of the correlations between similar patches, or implicitly using hand-crafted models. Both approaches have a limited ability to represent image-specific self-similarity statistics, which limits the accuracy of the results. To address this problem, we propose a novel multi-observation patch model (MOPM) for image recovery. The MOPM enables the recovery of a clean patch from multiple noisy observations by using a linear filtering operation on a specific manifold. More importantly, it can be adaptively learned from the intermediate recovered image with a latent variable-based Bayesian learning approach. Thus, the MOPM obtains better representation of the image-specific internal statistics. In addition, the MOPM is naturally integrated into a half-quadratic splitting framework, in which the MOPM can be constantly refined through iterations and ultimately produce promising results. The experimental results on denoising and compressive sensing demonstrate the effectiveness of the MOPM for image recovery.

[1]  David Zhang,et al.  Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Junzhou Huang,et al.  Compressive Sensing MRI with Wavelet Tree Sparsity , 2012, NIPS.

[3]  Michal Irani,et al.  Separating Signal from Noise Using Patch Recurrence across Scales , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Yuan Yan Tang,et al.  Simultaneous Spectral-Spatial Feature Selection and Extraction for Hyperspectral Images , 2019, IEEE Transactions on Cybernetics.

[5]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[6]  Suk-Ju Kang,et al.  Geodesic Path-Based Diffusion Acceleration for Image Denoising , 2018, IEEE Transactions on Multimedia.

[7]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[8]  John W. Fisher,et al.  Low level vision via switchable Markov random fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Lei Zhang,et al.  Exploiting Clustering Manifold Structure for Hyperspectral Imagery Super-Resolution , 2018, IEEE Transactions on Image Processing.

[10]  Lei Zhang,et al.  Robust Principal Component Analysis with Complex Noise , 2014, ICML.

[11]  Stamatios Lefkimmiatis,et al.  Universal Denoising Networks : A Novel CNN Architecture for Image Denoising , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Jian Sun,et al.  Image Completion Approaches Using the Statistics of Similar Patches , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Bo Du,et al.  Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding , 2015, Pattern Recognit..

[14]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[15]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[16]  Wei Wei,et al.  Reweighted laplace prior based hyperspectral compressive sensing for unknown sparsity , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[18]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[19]  Junzhou Huang,et al.  Learning with structured sparsity , 2009, ICML '09.

[20]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[21]  Marshall F. Tappen,et al.  Learning non-local range Markov Random field for image restoration , 2011, CVPR 2011.

[22]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[23]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Wei Wei,et al.  Exploring Structured Sparsity by a Reweighted Laplace Prior for Hyperspectral Compressive Sensing , 2016, IEEE Transactions on Image Processing.

[25]  Yuan Yuan,et al.  Learning From Errors in Super-Resolution , 2014, IEEE Transactions on Cybernetics.

[26]  Thomas S. Huang,et al.  Close the loop: Joint blind image restoration and recognition with sparse representation prior , 2011, 2011 International Conference on Computer Vision.

[27]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[28]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[29]  Volkan Cevher,et al.  Low-Dimensional Models for Dimensionality Reduction and Signal Recovery: A Geometric Perspective , 2010, Proceedings of the IEEE.

[30]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[31]  Xueming Qian,et al.  Efficient and Robust Image Coding and Transmission Based on Scrambled Block Compressive Sensing , 2018, IEEE Transactions on Multimedia.

[32]  Lei Zhang,et al.  External Patch Prior Guided Internal Clustering for Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Qi Gao,et al.  A generative perspective on MRFs in low-level vision , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[35]  T. Hesterberg,et al.  Least angle and ℓ1 penalized regression: A review , 2008, 0802.0964.

[36]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[37]  Jason Sheng-Hong Tsai,et al.  Evolutionary Fuzzy Block-Matching-Based Camera Raw Image Denoising , 2017, IEEE Transactions on Cybernetics.

[38]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[39]  Aswin C. Sankaranarayanan,et al.  SpaRCS: Recovering low-rank and sparse matrices from compressive measurements , 2011, NIPS.

[40]  Michal Irani,et al.  Internal statistics of a single natural image , 2011, CVPR 2011.

[41]  Wei Wei,et al.  Cluster Sparsity Field: An Internal Hyperspectral Imagery Prior for Reconstruction , 2018, International Journal of Computer Vision.

[42]  Dacheng Tao,et al.  DCT Regularized Extreme Visual Recovery , 2017, IEEE Transactions on Image Processing.

[43]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[44]  Thomas S. Huang,et al.  Image and Video Restorations via Nonlocal Kernel Regression , 2013, IEEE Transactions on Cybernetics.

[45]  Cong Wang,et al.  Intracluster Structured Low-Rank Matrix Analysis Method for Hyperspectral Denoising , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[47]  Wei Xiong,et al.  Stacked Convolutional Denoising Auto-Encoders for Feature Representation , 2017, IEEE Transactions on Cybernetics.

[48]  Dacheng Tao,et al.  Beyond RPCA: Flattening Complex Noise in the Frequency Domain , 2017, AAAI.

[49]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Guangming Shi,et al.  Compressive Sensing via Nonlocal Low-Rank Regularization , 2014, IEEE Transactions on Image Processing.