Image Denoising by Exploring External and Internal Correlations

Single image denoising suffers from limited data collection within a noisy image. In this paper, we propose a novel image denoising scheme, which explores both internal and external correlations with the help of web images. For each noisy patch, we build internal and external data cubes by finding similar patches from the noisy and web images, respectively. We then propose reducing noise by a two-stage strategy using different filtering approaches. In the first stage, since the noisy patch may lead to inaccurate patch selection, we propose a graph based optimization method to improve patch matching accuracy in external denoising. The internal denoising is frequency truncation on internal cubes. By combining the internal and external denoising patches, we obtain a preliminary denoising result. In the second stage, we propose reducing noise by filtering of external and internal cubes, respectively, on transform domain. In this stage, the preliminary denoising result not only enhances the patch matching accuracy but also provides reliable estimates of filtering parameters. The final denoising image is obtained by fusing the external and internal filtering results. Experimental results show that our method constantly outperforms state-of-the-art denoising schemes in both subjective and objective quality measurements, e.g., it achieves >2 dB gain compared with BM3D at a wide range of noise levels.

[1]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Andrew Zisserman,et al.  Get Out of my Picture! Internet-based Inpainting , 2009, BMVC.

[4]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Anand Rangarajan,et al.  Image Denoising Using the Higher Order Singular Value Decomposition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[7]  David Zhang,et al.  Two-stage image denoising by principal component analysis with local pixel grouping , 2010, Pattern Recognit..

[8]  David Zhang,et al.  Texture Enhanced Image Denoising via Gradient Histogram Preservation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[11]  Tat-Jun Chin,et al.  Accelerated Hypothesis Generation for Multi-structure Robust Fitting , 2010, ECCV.

[12]  Ce Liu,et al.  Deformable Spatial Pyramid Matching for Fast Dense Correspondences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[14]  Xiaoyan Sun,et al.  CID: Combined Image Denoising in Spatial and Frequency Domains Using Web Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Stefan Harmeling,et al.  Learning How to Combine Internal and External Denoising Methods , 2013, GCPR.

[16]  James Hays,et al.  Super-resolution from internet-scale scene matching , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[17]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[18]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Michal Irani,et al.  Combining the power of Internal and External denoising , 2013, IEEE International Conference on Computational Photography (ICCP).

[20]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[21]  Xiaoyan Sun,et al.  Cloud-Based Image Coding for Mobile Devices—Toward Thousands to One Compression , 2013, IEEE Transactions on Multimedia.

[22]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[23]  Tao Gan,et al.  Image denoising using multi-stage sparse representations , 2010, 2010 IEEE International Conference on Image Processing.

[24]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[25]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[26]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[27]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[28]  Kostadin Dabov,et al.  BM3D Image Denoising with Shape-Adaptive Principal Component Analysis , 2009 .

[29]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[31]  Anton Osokin,et al.  Fast Approximate Energy Minimization with Label Costs , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[33]  Richard G. Baraniuk,et al.  Improved wavelet denoising via empirical Wiener filtering , 1997, Optics & Photonics.

[34]  Michal Irani,et al.  Separating Signal from Noise Using Patch Recurrence across Scales , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Nenghai Yu,et al.  Large scale image retrieval with visual groups , 2013, 2013 IEEE International Conference on Image Processing.

[36]  Xiaoyan Sun,et al.  Landmark Image Super-Resolution by Retrieving Web Images , 2013, IEEE Transactions on Image Processing.

[37]  Dani Lischinski,et al.  Deblurring by Example Using Dense Correspondence , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Stephen Lin,et al.  Intrinsic colorization , 2008, ACM Trans. Graph..

[39]  Truong Q. Nguyen,et al.  Image denoising by targeted external databases , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[41]  Stefan Harmeling,et al.  Improving Denoising Algorithms via a Multi-scale Meta-procedure , 2011, DAGM-Symposium.

[42]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[43]  Michal Irani,et al.  Internal statistics of a single natural image , 2011, CVPR 2011.

[44]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[45]  Marc Alexa,et al.  Photosketcher: Interactive Sketch-Based Image Synthesis , 2011, IEEE Computer Graphics and Applications.