Efficient Learning of Image Super-Resolution and Compression Artifact Removal with Semi-Local Gaussian Processes

Improving the quality of degraded images is a key problem in image processing, but the breadth of the problem leads to domain-specific approaches for tasks such as super-resolution and compression artifact removal. Recent approaches have shown that a general approach is possible by learning application-specific models from examples; however, learning models sophisticated enough to generate high-quality images is computationally expensive, and so specific per-application or per-dataset models are impractical. To solve this problem, we present an efficient semi-local approximation scheme to large-scale Gaussian processes. This allows efficient learning of task-specific image enhancements from example images without reducing quality. As such, our algorithm can be easily customized to specific applications and datasets, and we show the efficiency and effectiveness of our approach across five domains: single-image super-resolution for scene, human face, and text images, and artifact removal in JPEG- and JPEG 2000-encoded images.

[1]  Boris Polyak,et al.  The method of projections for finding the common point of convex sets , 1967 .

[2]  B. Silverman,et al.  Spline Smoothing: The Equivalent Variable Kernel Method , 1984 .

[3]  Hervé Bourlard,et al.  Connectionist Speech Recognition: A Hybrid Approach , 1993 .

[4]  Nikolas P. Galatsanos,et al.  Projection-based spatially adaptive reconstruction of block transform compressed images , 1994, Other Conferences.

[5]  Nikolas P. Galatsanos,et al.  Projection-based spatially adaptive reconstruction of block-transform compressed images , 1995, IEEE Trans. Image Process..

[6]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Aria Nosratinia.  Denoising JPEG images by re-application of JPEG , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[8]  Guoping Qiu,et al.  MLP for adaptive postprocessing block-coded images , 2000, IEEE Trans. Circuits Syst. Video Technol..

[9]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Aria Nosratinia,et al.  Enhancement of JPEG-Compressed Images by Re-application of JPEG , 2001, J. VLSI Signal Process..

[11]  Christopher M. Bishop,et al.  Bayesian Image Super-Resolution , 2002, NIPS.

[12]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[13]  Neil D. Lawrence,et al.  Fast Sparse Gaussian Process Methods: The Informative Vector Machine , 2002, NIPS.

[14]  Neil D. Lawrence,et al.  Fast Forward Selection to Speed Up Sparse Gaussian Process Regression , 2003, AISTATS.

[15]  Aria Nosratinia.  Postprocessing of JPEG-2000 images to remove compression artifacts , 2003, IEEE Signal Processing Letters.

[16]  Stephen J. Roberts,et al.  A Sampled Texture Prior for Image Super-Resolution , 2003, NIPS.

[17]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[18]  Bryan C. Russell,et al.  Exploiting the sparse derivative prior for super-resolution , 2003 .

[19]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[20]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[21]  Zhouchen Lin,et al.  Response to the Comments on "Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation' , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Bernhard Schölkopf,et al.  Face Detection - Efficient and Rank Deficient , 2004, NIPS.

[24]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[25]  Christopher K. I. Williams,et al.  Using the Equivalent Kernel to Understand Gaussian Process Regression , 2004, NIPS.

[26]  Xin Li.  Improved wavelet decoding via set theoretic estimation , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Taejeong Kim,et al.  Regression-based prediction for blocking artifact reduction in JPEG-compressed images , 2005, IEEE Transactions on Image Processing.

[28]  Gavin C. Cawley,et al.  Estimating Predictive Variances with Kernel Ridge Regression , 2005, MLCW.

[29]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[30]  Rachid Deriche,et al.  Vector-valued image regularization with PDEs: a common framework for different applications , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[32]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[33]  Deqing Sun,et al.  Postprocessing of Low Bit-Rate Block DCT Coded Images Based on a Fields of Experts Prior , 2007, IEEE Transactions on Image Processing.

[34]  Karen O. Egiazarian,et al.  Pointwise Shape-Adaptive DCT for High-Quality Denoising and Deblocking of Grayscale and Color Images , 2007, IEEE Transactions on Image Processing.

[35]  Peter J. Liu,et al.  Using Gaussian process regression to denoise images and remove artefacts from microarray data , 2007 .

[36]  Zoubin Ghahramani,et al.  Local and global sparse Gaussian process approximations , 2007, AISTATS.

[37]  Bernhard Schölkopf,et al.  Automatic 3D face reconstruction from single images or video , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[38]  Guangtao Zhai,et al.  JPEG2000 image postprocessing with novel trilateral deringing filter , 2008 .

[39]  Bernhard Schölkopf,et al.  Sparse multiscale gaussian process regression , 2008, ICML '08.

[40]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[41]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[42]  Jianfei Cai,et al.  Efficient quadtree based block-shift filtering for deblocking and deringing , 2009, J. Vis. Commun. Image Represent..

[43]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Valero Laparra,et al.  Image Denoising with Kernels Based on Natural Image Relations , 2010, J. Mach. Learn. Res..

[45]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[46]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[47]  Kwang In Kim,et al.  Efficient Learning-based Image Enhancement: Application to Super-resolution and Compression Artifact Removal , 2012, BMVC.

[48]  Bruce H. Pillman.  Super-Resolution Imaging , 2013, J. Electronic Imaging.