Texture Reconstruction Guided by a High-Resolution Patch

In this paper, we aim at super-resolving a low-resolution texture under the assumption that a high-resolution patch of the texture is available. To do so, we propose a variational method that combines two approaches that are texture synthesis and image reconstruction. The resulting objective function holds a nonconvex energy that involves a quadratic distance to the low-resolution image, a histogram-based distance to the high-resolution patch, and a nonlocal regularization that links the missing pixels with the patch pixels. As for the histogram-based measure, we use a sum of Wasserstein distances between the histograms of some linear transformations of the textures. The resulting optimization problem is efficiently solved with a primal-dual proximal method. Experiments show that our method leads to a significant improvement, both visually and numerically, with respect to the state-of-the-art algorithms for solving similar problems.

[1]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[2]  Andreas Weinmann,et al.  Jump-Sparse and Sparse Recovery Using Potts Functionals , 2013, IEEE Transactions on Signal Processing.

[3]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[4]  Karen O. Egiazarian,et al.  Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction , 2013, IEEE Transactions on Image Processing.

[5]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[6]  Christoph Schnörr,et al.  Convex Variational Image Restoration with Histogram Priors , 2013, SIAM J. Imaging Sci..

[7]  Camille Couprie,et al.  Dual constrained TV-based regularization , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  Alan C. Bovik,et al.  Range image quality assessment by Structural Similarity , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  Yann Gousseau,et al.  Constrained Sparse Texture Synthesis , 2013, SSVM.

[11]  Dani Lischinski,et al.  Self Tuning Texture Optimization , 2015, Comput. Graph. Forum.

[12]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[13]  Sylvain Lefebvre,et al.  Parallel controllable texture synthesis , 2005, ACM Trans. Graph..

[14]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[15]  Xianghua Xie,et al.  A Review of Recent Advances in Surface Defect Detection using Texture analysis Techniques , 2008 .

[16]  Long Quan,et al.  Image deblurring with blurred/noisy image pairs , 2007, SIGGRAPH 2007.

[17]  Nelly Pustelnik,et al.  A Nonlocal Structure Tensor-Based Approach for Multicomponent Image Recovery Problems , 2014, IEEE Transactions on Image Processing.

[18]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[19]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[20]  Eero P. Simoncelli,et al.  Image denoising via adjustment of wavelet coefficient magnitude correlation , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[21]  J. Moreau Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[22]  Michael Möller,et al.  Low Rank Priors for Color Image Regularization , 2015, EMMCVPR.

[23]  Luc Van Gool,et al.  Jointly Optimized Regressors for Image Super‐resolution , 2015, Comput. Graph. Forum.

[24]  D. Wildenschild,et al.  X-ray imaging and analysis techniques for quantifying pore-scale structure and processes in subsurface porous medium systems , 2013 .

[25]  I. M. Otivation Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems , 2018 .

[26]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[27]  Patrick L. Combettes,et al.  Image restoration subject to a total variation constraint , 2004, IEEE Transactions on Image Processing.

[28]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[29]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[30]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[31]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[32]  G. Burton TOPICS IN OPTIMAL TRANSPORTATION (Graduate Studies in Mathematics 58) By CÉDRIC VILLANI: 370 pp., US$59.00, ISBN 0-8218-3312-X (American Mathematical Society, Providence, RI, 2003) , 2004 .

[33]  Thomas S. Huang,et al.  Coupled Dictionary Training for Image Super-Resolution , 2012, IEEE Transactions on Image Processing.

[34]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[35]  Bruno Galerne,et al.  Random Phase Textures: Theory and Synthesis , 2011, IEEE Transactions on Image Processing.

[36]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[37]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[38]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[39]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Michael Möller,et al.  The Primal-Dual Hybrid Gradient Method for Semiconvex Splittings , 2014, SIAM J. Imaging Sci..

[41]  Mathews Jacob,et al.  Higher Degree Total Variation (HDTV) Regularization for Image Recovery , 2012, IEEE Transactions on Image Processing.

[42]  Shunsuke Ono,et al.  Second-order total Generalized Variation constraint , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Marcus A. Magnor,et al.  ZIPMAPS: Zoom-Into-Parts Texture Maps , 2010, VMV.

[44]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[45]  Daniel Cremers,et al.  Video Super Resolution Using Duality Based TV-L1 Optical Flow , 2009, DAGM-Symposium.

[46]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[47]  Mathews Jacob,et al.  Generalized Higher Degree Total Variation (HDTV) Regularization , 2014, IEEE Transactions on Image Processing.

[48]  Yann Gousseau,et al.  Variational Texture Synthesis with Sparsity and Spectrum Constraints , 2015, Journal of Mathematical Imaging and Vision.

[49]  Julien Rabin,et al.  Wasserstein regularization of imaging problem , 2011, 2011 18th IEEE International Conference on Image Processing.

[50]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[51]  Eitan Grinspun,et al.  Multiscale texture synthesis , 2008, ACM Trans. Graph..

[52]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[54]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[55]  Horst Bischof,et al.  Fast and accurate image upscaling with super-resolution forests , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Daniel Cremers,et al.  Real-Time Minimization of the Piecewise Smooth Mumford-Shah Functional , 2014, ECCV.

[57]  Jean-François Aujol,et al.  Image Zoom Completion , 2016, IEEE Transactions on Image Processing.

[58]  Ronan Fablet,et al.  Missing data super-resolution using non-local and statistical priors , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[59]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[60]  Nelly Pustelnik,et al.  Epigraphical Projection and Proximal Tools for Solving Constrained Convex Optimization Problems: Part I , 2012, ArXiv.

[61]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[62]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[63]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[64]  Arjan Kuijper,et al.  Scale Space and Variational Methods in Computer Vision , 2013, Lecture Notes in Computer Science.

[65]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.

[66]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[67]  Luc Van Gool,et al.  The Synthesizability of Texture Examples , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.