Image Restoration: A Data-Driven Perspective

We are living in the era of big data. The discovery, interpretation and usage of the information, knowledge and resources hidden in all sorts of data to benefit human beings and to improve everyone's day to day life is a challenge to all of us. The huge amount of data we collect nowadays is so complicated, and yet what we expect from it is so much. It is hard to imagine that one can characterize these complicated data sets and solve real life problems by solving merely a few mathematical equations. However, generic mathematical models can be used to obtain a coarse level approximation (or low accuracy solution) to the answers we are seeking. The first question is how to use generic prior knowledge of the underlying solutions of the problem in hand and to set up a proper model for a good low level approximation? The second question is whether we are able to use the knowledge and information from the approximate solution derived from the given data to further improve the model itself so that more accurate solutions can be obtained? That is: how to engage an interactive data-driven approach to solve complex problems? As images are one of the most useful and commonly used types of data, in this article, we review the development of the wavelet frame (or more general redundant system) based approach for image restoration from a data-driven perspective. We will observe that a good system for approximating any function, including images, should be capable of e! ectively capturing both global patterns and local features of the function. A wavelet frame is one of the examples of such system. We will show how algorithms of wavelet frame based image restoration are developed via the generic knowledge of images. Then, we will show how specific information of ag iven image can be used to further improve the models and algorithms. Through this process, we shall reveal some insights and understandings of the wavelet frame based approach for image restoration. We hope that this also leads to new ideas on how to analyze more complex data set sg enerated from other real life problems.

[1]  R. Duffin,et al.  A class of nonharmonic Fourier series , 1952 .

[2]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[3]  M. Powell.  A method for nonlinear constraints in minimization problems , 1969 .

[4]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[5]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[6]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[7]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[8]  S. C. Malik,et al.  Introduction to convergence , 1984 .

[9]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[10]  Yves Meyer,et al.  Wavelets - tools for science and technology , 1987 .

[11]  I. Daubechies.  Orthonormal bases of compactly supported wavelets , 1988 .

[12]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[13]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[14]  S. Mallat.  Multiresolution approximations and wavelet orthonormal bases of L^2(R) , 1989 .

[15]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  L. Rudin,et al.  Feature-oriented image enhancement using shock filters , 1990 .

[17]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[18]  S. Jaffard.  Pointwise smoothness, two-microlocalization and wavelet coefficients , 1991 .

[19]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[20]  I. Johnstone,et al.  Maximum Entropy and the Nearly Black Object , 1992 .

[21]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[22]  G. D. Maso,et al.  An Introduction to-convergence , 1993 .

[23]  Y. Meyer.  Wavelets and Operators , 1993 .

[24]  I. Johnstone,et al.  Threshold selection for wavelet shrinkage of noisy data , 1994, Proceedings of 16th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  A. Ron,et al.  Weyl-Heisenberg Frames and Riesz Bases in L2(Rd). , 1994 .

[26]  A. Ron,et al.  Frames and Stable Bases for Shift-Invariant Subspaces of L2(ℝd) , 1995, Canadian Journal of Mathematics.

[27]  D. Donoho,et al.  Translation-Invariant De-Noising , 1995 .

[28]  A. Ron,et al.  Affine systems inL2 (ℝd) II: Dual systems , 1997 .

[29]  R. Tyrrell Rockafellar,et al.  Convergence Rates in Forward-Backward Splitting , 1997, SIAM J. Optim..

[30]  A. Ron,et al.  Weyl-Heisenberg frames and Riesz bases in $L_2(\mathbb{R}^d)$ , 1997 .

[31]  Jianqing Fan,et al.  Comments on «Wavelets in statistics: A review» by A. Antoniadis , 1997 .

[32]  A. Ron,et al.  Affine Systems inL2(Rd): The Analysis of the Analysis Operator , 1997 .

[33]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[34]  Zuowei Shen,et al.  Compactly supported tight affine spline frames in L2(Rd) , 1998, Math. Comput..

[35]  A. Chambolle.  FINITE-DIFFERENCES DISCRETIZATIONS OF THE MUMFORD-SHAH FUNCTIONAL , 1999 .

[36]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[37]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[38]  Karlheinz Gröchenig,et al.  Foundations of Time-Frequency Analysis , 2000, Applied and numerical harmonic analysis.

[39]  C. Chui,et al.  Compactly supported tight and sibling frames with maximum vanishing moments , 2001 .

[40]  G. Sapiro,et al.  Geometric partial differential equations and image analysis [Book Reviews] , 2001, IEEE Transactions on Medical Imaging.

[41]  Jianqing Fan,et al.  Regularization of Wavelet Approximations , 2001 .

[42]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[43]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[44]  I. Daubechies,et al.  Framelets: MRA-based constructions of wavelet frames☆☆☆ , 2003 .

[45]  Joseph F. Murray,et al.  Dictionary Learning Algorithms for Sparse Representation , 2003, Neural Computation.

[46]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[47]  Raymond H. Chan,et al.  Wavelet Algorithms for High-Resolution Image Reconstruction , 2002, SIAM J. Sci. Comput..

[48]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[49]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[50]  Robert D. Nowak,et al.  A bound optimization approach to wavelet-based image deconvolution , 2005, IEEE International Conference on Image Processing 2005.

[51]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[52]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[53]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[54]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[55]  T. Chan,et al.  Variational image inpainting , 2005 .

[56]  A. Ron,et al.  Generalized Shift-Invariant Systems , 2005 .

[57]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[58]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[59]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[60]  Jean-Luc Starck,et al.  Sparse representations and bayesian image inpainting , 2005 .

[61]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[62]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[63]  L. He,et al.  MR Image Reconstruction from Sparse Radial Samples Using Bregman Iteration , 2006 .

[64]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[65]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[66]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[67]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[68]  Wotao Yin,et al.  A Fixed-Point Continuation Method for L_1-Regularization with Application to Compressed Sensing , 2007 .

[69]  I. Daubechies,et al.  Iteratively solving linear inverse problems under general convex constraints , 2007 .

[70]  Zuowei Shen,et al.  PSEUDO-SPLINES, WAVELETS AND FRAMELETS , 2007 .

[71]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[72]  Jian-Feng Cai,et al.  A framelet-based image inpainting algorithm , 2008 .

[73]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[74]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[75]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[76]  K. Bredies.  A forward–backward splitting algorithm for the minimization of non-smooth convex functionals in Banach space , 2008, 0807.0778.

[77]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[78]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[79]  Mohamed-Jalal Fadili,et al.  Inpainting and Zooming Using Sparse Representations , 2009, Comput. J..

[80]  Jian-Feng Cai,et al.  Blind motion deblurring from a single image using sparse approximation , 2009, CVPR.

[81]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[82]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[83]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[84]  Zuowei Shen,et al.  Dual Wavelet Frames and Riesz Bases in Sobolev Spaces , 2009 .

[85]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[86]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[87]  Raymond H. Chan,et al.  Convergence analysis of tight framelet approach for missing data recovery , 2009, Adv. Comput. Math..

[88]  Jian-Feng Cai,et al.  Blind motion deblurring using multiple images , 2009, J. Comput. Phys..

[89]  Jian-Feng Cai,et al.  Split Bregman Methods and Frame Based Image Restoration , 2009, Multiscale Model. Simul..

[90]  Patrice Abry,et al.  Wavelet Leader multifractal analysis for texture classification , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[91]  Jian-Feng Cai,et al.  Linearized Bregman Iterations for Frame-Based Image Deblurring , 2009, SIAM J. Imaging Sci..

[92]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[93]  Zhaosong Lu,et al.  Penalty Decomposition Methods for $L0$-Norm Minimization , 2010, ArXiv.

[94]  Ronald R. Coifman,et al.  Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning , 2010, ICML.

[95]  Stanley Osher,et al.  Image Recovery via Nonlocal Operators , 2010, J. Sci. Comput..

[96]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[97]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[98]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[99]  Zuowei Shen,et al.  Robust video denoising using low rank matrix completion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[100]  Jian-Feng Cai,et al.  Framelet based deconvolution , 2010 .

[101]  Jian-Feng Cai,et al.  Simultaneous cartoon and texture inpainting , 2010 .

[102]  Lei Zhang,et al.  Sparsity-based image denoising via dictionary learning and structural clustering , 2011, CVPR 2011.

[103]  Zuowei Shen.  Wavelet Frames and Image Restorations , 2011 .

[104]  Hongkai Zhao,et al.  Robust principal component analysis-based four-dimensional computed tomography , 2011, Physics in medicine and biology.

[105]  Qingtang Jiang,et al.  Highly symmetric bi-frames for triangle surface multiresolution processing , 2011 .

[106]  Zuowei Shen,et al.  Robust Video Restoration by Joint Sparse and Low Rank Matrix Approximation , 2011, SIAM J. Imaging Sci..

[107]  Steve B. Jiang,et al.  GPU-based iterative cone-beam CT reconstruction using tight frame regularization , 2010, Physics in medicine and biology.

[108]  Kim-Chuan Toh,et al.  An Accelerated Proximal Gradient Algorithm for Frame-Based Image Restoration via the Balanced Approach , 2011, SIAM J. Imaging Sci..

[109]  Bin Dong,et al.  Fast Linearized Bregman Iteration for Compressive Sensing and Sparse Denoising , 2011, ArXiv.

[110]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[111]  Aichi Chien,et al.  Frame based segmentation for medical images , 2011 .

[112]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[113]  S. Osher,et al.  Image restoration: Total variation, wavelet frames, and beyond , 2012 .

[114]  Zuowei Shen,et al.  Wavelet frame based color image demosaicing , 2013 .

[115]  Bin Dong,et al.  ℓ0 Minimization for wavelet frame based image restoration , 2011, Math. Comput..

[116]  Zuowei Shen,et al.  Recovering Over-/Underexposed Regions in Photographs , 2013, SIAM J. Imaging Sci..

[117]  Dimitri Van De Ville,et al.  Tight Wavelet Frames on Multislice Graphs , 2013, IEEE Transactions on Signal Processing.

[118]  Bin Dong,et al.  X-Ray CT Image Reconstruction via Wavelet Frame Based Regularization and Radon Domain Inpainting , 2013, J. Sci. Comput..

[119]  Bin Dong,et al.  MRA-based wavelet frames and applications , 2013 .

[120]  Zhang Gong,et al.  Wavelet Frame Based Blind Image Inpainting , 2013 .

[121]  Fei Yang,et al.  Adaptive low rank and sparse decomposition of video using compressive sensing , 2013, 2013 IEEE International Conference on Image Processing.

[122]  Zuowei Shen,et al.  Wavelet Frame Based Multiphase Image Segmentation , 2013, SIAM J. Imaging Sci..

[123]  Hong Jiang,et al.  Surveillance Video Processing Using Compressive Sensing , 2012, ArXiv.

[124]  Bin Dong,et al.  An Efficient Algorithm for ℓ0 Minimization in Wavelet Frame Based Image Restoration , 2013, J. Sci. Comput..

[125]  B. Han.  Matrix splitting with symmetry and symmetric tight framelet filter banks with two high-pass filters , 2013 .

[126]  Ming Li,et al.  Wavelet Frame Based Algorithm for 3D Reconstruction in Electron Microscopy , 2014, SIAM J. Sci. Comput..

[127]  Hong Jiang,et al.  Surveillance Video Analysis Using Compressive Sensing With Low Latency , 2014, Bell Labs Tech. J..

[128]  Bin Dong.  Sparse Representation on Graphs by Tight Wavelet Frames and Applications , 2014, 1411.2643.

[129]  Kim-Chuan Toh,et al.  Image Restoration with Mixed or Unknown Noises , 2014, Multiscale Model. Simul..

[130]  Jian-Feng Cai,et al.  Data-driven tight frame construction and image denoising , 2014 .

[131]  Steve B. Jiang,et al.  Cine Cone Beam CT Reconstruction Using Low-Rank Matrix Factorization: Algorithm and a Proof-of-Principle Study , 2012, IEEE Transactions on Medical Imaging.

[132]  B. Han.  Symmetric tight framelet filter banks with three high-pass filters☆ , 2014 .

[133]  Zuowei Shen,et al.  L0 Norm Based Dictionary Learning by Proximal Methods with Global Convergence , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[134]  Qingtang Jiang,et al.  Tight wavelet frames in low dimensions with canonical filters , 2015, J. Approx. Theory.

[135]  Chenglong Bao,et al.  Convergence analysis for iterative data-driven tight frame construction scheme , 2015 .

[136]  Zuowei Shen,et al.  Data-Driven Multi-scale Non-local Wavelet Frame Construction and Image Recovery , 2014, Journal of Scientific Computing.

[137]  Jian-Feng Cai,et al.  Image restoration: A wavelet frame based model for piecewise smooth functions and beyond , 2016 .

[138]  Zuowei Shen,et al.  Duality for Frames , 2016 .

[139]  Zuowei Shen,et al.  Dual Gramian analysis: Duality principle and unitary extension principle , 2015, Math. Comput..

[140]  Hui Ji,et al.  Image recovery via geometrically structured approximation , 2016 .

[141]  E Weinan,et al.  Multiscale Adaptive Representation of Signals: I. The Basic Framework , 2015, J. Mach. Learn. Res..

[142]  Zuowei Shen,et al.  Multiscale representation of surfaces by tight wavelet frames with applications to denoising , 2016 .

[143]  Bin Dong,et al.  Image Restoration: Wavelet Frame Shrinkage, Nonlinear Evolution PDEs, and Beyond , 2017, Multiscale Model. Simul..

[144]  Hui Ji,et al.  Directional Frames for Image Recovery: Multi-scale Discrete Gabor Frames , 2016, Journal of Fourier Analysis and Applications.