Trust your source: quantifying source condition elements for variational regularisation methods

Source conditions are a key tool in variational regularisation to derive error estimates and convergence rates for ill-posed inverse problems. In this paper, we provide a recipe to practically compute source condition elements as the solution of convex minimisation problems that can be solved with first-order algorithms. We demonstrate the validity of our approach by testing it for two inverse problem case studies in machine learning and image processing: sparse coefficient estimation of a polynomial via LASSO regression and recovering an image from a subset of the coefficients of its Fourier transform. We further demonstrate that the proposed approach can easily be modified to solve the machine learning task of identifying the optimal sampling pattern in the Fourier domain for given image and variational regularisation method, which has applications in the context of sparsity promoting reconstruction from magnetic resonance imaging data.

[1]  M. Benning,et al.  Lifted Bregman Training of Neural Networks , 2022, ArXiv.

[2]  Luca Ratti,et al.  Shearlet-based regularization in statistical inverse learning with an application to x-ray tomography , 2021, Inverse Problems.

[3]  Carola-Bibiane Schönlieb,et al.  Learning convex regularizers satisfying the variational source condition for inverse problems , 2021, ArXiv.

[4]  Martin Burger,et al.  Convex regularization in statistical inverse learning problems , 2021, Inverse Problems and Imaging.

[5]  Guy Gilboa,et al.  Nonlinear Power Method for Computing Eigenvectors of Proximal Operators and Neural Networks , 2020, SIAM J. Imaging Sci..

[6]  Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging , 2021 .

[7]  Martin Benning,et al.  Bregman Methods for Large-Scale Optimisation with Applications in Imaging , 2021 .

[8]  Farid Bozorgnia,et al.  The Infinity Laplacian eigenvalue problem: reformulation and a numerical scheme , 2020, ArXiv.

[9]  Carola-Bibiane Schönlieb,et al.  Learning the Sampling Pattern for MRI , 2019, IEEE Transactions on Medical Imaging.

[10]  T. Hohage,et al.  Optimal convergence rates for sparsity promoting wavelet-regularization in Besov spaces , 2018, Inverse Problems.

[11]  Guy Gilboa,et al.  Nonlinear Eigenproblems in Image Processing and Computer Vision , 2018, Advances in Computer Vision and Pattern Recognition.

[12]  Martin Burger,et al.  Modern regularization methods for inverse problems , 2018, Acta Numerica.

[13]  Guy Gilboa,et al.  Flows Generating Nonlinear Eigenfunctions , 2016, J. Sci. Comput..

[14]  Martin Burger,et al.  Large noise in variational regularization , 2016, Transactions of Mathematics and Its Applications.

[15]  Thorsten Hohage,et al.  Characterizations of Variational Source Conditions, Converse Results, and Maxisets of Spectral Regularization Methods , 2016, SIAM J. Numer. Anal..

[16]  Martin Benning,et al.  Inverse scale space decomposition , 2016, 1612.09203.

[17]  M. Burger,et al.  Nonlinear Spectral Analysis via One-Homogeneous Functionals: Overview and Future Prospects , 2016, Journal of Mathematical Imaging and Vision.

[18]  Antonin Chambolle,et al.  An introduction to continuous optimization for imaging , 2016, Acta Numerica.

[19]  Michael Möller,et al.  Spectral Decompositions Using One-Homogeneous Functionals , 2016, SIAM J. Imaging Sci..

[20]  Lea Fleischer,et al.  Regularization of Inverse Problems , 1996 .

[21]  Leon Hirsch,et al.  Fundamentals Of Convex Analysis , 2016 .

[22]  Michael Möller,et al.  Spectral Representations of One-Homogeneous Functionals , 2015, SSVM.

[23]  Stephen J. Wright.  Coordinate descent algorithms , 2015, Mathematical Programming.

[24]  Guy Gilboa,et al.  A Total Variation Spectral Framework for Scale and Texture Analysis , 2014, SIAM J. Imaging Sci..

[25]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[26]  Guy Gilboa,et al.  Nonlinear band-pass filtering using the TV transform , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[27]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[28]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[29]  Martin Burger,et al.  Ground States and Singular Vectors of Convex Variational Regularization Methods , 2012, 1211.2057.

[30]  O. Scherzer,et al.  Necessary and sufficient conditions for linear convergence of ℓ1‐regularization , 2011 .

[31]  Martin Burger,et al.  ERROR ESTIMATES FOR GENERAL FIDELITIES , 2011 .

[32]  Ronny Ramlau,et al.  CONVERGENCE RATES FOR REGULARIZATION WITH SPARSITY CONSTRAINTS , 2010 .

[33]  Bernd Hofmann,et al.  Approximate source conditions for nonlinear ill-posed problems—chances and limitations , 2009 .

[34]  Lin He,et al.  Error estimation for Bregman iterations and inverse scale space methods in image restoration , 2007, Computing.

[35]  O. Scherzer,et al.  A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators , 2007 .

[36]  Bernd Hofmann,et al.  Approximate source conditions in Tikhonov regularization‐new analytical results and some numerical studies , 2006 .

[37]  E. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[38]  E. Resmerita.  Regularization of ill-posed problems in Banach spaces: convergence rates , 2005 .

[39]  S. Osher,et al.  Convergence rates of convex variational regularization , 2004 .

[40]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[41]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[42]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[43]  Ulrich Tautenhahn,et al.  Optimality for ill-posed problems under general source conditions , 1998 .

[44]  K. Kunisch,et al.  Regularization of linear least squares problems by total bounded variation , 1997 .

[45]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[46]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[47]  D. Donoho.  Superresolution via sparsity constraints , 1992 .

[48]  H. Engl,et al.  Convergence rates for Tikhonov regularisation of non-linear ill-posed problems , 1989 .

[49]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[50]  H W Engl,et al.  Convergence rates for Tikhonov regularisation of non-linear ill-posed problems , 1989 .

[51]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[52]  Mario Bertero,et al.  The Stability of Inverse Problems , 1980 .

[53]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[54]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[55]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[56]  A. Tikhonov.  On the stability of inverse problems , 1943 .