Regularization Graphs -- A unified framework for variational regularization of inverse problems

We introduce and study a mathematical framework for a broad class of regularization functionals for ill-posed inverse problems: Regularization Graphs. Regularization graphs allow to construct functionals using as building blocks linear operators and convex functionals, assembled by means of operators that can be seen as generalizations of classical infimal convolution operators. This class of functionals exhaustively covers existing regularization approaches and it is flexible enough to craft new ones in a simple and constructive way. We provide well-posedness and convergence results with the proposed class of functionals in a general setting. Further, we consider a bilevel optimization approach to learn optimal weights for such regularization graphs from training data. We demonstrate that this approach is capable of optimizing the structure and the complexity of a regularization graph, allowing, for example, to automatically select a combination of regularizers that is optimal for given training data.

[1]  K. Bredies.  Symmetric tensor fields of bounded deformation , 2013 .

[2]  Lea Fleischer,et al.  Regularization of Inverse Problems , 1996 .

[3]  Yiming Gao,et al.  Infimal Convolution of Oscillation Total Generalized Variation for the Recovery of Images with Structured Texture , 2017, SIAM J. Imaging Sci..

[4]  Otmar Scherzer,et al.  Variational Methods in Imaging , 2008, Applied mathematical sciences.

[5]  Kristian Bredies,et al.  Higher-order total variation approaches and generalisations , 2019, Inverse Problems.

[6]  Valeriya Naumova,et al.  Multi-penalty regularization with a component-wise penalization , 2013 .

[7]  Carola-Bibiane Schönlieb,et al.  Higher-Order Total Directional Variation: Analysis , 2020, SIAM J. Imaging Sci..

[8]  K. Bredies,et al.  Infimal convolution of total generalized variation functionals for dynamic MRI , 2017, Magnetic resonance in medicine.

[9]  Carola-Bibiane Schönlieb,et al.  A total variation based regularizer promoting piecewise-Lipschitz reconstructions , 2019, SSVM.

[10]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[11]  Karl Kunisch,et al.  On Infimal Convolution of TV-Type Functionals and Applications to Video and Image Reconstruction , 2014, SIAM J. Imaging Sci..

[12]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[13]  K. Bredies,et al.  Regularization of linear inverse problems with total generalized variation , 2014 .

[14]  Xiaoping Yang,et al.  Infimal convolution type regularization of TGV and shearlet transform for image restoration , 2019, Comput. Vis. Image Underst..

[15]  Petros Maragos,et al.  Structure Tensor Total Variation , 2015, SIAM J. Imaging Sci..

[16]  J. Borwein,et al.  Convex Functions: Constructions, Characterizations and Counterexamples , 2010 .

[17]  Gitta Kutyniok,et al.  Clustered Sparsity and Separation of Cartoon and Texture , 2012, SIAM J. Imaging Sci..

[18]  Carola-Bibiane Schönlieb,et al.  Bilevel Parameter Learning for Higher-Order Total Variation Regularisation Models , 2015, Journal of Mathematical Imaging and Vision.

[19]  K. Bredies,et al.  Inverse problems in spaces of measures , 2013 .

[20]  H. Brezis.  Functional Analysis, Sobolev Spaces and Partial Differential Equations , 2010 .

[21]  Wotao Yin,et al.  A New Detail-Preserving Regularization Scheme , 2014, SIAM J. Imaging Sci..

[22]  Irene Fonseca,et al.  Adaptive Image Processing: First Order PDE Constraint Regularizers and a Bilevel Training Scheme , 2019, Journal of Nonlinear Science.

[23]  Maïtine Bergounioux,et al.  Mathematical Image Processing , 2011 .

[24]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[25]  O. Scherzer,et al.  Texture Enhancing Based on Variational Image Decomposition , 2011 .

[26]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[27]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[28]  Otmar Scherzer,et al.  Variational Methods on the Space of Functions of Bounded Hessian for Convexification and Denoising , 2005, Computing.

[29]  Martin Burger,et al.  Unified Models for Second-Order TV-Type Regularisation in Imaging: A New Perspective Based on Vector Operators , 2018, Journal of Mathematical Imaging and Vision.

[30]  Carola-Bibiane Schönlieb,et al.  Bilevel approaches for learning of variational imaging models , 2015, ArXiv.

[31]  Jean-Michel Morel,et al.  Secrets of image denoising cuisine* , 2012, Acta Numerica.

[32]  Carola-Bibiane Schönlieb,et al.  A Combined First and Second Order Variational Approach for Image Reconstruction , 2012, Journal of Mathematical Imaging and Vision.

[33]  A. Chambolle,et al.  A Convex Variational Model for Learning Convolutional Image Atoms from Incomplete Data , 2018, Journal of Mathematical Imaging and Vision.

[34]  Martin Burger,et al.  Infimal convolution regularisation functionals of BV and $\mathrm{L}^{p}$ spaces. Part I: The finite $p$ case , 2015, 1504.01956.

[35]  Otmar Scherzer,et al.  The residual method for regularizing ill-posed problems , 2009, Appl. Math. Comput..

[36]  Bangti Jin,et al.  Multi-Parameter Tikhonov Regularization , 2011, ArXiv.

[37]  Tony F. Chan,et al.  A fourth order dual method for staircase reduction in texture extraction and image restoration problems , 2010, 2010 IEEE International Conference on Image Processing.

[38]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[39]  O. Scherzer,et al.  A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators , 2007 .

[40]  Xavier Bresson,et al.  Adaptive Regularization With the Structure Tensor , 2015, IEEE Transactions on Image Processing.

[41]  R. Tyrrell Rockafellar.  Conjugate Duality and Optimization , 1974 .

[42]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[43]  Carola-Bibiane Schönlieb,et al.  Infimal Convolution Regularisation Functionals of BV and Lp Spaces. The Case p=∞ , 2015, System Modelling and Optimization.

[44]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2002, IEEE Trans. Image Process..

[45]  H. Attouch,et al.  Duality for the Sum of Convex Functions in General Banach Spaces , 1986 .

[46]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[47]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .