Optimal Selection of the Regularization Function in a Weighted Total Variation Model. Part I: Modelling and Theory

A weighted total variation model with a spatially varying regularization weight is considered. Existence of a solution is shown, and the associated Fenchel predual problem is derived. For automatically selecting the regularization function, a bilevel optimization framework is proposed. In this context, the lower-level problem, which is parameterized by the regularization weight, is the Fenchel predual of the weighted total variation model and the upper-level objective penalizes violations of a variance corridor. The latter object relies on a localization of the image residual as well as on lower and upper bounds inspired by the statistics of the extremes.

[1]  Karl Kunisch,et al.  Total Bounded Variation Regularization as a Bilaterally Constrained Optimization Problem , 2004, SIAM J. Appl. Math..

[2]  V. Barbu,et al.  Convexity and optimization in banach spaces , 1972 .

[3]  Thomas Brox,et al.  Bilevel Optimization with Nonsmooth Lower Level Problems , 2015, SSVM.

[4]  C. Vogel.  Computational Methods for Inverse Problems , 1987 .

[5]  Mila Nikolova,et al.  Bilevel Image Denoising Using Gaussianity Tests , 2015, SSVM.

[6]  J. Rodrigues.  Obstacle Problems in Mathematical Physics , 1987 .

[7]  Khalid Jalalzai.  Discontinuities of the minimizers of the weighted or anisotropic total variation for image reconstruction , 2014, 1402.0026.

[8]  Michael Hintermüller,et al.  Expected absolute value estimators for a spatially adapted regularization parameter choice rule in L1-TV-based image restoration , 2010 .

[9]  Carola-Bibiane Schonlieb,et al.  Image denoising: learning noise distribution via PDE-constrained optimization , 2012, 1207.3425.

[10]  Yiqiu Dong,et al.  Automated Regularization Parameter Selection in Multi-Scale Total Variation Models for Image Restoration , 2011, Journal of Mathematical Imaging and Vision.

[11]  M. Hintermueller,et al.  Optimal selection of the regularization function in a generalized total variation model. Part I: Modelling and theory , 2016 .

[12]  Prashant A. Athavale,et al.  Weighted TV minimization and applications to vortex density models , 2015, 1509.03713.

[13]  Khalid Jalalzai,et al.  Regularization of inverse problems in image processing , 2012 .

[14]  Bernard Rougé,et al.  TV Based Image Restoration with Local Constraints , 2003, J. Sci. Comput..

[15]  U. Mosco.  Convergence of convex sets and of solutions of variational inequalities , 1969 .

[16]  Carola-Bibiane Schönlieb,et al.  Bilevel Parameter Learning for Higher-Order Total Variation Regularisation Models , 2015, Journal of Mathematical Imaging and Vision.

[17]  Stefan Roth,et al.  Shrinkage Fields for Effective Image Restoration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[19]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[20]  Karl Kunisch,et al.  A Bilevel Optimization Approach for Parameter Learning in Variational Models , 2013, SIAM J. Imaging Sci..

[21]  Lucio Boccardo,et al.  Homogenization of Nonlinear Unilateral Problems , 1991 .

[22]  Michal Kočvara,et al.  Nonsmooth approach to optimization problems with equilibrium constraints : theory, applications, and numerical results , 1998 .

[23]  Ivan P. Gavrilyuk,et al.  Variational analysis in Sobolev and BV spaces , 2007, Math. Comput..

[24]  J. C. De los Reyes,et al.  Learning optimal spatially-dependent regularization parameters in total variation image denoising , 2016, 1603.09155.

[25]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[26]  Rachid Deriche,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004 .

[27]  Yiqiu Dong,et al.  A Multi-Scale Vectorial Lτ-TV Framework for Color Image Restoration , 2011, International Journal of Computer Vision.

[28]  Axel Munk,et al.  Statistical Multiresolution Dantzig Estimation in Imaging: Fundamental Concepts and Algorithmic Framework , 2011, ArXiv.

[29]  E. Gumbel,et al.  Statistics of extremes , 1960 .

[30]  Carola-Bibiane Schönlieb,et al.  Bilevel approaches for learning of variational imaging models , 2015, ArXiv.

[31]  Thomas Pock,et al.  Continuous Hyper-parameter Learning for Support Vector Machines , 2015 .

[32]  F. Tröltzsch.  Optimal Control of Partial Differential Equations: Theory, Methods and Applications , 2010 .

[33]  Michael Hintermüller,et al.  An Infeasible Primal-Dual Algorithm for Total Bounded Variation-Based Inf-Convolution-Type Image Restoration , 2006, SIAM J. Sci. Comput..

[34]  C. Vogel,et al.  Analysis of bounded variation penalty methods for ill-posed problems , 1994 .

[35]  E. Gumbel,et al.  Les valeurs extrêmes des distributions statistiques , 1935 .

[36]  F. Smithies.  Linear Operators , 2019, Nature.

[37]  Gjlles Aubert,et al.  Mathematical problems in image processing , 2001 .

[38]  Andrés Almansa,et al.  A TV Based Restoration Model with Local Constraints , 2008, J. Sci. Comput..

[39]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[40]  Michael Hintermüller,et al.  Mathematical Programs with Complementarity Constraints in Function Space: C- and Strong Stationarity and a Path-Following Algorithm , 2009, SIAM J. Optim..

[41]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[42]  Carola-Bibiane Schönlieb,et al.  The structure of optimal parameters for image restoration problems , 2015, ArXiv.

[43]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[44]  Bethany L. Nicholson,et al.  Mathematical Programs with Equilibrium Constraints , 2021, Pyomo — Optimization Modeling in Python.

[45]  R. Showalter.  Monotone operators in Banach space and nonlinear partial differential equations , 1996 .

[46]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[47]  Ronald F. Gariepy.  FUNCTIONS OF BOUNDED VARIATION AND FREE DISCONTINUITY PROBLEMS (Oxford Mathematical Monographs) , 2001 .

[48]  Michael Hintermüller,et al.  Optimal Selection of the Regularization Function in a Weighted Total Variation Model. Part II: Algorithm, Its Analysis and Numerical Tests , 2016, Journal of Mathematical Imaging and Vision.

[49]  J. Zowe,et al.  Regularity and stability for the mathematical programming problem in Banach spaces , 1979 .

[50]  Tao Wu,et al.  Bilevel Optimization for Calibrating Point Spread Functions in Blind Deconvolution , 2015 .

[51]  V. Barbu.  Optimal control of variational inequalities , 1984 .

[52]  Michael Hintermüller,et al.  A Second Order Shape Optimization Approach for Image Segmentation , 2004, SIAM J. Appl. Math..

[53]  Wei Yu,et al.  On learning optimized reaction diffusion processes for effective image restoration , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  M. Hintermüller,et al.  Density of convex intersections and applications , 2016, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[55]  Jürgen Appell,et al.  Nonlinear superposition operators: Preface , 1990 .

[56]  E. Giusti.  Minimal surfaces and functions of bounded variation , 1977 .

[57]  Axel Munk,et al.  Locally adaptive image denoising by a statistical multiresolution criterion , 2012, Comput. Stat. Data Anal..

[58]  Vivette Girault,et al.  Finite Element Methods for Navier-Stokes Equations - Theory and Algorithms , 1986, Springer Series in Computational Mathematics.

[59]  Eduardo Casas,et al.  Optimal Control of Partial Differential Equations , 2017 .

[60]  Michael Hintermüller,et al.  On the density of classes of closed convex sets with pointwise constraints in Sobolev spaces , 2015 .

[61]  Boris S. Mordukhovich,et al.  Several approaches for the derivation of stationarity conditions for elliptic MPECs with upper-level control constraints , 2014, Math. Program..

[62]  C. Schönlieb,et al.  Image denoising: Learning the noise model via nonsmooth PDE-constrained optimization , 2013 .