Bilevel Optimization for Calibrating Point Spread Functions in Blind Deconvolution

Blind deconvolution problems arise in many imaging modalities, where both the underlying point spread function, which parameterizes the convolution operator, and the source image need to be identified. In this work, a novel bilevel optimization approach to blind deconvolution is proposed. The lower-level problem refers to the minimization of a total-variation model, as is typically done in non-blind image deconvolution. The upper-level objective takes into account additional statistical information depending on the particular imaging modality. Bilevel problems of such type are investigated systematically. Analytical properties of the lower-level solution mapping are established based on Robinson's strong regularity condition. Furthermore, several stationarity conditions are derived from the variational geometry induced by the lower-level problem. Numerically, a projected-gradient-type method is employed to obtain a Clarke-type stationary point and its convergence properties are analyzed. We also implement an efficient version of the proposed algorithm and test it through the experiments on point spread function calibration and multiframe blind deconvolution.

[1]  Ronny Ramlau,et al.  A non-iterative regularization approach to blind deconvolution , 2006 .

[2]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[3]  Jian-Feng Cai,et al.  Blind motion deblurring using multiple images , 2009, J. Comput. Phys..

[4]  Michael Hintermüller,et al.  A bundle-free implicit programming approach for a class of elliptic MPECs in function space , 2016, Mathematical Programming.

[5]  ADAM B. LEVY,et al.  Solution Sensitivity from General Principles , 2001, SIAM J. Control. Optim..

[6]  Mostafa Kaveh,et al.  A regularization approach to joint blur identification and image restoration , 1996, IEEE Trans. Image Process..

[7]  Bethany L. Nicholson,et al.  Mathematical Programs with Equilibrium Constraints , 2021, Pyomo — Optimization Modeling in Python.

[8]  Jean-Luc Starck,et al.  Deconvolution and Blind Deconvolution in Astronomy , 2007 .

[9]  Alexander Shapiro,et al.  Sensitivity Analysis of Parameterized Variational Inequalities , 2005, Math. Oper. Res..

[10]  Stephen M. Robinson,et al.  Strongly Regular Generalized Equations , 1980, Math. Oper. Res..

[11]  Deepa Kundur,et al.  Blind image deconvolution revisited , 1996 .

[12]  R. Tyrrell Rockafellar,et al.  Robinson’s implicit function theorem and its extensions , 2008, Math. Program..

[13]  Alfred S. Carasso,et al.  Apex Blind Deconvolution of Color Hubble Space Telescope Imagery and Other Astronomical Data , 2006 .

[14]  Jirí V. Outrata,et al.  A Generalized Mathematical Program with Equilibrium Constraints , 2000, SIAM J. Control. Optim..

[15]  Michael Hintermüller,et al.  Mathematical Programs with Complementarity Constraints in Function Space: C- and Strong Stationarity and a Path-Following Algorithm , 2009, SIAM J. Optim..

[16]  Alfred S. Carasso,et al.  False Characteristic Functions and Other Pathologies in Variational Blind Deconvolution. A Method of Recovery , 2009, SIAM J. Appl. Math..

[17]  Stefan Scholtes,et al.  Mathematical Programs with Complementarity Constraints: Stationarity, Optimality, and Sensitivity , 2000, Math. Oper. Res..

[18]  Gjlles Aubert,et al.  Mathematical problems in image processing , 2001 .

[19]  K. Kunisch,et al.  An active set strategy based on the augmented Lagrangian formulation for image restoration , 1999 .

[20]  Stefan Scholtes,et al.  Convergence Properties of a Regularization Scheme for Mathematical Programs with Complementarity Constraints , 2000, SIAM J. Optim..

[21]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[22]  Alfred S. Carasso,et al.  Direct Blind Deconvolution , 2001, SIAM J. Appl. Math..

[23]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[24]  C. Schönlieb,et al.  Image denoising: Learning the noise model via nonsmooth PDE-constrained optimization , 2013 .

[25]  Michael Hintermüller,et al.  A superlinearly convergent R-regularized Newton scheme for variational models with concave sparsity-promoting priors , 2013, Computational Optimization and Applications.

[26]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[27]  H. V. Sorensen,et al.  An overview of sigma-delta converters , 1996, IEEE Signal Process. Mag..

[28]  Masao Fukushima,et al.  A Globally Convergent Sequential Quadratic Programming Algorithm for Mathematical Programs with Linear Complementarity Constraints , 1998, Comput. Optim. Appl..

[29]  Boris Polyak,et al.  B.S. Mordukhovich. Variational Analysis and Generalized Differentiation. I. Basic Theory, II. Applications , 2009 .

[30]  Luís B. Almeida,et al.  Blind and Semi-Blind Deblurring of Natural Images , 2010, IEEE Transactions on Image Processing.

[31]  Karl Kunisch,et al.  Total Bounded Variation Regularization as a Bilaterally Constrained Optimization Problem , 2004, SIAM J. Appl. Math..

[32]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Lin He,et al.  Blind deconvolution using TV regularization and Bregman iteration , 2005, Int. J. Imaging Syst. Technol..

[34]  K. Egiazarian,et al.  Blind image deconvolution , 2007 .

[35]  Alfred S. Carasso The APEX Method in Image Sharpening and the Use of Low Exponent L[e-acute]vy Stable Laws , 2003, SIAM J. Appl. Math..

[36]  Stuart Jefferies,et al.  A computational method for the restoration of images with an unknown, spatially-varying blur. , 2006, Optics express.

[37]  Tony F. Chan,et al.  Image processing and analysis , 2005 .

[38]  Anat Levin,et al.  Blind Motion Deblurring Using Image Statistics , 2006, NIPS.

[39]  E. A. Nurminskii Convergence of the gradient projection method , 1973 .

[40]  B. Mordukhovich Variational analysis and generalized differentiation , 2006 .

[41]  Deepa Kundur,et al.  Blind Image Deconvolution , 2001 .

[42]  Michal Kočvara,et al.  Nonsmooth approach to optimization problems with equilibrium constraints : theory, applications, and numerical results , 1998 .

[43]  D. A. Fish,et al.  Blind deconvolution by means of the Richardson-Lucy algorithm. , 1995 .

[44]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, SIGGRAPH 2008.

[45]  Sven Leyffer,et al.  Local Convergence of SQP Methods for Mathematical Programs with Equilibrium Constraints , 2006, SIAM J. Optim..

[46]  Otmar Scherzer,et al.  Regularization Methods for Blind Deconvolution and Blind Source Separation Problems , 2001, Math. Control. Signals Syst..

[47]  Jane J. Ye,et al.  Necessary and sufficient optimality conditions for mathematical programs with equilibrium constraints , 2005 .

[48]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[49]  Yasuyuki Matsushita,et al.  Removing Non-Uniform Motion Blur from Images , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[50]  Michael Hintermüller,et al.  Nonconvex TVq-Models in Image Restoration: Analysis and a Trust-Region Regularization-Based Superlinearly Convergent Solver , 2013, SIAM J. Imaging Sci..

[51]  Michael Hintermüller,et al.  An Infeasible Primal-Dual Algorithm for Total Bounded Variation-Based Inf-Convolution-Type Image Restoration , 2006, SIAM J. Sci. Comput..

[52]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[53]  S. M. Robinson Local structure of feasible sets in nonlinear programming, Part III: Stability and sensitivity , 1987 .

[54]  Karl Kunisch,et al.  A Bilevel Optimization Approach for Parameter Learning in Variational Models , 2013, SIAM J. Imaging Sci..

[55]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[56]  Jane J. Ye,et al.  Exact Penalization and Necessary Optimality Conditions for Generalized Bilevel Programming Problems , 1997, SIAM J. Optim..