Efficient, Quantitative Numerical Methods for Statistical Image Deconvolution and Denoising

We review the development of efficient numerical methods for statistical multi-resolution estimation of optical imaging experiments. In principle, this involves constrained linear deconvolution and denoising, and so these types of problems can be formulated as convex constrained, or even unconstrained, optimization. We address two main challenges: first of these is to quantify convergence of iterative algorithms; the second challenge is to develop efficient methods for these large-scale problems without sacrificing the quantification of convergence. We review the state of the art for these challenges.

[1]  Paul Tseng,et al.  Exact Regularization of Convex Programs , 2007, SIAM J. Optim..

[2]  Gabriele Steidl,et al.  First order algorithms in variational image processing , 2014, ArXiv.

[3]  M. Ferris,et al.  Weak sharp minima in mathematical programming , 1993 .

[4]  D. Russell Luke,et al.  A globally linearly convergent method for pointwise quadratically supportable convex–concave saddle point problems , 2017, 1702.08770.

[5]  D. Russell Luke,et al.  Nonconvex Notions of Regularity and Convergence of Fundamental Algorithms for Feasibility Problems , 2012, SIAM J. Optim..

[6]  Marc Teboulle,et al.  Necessary conditions for linear convergence of iterated expansive, set-valued mappings , 2017, Math. Program..

[7]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[8]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[9]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[10]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[11]  J. Borwein,et al.  Convex Functions: Constructions, Characterizations and Counterexamples , 2010 .

[12]  S. Hell,et al.  Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. , 1994, Optics letters.

[13]  Bingsheng He,et al.  On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers , 2014, Numerische Mathematik.

[14]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[15]  Jonathan Eckstein.  Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[16]  R. Vershynin,et al.  A Randomized Kaczmarz Algorithm with Exponential Convergence , 2007, math/0702226.

[17]  Daniel Boley,et al.  Local Linear Convergence of the Alternating Direction Method of Multipliers on Quadratic or Linear Programs , 2013, SIAM J. Optim..

[18]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[19]  D. Russell Luke,et al.  Alternating Projections and Douglas-Rachford for Sparse Affine Feasibility , 2013, IEEE Transactions on Signal Processing.

[20]  Peter Richtárik,et al.  Accelerated, Parallel, and Proximal Coordinate Descent , 2013, SIAM J. Optim..

[21]  Axel Munk,et al.  Statistical Multiresolution Dantzig Estimation in Imaging: Fundamental Concepts and Algorithmic Framework , 2011, ArXiv.

[22]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[23]  D. Russell Luke,et al.  Quantitative Convergence Analysis of Iterated Expansive, Set-Valued Mappings , 2016, Math. Oper. Res..

[24]  Marc Teboulle,et al.  A simple algorithm for a class of nonsmooth convex-concave saddle-point problems , 2015, Oper. Res. Lett..

[25]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[26]  Francis Bach,et al.  SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives , 2014, NIPS.

[27]  Nikos Komodakis,et al.  Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems , 2014, IEEE Signal Process. Mag..

[28]  Shoham Sabach,et al.  Proximal Heterogeneous Block Implicit-Explicit Method and Application to Blind Ptychographic Diffraction Imaging , 2015, SIAM J. Imaging Sci..

[29]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[30]  Hung M. Phan,et al.  Linear convergence of the Douglas–Rachford method for two closed sets , 2014, 1401.6509.

[31]  D. Russell Luke,et al.  Local Linear Convergence of the ADMM/Douglas-Rachford Algorithms without Strong Convexity and Application to Statistical Imaging , 2015, SIAM J. Imaging Sci..

[32]  D. R. Luke,et al.  Block-Coordinate Primal-Dual Method for Nonsmooth Minimization over Linear Constraints , 2018, 1801.04782.

[33]  S. Hell,et al.  Fluorescence microscopy with diffraction resolution barrier broken by stimulated emission. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Antonin Chambolle,et al.  An introduction to continuous optimization for imaging , 2016, Acta Numerica.

[35]  Benar Fux Svaiter,et al.  On Weak Convergence of the Douglas-Rachford Method , 2010, SIAM J. Control. Optim..

[36]  Tong Zhang,et al.  Accelerating Stochastic Gradient Descent using Predictive Variance Reduction , 2013, NIPS.

[37]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..