Bregman Methods for Large-Scale Optimisation with Applications in Imaging

[1]  Stephen P. Boyd,et al.  Stochastic Mirror Descent in Variationally Coherent Optimization Problems , 2017, NIPS.

[2]  Amir Beck,et al.  First-Order Methods in Optimization , 2017 .

[3]  Antonin Chambolle,et al.  On the ergodic convergence rates of a first-order primal–dual algorithm , 2016, Math. Program..

[4]  Antonin Chambolle,et al.  Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications , 2017, SIAM J. Optim..

[5]  Masoud Ahookhosh,et al.  Multi-block Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization , 2019, Computational Optimization and Applications.

[6]  Wei Peng,et al.  Proximal-Like Incremental Aggregated Gradient Method with Linear Convergence Under Bregman Distance Growth Conditions , 2017, Math. Oper. Res..

[7]  E. Hairer,et al.  Geometric Numerical Integration: Structure Preserving Algorithms for Ordinary Differential Equations , 2004 .

[8]  Martin Burger,et al.  Modern regularization methods for inverse problems , 2018, Acta Numerica.

[9]  J. Frédéric Bonnans,et al.  A family of variable metric proximal methods , 1995, Math. Program..

[10]  Stephen J. Wright.  Coordinate descent algorithms , 2015, Mathematical Programming.

[11]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[12]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[13]  Giorgio C. Buttazzo,et al.  Variational Analysis in Sobolev and BV Spaces - Applications to PDEs and Optimization, Second Edition , 2014, MPS-SIAM series on optimization.

[14]  Y. Censor,et al.  An iterative row-action method for interval convex programming , 1981 .

[15]  Carola-Bibiane Schönlieb,et al.  Preconditioned ADMM with Nonlinear Operator Constraint , 2015, System Modelling and Optimization.

[16]  C. R. Rao,et al.  On the convexity of higher order Jensen differences based on entropy functions , 1982, IEEE Trans. Inf. Theory.

[17]  Shiqian Ma,et al.  Accelerated Linearized Bregman Method , 2011, J. Sci. Comput..

[18]  Michael Möller,et al.  Nonlinear Spectral Analysis via One-Homogeneous Functionals: Overview and Future Prospects , 2015, Journal of Mathematical Imaging and Vision.

[19]  Valeria Ruggiero,et al.  Inertial Variable Metric Techniques for the Inexact Forward-Backward Algorithm , 2018, SIAM J. Sci. Comput..

[20]  Y. Censor,et al.  Proximal minimization algorithm withD-functions , 1992 .

[21]  C. R. Rao,et al.  On the convexity of some divergence measures based on entropy functions , 1982, IEEE Trans. Inf. Theory.

[22]  V. Morozov.  Regularization of incorrectly posed problems and the choice of regularization parameter , 1966 .

[23]  Frank Nielsen,et al.  The Burbea-Rao and Bhattacharyya Centroids , 2010, IEEE Transactions on Information Theory.

[24]  Lorenzo Rosasco,et al.  Don't relax: early stopping for convex regularization , 2017, ArXiv.

[25]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[26]  Marc Teboulle,et al.  First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems , 2017, SIAM J. Optim..

[27]  Heinz H. Bauschke,et al.  Bregman Monotone Optimization Algorithms , 2003, SIAM J. Control. Optim..

[28]  Nobuo Yamashita,et al.  Block coordinate proximal gradient methods with variable Bregman functions for nonsmooth separable optimization , 2016, Math. Program..

[29]  P. L. Combettes,et al.  Variable metric forward–backward splitting with applications to monotone inclusions in duality , 2012, 1206.6791.

[30]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[31]  Michael Möller,et al.  The Primal-Dual Hybrid Gradient Method for Semiconvex Splittings , 2014, SIAM J. Imaging Sci..

[32]  S. Osher,et al.  Convergence rates of convex variational regularization , 2004 .

[33]  Jian-Feng Cai,et al.  Linearized Bregman Iterations for Frame-Based Image Deblurring , 2009, SIAM J. Imaging Sci..

[34]  Michael Möller,et al.  Proximal Backpropagation , 2017, ICLR.

[35]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[36]  Jian-Feng Cai,et al.  Convergence of the linearized Bregman iteration for ℓ1-norm minimization , 2009, Math. Comput..

[37]  Tianxiang Gao,et al.  On the Convergence of Randomized Bregman Coordinate Descent for Non-Lipschitz Composite Problems , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  Lin He,et al.  Error estimation for Bregman iterations and inverse scale space methods in image restoration , 2007, Computing.

[39]  Martin Benning,et al.  Joint phase reconstruction and magnitude segmentation from velocity-encoded MRI data , 2019, Time-dependent Problems in Imaging and Parameter Identification.

[40]  Dirk A. Lorenz,et al.  The Linearized Bregman Method via Split Feasibility Problems: Analysis and Generalizations , 2013, SIAM J. Imaging Sci..

[41]  T. Itoh,et al.  Hamiltonian-conserving discrete canonical equations based on variational difference quotients , 1988 .

[42]  Lin Xiao,et al.  Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization , 2009, J. Mach. Learn. Res..

[43]  Volker Grimm,et al.  Discrete gradient methods for solving variational image regularisation models , 2017 .

[44]  Krzysztof C. Kiwiel,et al.  Free-Steering Relaxation Methods for Problems with Strictly Convex Costs and Linear Constraints , 1997, Math. Oper. Res..

[45]  Carola-Bibiane Schönlieb,et al.  Variational Image Regularization with Euler's Elastica Using a Discrete Gradient Scheme , 2017, SIAM J. Imaging Sci..

[46]  Martin Benning,et al.  Choose Your Path Wisely: Gradient Descent in a Bregman Distance Framework , 2017, SIAM J. Imaging Sci..

[47]  P. Oswald,et al.  Convergence analysis for Kaczmarz-type methods in a Hilbert space framework , 2015 .

[48]  Peter Richtárik,et al.  Randomized Iterative Methods for Linear Systems , 2015, SIAM J. Matrix Anal. Appl..

[49]  G. Herman,et al.  Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. , 1970, Journal of theoretical biology.

[50]  Thomas Pock,et al.  Variational Networks: Connecting Variational Methods and Deep Learning , 2017, GCPR.

[51]  Arindam Banerjee,et al.  Bregman Alternating Direction Method of Multipliers , 2013, NIPS.

[52]  Stephan Antholzer,et al.  NETT: solving inverse problems with deep neural networks , 2018, Inverse Problems.

[53]  Otmar Scherzer,et al.  Convex Inverse Scale Spaces , 2007, SSVM.

[54]  Jean-Christophe Pesquet,et al.  Deep unfolding of a proximal interior point method for image restoration , 2018, Inverse Problems.

[55]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[56]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Babak Hassibi,et al.  Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization , 2018, ICLR.

[58]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[59]  Lorenzo Rosasco,et al.  Iterative Regularization via Dual Diagonal Descent , 2016, Journal of Mathematical Imaging and Vision.

[60]  Christian Clason,et al.  Primal-Dual Extragradient Methods for Nonlinear Nonsmooth PDE-Constrained Optimization , 2016, SIAM J. Optim..

[61]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[62]  Wotao Yin,et al.  Analysis and Generalizations of the Linearized Bregman Method , 2010, SIAM J. Imaging Sci..

[63]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[64]  Volkan Cevher,et al.  Mirrored Langevin Dynamics , 2018, NeurIPS.

[65]  Matthias Joachim Ehrhardt,et al.  A geometric integration approach to smooth optimisation: Foundations of the discrete gradient method , 2018, 1805.06444.

[66]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[67]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[68]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[69]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[70]  Martin Benning,et al.  Inverse scale space decomposition , 2016, 1612.09203.

[71]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[72]  Yuto Miyatake,et al.  On the equivalence between SOR-type methods for linear systems and the discrete gradient methods for gradient systems , 2017, J. Comput. Appl. Math..

[73]  Juan Peypouquet,et al.  Splitting Methods with Variable Metric for Kurdyka–Łojasiewicz Functions and General Convergence Rates , 2015, J. Optim. Theory Appl..

[74]  Martin Burger,et al.  ERROR ESTIMATES FOR GENERAL FIDELITIES , 2011 .

[75]  Martin Burger,et al.  Bregman Distances in Inverse Problems and Partial Differential Equations , 2015, 1505.05191.

[76]  Otmar Scherzer,et al.  Inverse Scale Space Theory for Inverse Problems , 2001, Scale-Space.

[77]  Carola-Bibiane Schonlieb,et al.  A Geometric Integration Approach to Nonsmooth, Nonconvex Optimisation , 2018, Foundations of Computational Mathematics.

[78]  Andreas Neubauer,et al.  On Nesterov acceleration for Landweber iteration of linear ill-posed problems , 2016 .

[79]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[80]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[81]  Otmar Scherzer,et al.  Inverse Total Variation Flow , 2007, Multiscale Model. Simul..

[82]  Tuomo Valkonen,et al.  A primal–dual hybrid gradient method for nonlinear operators with applications to MRI , 2013, 1309.5032.

[83]  T. Hohage,et al.  A Generalization of the Chambolle-Pock Algorithm to Banach Spaces with Applications to Inverse Problems , 2014, 1412.0126.

[84]  Mark W. Schmidt,et al.  Minimizing finite sums with the stochastic average gradient , 2013, Mathematical Programming.

[85]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[86]  A. Chambolle,et al.  On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm” , 2015, J. Optim. Theory Appl..

[87]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[88]  G. Quispel,et al.  Foundations of Computational Mathematics: Six lectures on the geometric integration of ODEs , 2001 .

[89]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[90]  Thinh T. Doan,et al.  Convergence of the Iterates in Mirror Descent Methods , 2018, IEEE Control Systems Letters.

[91]  Lorenzo Rosasco,et al.  Accelerated Iterative Regularization via Dual Diagonal Descent , 2019, SIAM J. Optim..

[92]  Christian Clason,et al.  Acceleration and Global Convergence of a First-Order Primal-Dual Method for Nonconvex Problems , 2018, SIAM J. Optim..

[93]  Arkadi Nemirovski,et al.  The Ordered Subsets Mirror Descent Optimization Method with Applications to Tomography , 2001, SIAM J. Optim..

[94]  Thomas Brox,et al.  Bilevel Optimization with Nonsmooth Lower Level Problems , 2015, SSVM.

[95]  Michael Möller,et al.  Spectral Decompositions Using One-Homogeneous Functionals , 2016, SIAM J. Imaging Sci..

[96]  Sajid Javed,et al.  On the Applications of Robust PCA in Image and Video Processing , 2018, Proceedings of the IEEE.

[97]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[98]  Stephen P. Boyd,et al.  A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights , 2014, J. Mach. Learn. Res..

[99]  Martin Benning,et al.  Gradient descent in a generalised Bregman distance framework , 2016 .

[100]  G. Quispel,et al.  Geometric integration using discrete gradients , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[101]  Marc Teboulle,et al.  A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications , 2017, Math. Oper. Res..

[102]  Peter Richtarik,et al.  Accelerated Bregman proximal gradient methods for relatively smooth convex optimization , 2018, Computational Optimization and Applications.

[103]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[104]  Simon R. Arridge,et al.  Solving inverse problems using data-driven models , 2019, Acta Numerica.

[105]  Marc Teboulle,et al.  Entropic Proximal Mappings with Applications to Nonlinear Programming , 1992, Math. Oper. Res..

[106]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[107]  Jian-Feng Cai,et al.  Linearized Bregman iterations for compressed sensing , 2009, Math. Comput..

[108]  Marc Teboulle,et al.  Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman Functions , 1993, SIAM J. Optim..

[109]  Jonas Adler,et al.  Learned Primal-Dual Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[110]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[111]  Alfred O. Hero,et al.  A Convergent Incremental Gradient Method with a Constant Step Size , 2007, SIAM J. Optim..

[112]  Antonin Chambolle,et al.  An introduction to continuous optimization for imaging , 2016, Acta Numerica.

[113]  Marc Teboulle,et al.  Mirror descent and nonlinear projected subgradient methods for convex optimization , 2003, Oper. Res. Lett..

[114]  Émilie Chouzenoux,et al.  Variable Metric Forward–Backward Algorithm for Minimizing the Sum of a Differentiable Function and a Convex Function , 2013, Journal of Optimization Theory and Applications.

[115]  Martin Burger,et al.  Iterative total variation schemes for nonlinear inverse problems , 2009 .

[116]  Alexander G. Gray,et al.  Stochastic Alternating Direction Method of Multipliers , 2013, ICML.

[117]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[118]  Alexandre M. Bayen,et al.  Accelerated Mirror Descent in Continuous and Discrete Time , 2015, NIPS.

[119]  Alexandre d'Aspremont,et al.  Optimal Complexity and Certification of Bregman First-Order Methods , 2021, Mathematical Programming.

[120]  Jonathan Eckstein,et al.  Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming , 1993, Math. Oper. Res..

[121]  K. Kunisch,et al.  Regularization of linear least squares problems by total bounded variation , 1997 .

[122]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[123]  K. Kiwiel.  Proximal Minimization Methods with Generalized Bregman Functions , 1997 .

[124]  Martin Benning,et al.  Enhancing joint reconstruction and segmentation with non-convex Bregman iteration , 2018, Inverse Problems.

[125]  Dirk A. Lorenz,et al.  Linear convergence of the randomized sparse Kaczmarz method , 2016, Mathematical Programming.

[126]  O. Scherzer,et al.  Error estimates for non-quadratic regularization and the relation to enhancement , 2006 .

[127]  A. Juditsky,et al.  5 First-Order Methods for Nonsmooth Convex Large-Scale Optimization , I : General Purpose Methods , 2010 .

[128]  Marcus A. Magnor,et al.  A sparse Kaczmarz solver and a linearized Bregman method for online compressed sensing , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[129]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[130]  Mohamed-Jalal Fadili,et al.  Wasserstein Control of Mirror Langevin Monte Carlo , 2020, COLT.

[131]  S. Osher,et al.  Nonlinear inverse scale space methods , 2006 .

[132]  Marc Teboulle,et al.  A simplified view of first order methods for optimization , 2018, Math. Program..

[133]  Michael Möller,et al.  Variational Depth From Focus Reconstruction , 2014, IEEE Transactions on Image Processing.

[134]  Dimitri P. Bertsekas,et al.  Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey , 2015, ArXiv.

[135]  Daniel Cremers,et al.  An algorithm for minimizing the Mumford-Shah functional , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[136]  Min Li,et al.  Adaptive Primal-Dual Splitting Methods for Statistical Learning and Image Processing , 2015, NIPS.

[137]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[138]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[139]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[140]  Carola-Bibiane Schönlieb,et al.  Bregman Itoh–Abe Methods for Sparse Optimisation , 2019, Journal of Mathematical Imaging and Vision.

[141]  E. Hellinger,et al.  Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. , 1909 .

[142]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[143]  Barbara Kaltenbacher,et al.  Regularization Methods in Banach Spaces , 2012, Radon Series on Computational and Applied Mathematics.

[144]  Michael Möller,et al.  An adaptive inverse scale space method for compressed sensing , 2012, Math. Comput..