Bounded perturbation resilience of projected scaled gradient methods

We investigate projected scaled gradient (PSG) methods for convex minimization problems. These methods perform a descent step along a diagonally scaled gradient direction followed by a feasibility regaining step via orthogonal projection onto the constraint set. This constitutes a generalized algorithmic structure that encompasses as special cases the gradient projection method, the projected Newton method, the projected Landweber-type methods and the generalized expectation-maximization (EM)-type methods. We prove the convergence of the PSG methods in the presence of bounded perturbations. This resilience to bounded perturbations is relevant to the ability to apply the recently developed superiorization methodology to PSG methods, in particular to the EM algorithm.

[1]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[2]  Wolfgang Osten,et al.  Introduction to Inverse Problems in Imaging , 1999 .

[3]  R. W. Schulte,et al.  Feasibility-Seeking and Superiorization Algorithms Applied to Inverse Treatment Planning in Radiation Therapy , 2014 .

[4]  Ran Davidi,et al.  Perturbation resilience and superiorization of iterative algorithms , 2010, Inverse problems.

[5]  L. Zanni,et al.  A scaled gradient projection method for constrained image deblurring , 2008 .

[6]  Gabor T Herman,et al.  Data fusion in X-ray computed tomography using a superiorization approach. , 2014, The Review of scientific instruments.

[7]  Yair Censor,et al.  Proximity Function Minimization Using Multiple Bregman Projections, with Applications to Split Feasibility and Kullback–Leibler Distance Minimization , 2001, Ann. Oper. Res..

[8]  Alvaro R. De Pierro,et al.  Convergence results for scaled gradient algorithms in positron emission tomography , 2005 .

[9]  Olvi L. Mangasarian.  Convergence of Iterates of an Inexact Matrix Splitting Algorithm for the Symmetric Monotone Linear Complementarity Problem , 1991, SIAM J. Optim..

[10]  Charles L. Byrne,et al.  Applied Iterative Methods , 2007 .

[11]  Ran Davidi,et al.  Perturbation-resilient block-iterative projection methods with application to image reconstruction from projections , 2009, Int. Trans. Oper. Res..

[12]  P. Tseng,et al.  On the linear convergence of descent methods for convex essentially smooth minimization , 1992 .

[13]  Gabor T. Herman,et al.  Superiorization of the ML-EM Algorithm , 2014, IEEE Transactions on Nuclear Science.

[14]  Ming Jiang,et al.  Convergence Studies on Iterative Algorithms for Image Reconstruction , 2003, IEEE Trans. Medical Imaging.

[15]  M. Solodov,et al.  Error Stability Properties of Generalized Gradient-Type Algorithms , 1998 .

[16]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[17]  Yair Censor,et al.  On Diagonally Relaxed Orthogonal Projection Methods , 2007, SIAM J. Sci. Comput..

[18]  Jong-Shi Pang,et al.  Error bounds in mathematical programming , 1997, Math. Program..

[19]  Z.-Q. Luo,et al.  Error bounds and convergence analysis of feasible descent methods: a general approach , 1993, Ann. Oper. Res..

[20]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Parallel Distributed Comput. Pract..

[21]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[22]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[23]  G. Rodrigue,et al.  A uniform approach to gradient methods for linear operator equations , 1975 .

[24]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[25]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[26]  Ming Jiang,et al.  A Heuristic Superiorization-Like Approach to Bioluminescence Tomography , 2013 .

[27]  Shousheng Luo,et al.  Superiorization of EM Algorithm and Its Application in Single-Photon Emission Computed Tomography(SPECT) , 2012, 1209.6116.

[28]  Y. C. Cheng.  On the gradient-projection method for solving the nonsymmetric linear complementarity problem , 1984 .

[29]  A. Cegielski.  Iterative Methods for Fixed Point Problems in Hilbert Spaces , 2012 .

[30]  M. Bertero,et al.  Iterative image reconstruction : a point of view , 2007 .

[31]  L. Landweber.  An iteration formula for Fredholm integral equations of the first kind , 1951 .

[32]  M. Solodov.  Convergence Analysis of Perturbed Feasible Descent Methods , 1997 .

[33]  Ran Davidi,et al.  Superiorization: An optimization heuristic for medical physics , 2012, Medical physics.

[34]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[35]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[36]  G. Herman,et al.  Accelerated perturbation-resilient block-iterative projection methods with application to image reconstruction , 2012, Inverse problems.

[37]  Gabor T. Herman,et al.  Superiorization for Image Analysis , 2014, IWCIA.

[38]  Ming Jiang,et al.  Mathematical methods in biomedical imaging and intensity-modulated radiation therapy (IMRT) , 2008 .

[39]  P. L. Combettes,et al.  Quasi-Fejérian Analysis of Some Optimization Algorithms , 2001 .

[40]  L. Shepp,et al.  Maximum Likelihood Reconstruction for Emission Tomography , 1983, IEEE Transactions on Medical Imaging.

[41]  Gene H. Golub,et al.  Matrix computations , 1983 .

[42]  M. Bertero,et al.  Projected Landweber method and preconditioning , 1997 .

[43]  D. Bertsekas,et al.  TWO-METRIC PROJECTION METHODS FOR CONSTRAINED OPTIMIZATION* , 1984 .

[44]  H. Trussell,et al.  The Landweber iteration and projection onto convex sets , 1985, IEEE Trans. Acoust. Speech Signal Process..

[45]  Wu Li.  Remarks on Convergence of the Matrix Splitting Algorithm for the Symmetric Linear Complementarity Problem , 1993, SIAM J. Optim..

[46]  Alfredo N. Iusem.  Convergence analysis for a multiplicatively relaxed EM algorithm , 1991 .

[47]  A. Goldstein.  Convex programming in Hilbert space , 1964 .

[48]  Yair Censor,et al.  Convergence and perturbation resilience of dynamic string-averaging projection methods , 2012, Computational Optimization and Applications.

[49]  Gabor T. Herman,et al.  Fundamentals of Computerized Tomography: Image Reconstruction from Projections , 2009, Advances in Pattern Recognition.

[50]  Gabor T. Herman,et al.  Image reconstruction from projections : the fundamentals of computerized tomography , 1980 .

[51]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, CDC 1981.

[52]  Ming Jiang,et al.  Development of iterative algorithms for image reconstruction , 2002 .

[53]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[54]  Krzysztof C. Kiwiel,et al.  Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization , 2003, SIAM J. Optim..

[55]  John W. Chinneck,et al.  Feasibility and Infeasibility in Optimization:: Algorithms and Computational Methods , 2007 .

[56]  Jong-Shi Pang,et al.  A Posteriori Error Bounds for the Linearly-Constrained Variational Inequality Problem , 1987, Math. Oper. Res..

[57]  Alvaro R. De Pierro,et al.  Incremental Subgradients for Constrained Convex Optimization: A Unified Framework and New Methods , 2009, SIAM J. Optim..

[58]  H Lantéri,et al.  A general method to devise maximum-likelihood signal restoration multiplicative algorithms with non-negativity constraints , 2001, Signal Process..

[59]  A B Rosenfeld,et al.  Total variation superiorization schemes in proton computed tomography image reconstruction. , 2010, Medical physics.

[60]  D. Butnariu,et al.  Stable Convergence Behavior Under Summable Perturbations of a Class of Projection Methods for Convex Feasibility and Optimization Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[61]  Charles L. Byrne,et al.  Iterative image reconstruction algorithms based on cross-entropy minimization , 1992, Optics & Photonics.

[62]  Ran Davidi,et al.  Projected Subgradient Minimization Versus Superiorization , 2013, Journal of Optimization Theory and Applications.

[63]  Yair Censor,et al.  Weak and Strong Superiorization: Between Feasibility-Seeking and Minimization , 2014, 1410.0130.

[64]  Patrick L. Combettes,et al.  Inconsistent signal feasibility problems: least-squares solutions in a product space , 1994, IEEE Trans. Signal Process..