A majorize-minimize line search algorithm for barrier function optimization

Many signal and image estimation problems such as maximum entropy reconstruction and positron emission tomography, require the minimization of a criterion containing a barrier function i.e., an unbounded function at the boundary of the feasible solution domain. This function has to be carefully handled in the optimization algorithm. When an iterative descent method is used for the minimization, a search along the line supported by the descent direction is usually performed at each iteration. However, standard line search strategies tend to be inefficient in this context. In this paper, we propose an original line search algorithm based on the majorize-minimize principle. A tangent majorant function is built to approximate a scalar criterion containing a barrier function. This leads to a simple line search ensuring the convergence of several classical descent optimization strategies, including the most classical variants of nonlinear conjugate gradient. The practical efficiency of the proposal scheme is illustrated by means of two examples of signal and image reconstruction.

[1]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[2]  J.H. Manton,et al.  A Globally Convergent Conjugate Gradient Method for Minimizing Self-Concordant Functions with Application to Constrained Optimisation Problems , 2007, 2007 American Control Conference.

[3]  Walter Murray,et al.  A barrier algorithm for large nonlinear optimization problems , 2003 .

[4]  J. M. Ollinger,et al.  Positron Emission Tomography , 2018, Handbook of Small Animal Imaging.

[5]  Zhen-Jun Shi,et al.  Convergence of line search methods for unconstrained optimization , 2004, Appl. Math. Comput..

[6]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[7]  J. Skilling,et al.  Maximum entropy image reconstruction: general algorithm , 1984 .

[8]  Ariela Sofer,et al.  Interior-point methodology for 3-D PET reconstruction , 2000, IEEE Transactions on Medical Imaging.

[9]  Joseph A. O'Sullivan,et al.  Roughness penalties on finite domains , 1995, IEEE Trans. Image Process..

[10]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[11]  Jeffrey A. Fessler,et al.  An Expanded Theoretical Treatment of Iteration-Dependent Majorize-Minimize Algorithms , 2007, IEEE Transactions on Image Processing.

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  J. ChristianLabatand,et al.  CONVERGENCE OF TRUNCATED HALF-QUADRATIC AND NEWTON ALGORITHMS, WITH APPLICATION TO IMAGE RESTORATION. , 2007 .

[14]  H Lantéri,et al.  A general method to devise maximum-likelihood signal restoration multiplicative algorithms with non-negativity constraints , 2001, Signal Process..

[15]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[16]  Jeffrey A. Fessler,et al.  Fast Monotonic Algorithms for Transmission Tomography , 1999, IEEE Trans. Medical Imaging.

[17]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[18]  Walter Murray,et al.  Line Search Procedures for the Logarithmic Barrier Function , 1994, SIAM J. Optim..

[19]  Margaret H. Wright,et al.  Interior methods for constrained optimization , 1992, Acta Numerica.

[20]  Saïd Moussaoui,et al.  A new line search method for barrier functions optimization with strong convergence properties , 2009 .

[21]  J. Idier,et al.  Convergence of Conjugate Gradient Methods with a Closed-Form Stepsize Formula , 2008 .

[22]  Ya-Xiang Yuan,et al.  A three-parameter family of nonlinear conjugate gradient methods , 2001, Math. Comput..

[23]  C. Roos,et al.  Inverse barrier methods for linear programming , 1994 .

[24]  Stephen G. Nash,et al.  A Barrier Method for Large-Scale Constrained Optimization , 1993, INFORMS J. Comput..

[25]  G Demoment,et al.  Maximum entropy image reconstruction in X-ray and diffraction tomography. , 1988, IEEE transactions on medical imaging.

[26]  David J. Thuente,et al.  Line search algorithms with guaranteed sufficient decrease , 1994, TOMS.

[27]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[28]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[29]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.