PIPA: A New Proximal Interior Point Algorithm for Large-Scale Convex Optimization

Interior point methods have been known for decades to be useful for the resolution of small to medium size constrained optimization problems. These approaches have the benefit of ensuring feasibility of the iterates through a logarithmic barrier. We propose to incorporate a proximal forward-backward step in the resolution of the barrier subproblem to account for non-necessarily differentiable terms arising in the objective function. The combination of this scheme with a novel line-search strategy gives rise to the so-called Proximal Interior Point Algorithm (PIPA) suitable for the minimization of the sum of a smooth convex function and a non-smooth convex one under general convex constraints. The convergence of PIPA is secured under mild assumptions. As demonstrated by numerical experiments carried out on a large-scale hyperspectral image unmixing application, the proposed method outperforms the state-of-the-art.

[1]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[2]  Jacek Gondzio,et al.  Performance of first- and second-order methods for ℓ1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{doc , 2015, Computational Optimization and Applications.

[3]  Paul Armand,et al.  A Feasible BFGS Interior Point Algorithm for Solving Convex Minimization Problems , 2000, SIAM J. Optim..

[4]  S. Moussaoui,et al.  Majorize–minimize linesearch for inversion methods involving barrier function optimization , 2012 .

[5]  Mila Nikolova,et al.  Efficient Minimization Methods of Mixed l2-l1 and l1-l1 Norms for Image Restoration , 2005, SIAM J. Sci. Comput..

[6]  Marc Teboulle,et al.  Entropy-Like Proximal Methods in Convex Programming , 1994, Math. Oper. Res..

[7]  Thomas Serafini,et al.  Non-negatively constrained image deblurring with an inexact interior point method , 2009, J. Comput. Appl. Math..

[8]  Margaret H. Wright,et al.  Interior methods for constrained optimization , 1992, Acta Numerica.

[9]  Émilie Chouzenoux,et al.  Dual Block-Coordinate Forward-Backward Algorithm with Application to Deconvolution and Deinterlacing of Video Sequences , 2017, Journal of Mathematical Imaging and Vision.

[10]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[12]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[13]  P. L. Combettes,et al.  Proximity for sums of composite functions , 2010, 1007.3535.

[14]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Jacek Gondzio,et al.  Interior point methods 25 years later , 2012, Eur. J. Oper. Res..

[16]  Patrick L. Combettes,et al.  A forward-backward view of some primal-dual optimization methods in image recovery , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[17]  Alfonso Fernández-Manso,et al.  Spectral unmixing , 2012 .

[18]  Nicholas I. M. Gould,et al.  Superlinear Convergence of Primal-Dual Interior Point Algorithms for Nonlinear Programming , 2000, SIAM J. Optim..

[19]  Émilie Chouzenoux,et al.  Fast Constrained Least Squares Spectral Unmixing Using Primal-Dual Interior-Point Optimization , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  José Yunier Bello Cruz,et al.  On the convergence of the forward–backward splitting method with linesearches , 2015, Optim. Methods Softw..

[21]  Ariela Sofer,et al.  Interior-point methodology for 3-D PET reconstruction , 2000, IEEE Transactions on Medical Imaging.

[22]  Émilie Chouzenoux,et al.  Variable Metric Forward–Backward Algorithm for Minimizing the Sum of a Differentiable Function and a Convex Function , 2013, Journal of Optimization Theory and Applications.

[23]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[24]  Saverio Salzo,et al.  The Variable Metric Forward-Backward Splitting Algorithm Under Mild Differentiability Assumptions , 2016, SIAM J. Optim..

[25]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[26]  I. M. Otivation Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems , 2018 .

[27]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..