Comparison of several fast algorithms for projection onto an ellipsoid

Abstract Projecting a point onto an ellipsoid is one of the fundamental problems in convex analysis and numerical algorithms. Recently, several fast algorithms were proposed for solving this problem such as Lin–Han algorithm, maximum 2-dimensional inside ball algorithm, sequential 2-dimensional projection algorithm and hybrid projection algorithms of Dai. In this paper, we rewrite the problem as a constrained convex optimization problem with separable objective functions, which enables the use of the alternating direction method of multipliers (ADMM). Furthermore, since the efficiency of ADMM depends on the penalty parameter, we choose it in a self-adaptive manner, resulting in the self-adaptive ADMM (S-ADMM). All these methods converge with a global linear rate. We compare them theoretically and numerically and find that S-ADMM is the most efficient one. We also illustrate the flexibility of ADMM by applying it to the more general problem of projecting a point onto the intersection of several ellipsoids, Dantzig selector, and image restoration and reconstruction.

[1]  Anhua Lin A Class of Methods for Projection on a Convex Set , 2003 .

[2]  Xiangfeng Wang,et al.  The Linearized Alternating Direction Method of Multipliers for Dantzig Selector , 2012, SIAM J. Sci. Comput..

[3]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[4]  Wei Hong Yang,et al.  Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems , 2016, SIAM J. Numer. Anal..

[5]  Daniel Boley,et al.  Local Linear Convergence of the Alternating Direction Method of Multipliers on Quadratic or Linear Programs , 2013, SIAM J. Optim..

[6]  Roger Fletcher,et al.  New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds , 2006, Math. Program..

[7]  Defeng Sun,et al.  Linear Rate Convergence of the Alternating Direction Method of Multipliers for Convex Composite Quadratic and Semi-Definite Programming , 2015, 1508.02134.

[8]  Defeng Sun,et al.  Linear Rate Convergence of the Alternating Direction Method of Multipliers for Convex Composite Programming , 2017, Math. Oper. Res..

[9]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[10]  Xiaoming Yuan,et al.  Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs , 2013, SIAM J. Numer. Anal..

[11]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[12]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[13]  Daniel Boley,et al.  Local Linear Convergence of ADMM on Quadratic or Linear Programs , 2012 .

[14]  Panos M. Pardalos,et al.  An algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds , 1990, Math. Program..

[15]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[16]  B. He,et al.  Alternating Direction Method with Self-Adaptive Penalty Parameters for Monotone Variational Inequalities , 2000 .

[17]  Zhiwei Wang,et al.  A simple self-adaptive alternating direction method for linear variational inequality problems , 2007, Comput. Math. Appl..

[18]  Anhua Lin,et al.  A Class of Methods for Projection on the Intersection of Several Ellipsoids , 2004, SIAM J. Optim..

[19]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[20]  Yong Zhang,et al.  An alternating direction method for finding Dantzig selectors , 2010, Comput. Stat. Data Anal..

[21]  Yu-Hong Dai,et al.  Fast Algorithms for Projection on an Ellipsoid , 2006, SIAM J. Optim..

[22]  Roger Fletcher,et al.  On the asymptotic behaviour of some new gradient methods , 2005, Math. Program..