The Supporting Halfspace-Quadratic Programming Strategy for the Dual of the Best Approximation Problem

We consider the best approximation problem (BAP) of projecting a point onto the intersection of a number of convex sets. It is known that Dykstra's algorithm is alternating minimization on the dual problem. We extend Dykstra's algorithm so that it can be enhanced by the SHQP strategy of using quadratic programming to project onto the intersection of supporting halfspaces generated by earlier projection operations. By looking at a structured alternating minimization problem, we show the convergence rate of Dykstra's algorithm when reasonable conditions are imposed to guarantee a dual minimizer. We also establish convergence of using a warmstart iterate for Dykstra's algorithm, show how all the results for the Dykstra's algorithm can be carried over to the simultaneous Dykstra's algorithm, and discuss a different way of incorporating the SHQP strategy. Lastly, we show that the dual of the best approximation problem can have an O(1/k^2) accelerated algorithm that also incorporates the SHQP strategy.

[1]  Marc Teboulle,et al.  Interior Gradient and Proximal Methods for Convex and Conic Optimization , 2006, SIAM J. Optim..

[2]  Luigi Grippof,et al.  Globally convergent block-coordinate techniques for unconstrained optimization , 1999 .

[3]  A. Chambolle,et al.  A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions , 2015 .

[4]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[5]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[6]  Peter Richtárik,et al.  Accelerated, Parallel, and Proximal Coordinate Descent , 2013, SIAM J. Optim..

[7]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[8]  A. Juditsky,et al.  5 First-Order Methods for Nonsmooth Convex Large-Scale Optimization , I : General Purpose Methods , 2010 .

[9]  Bruce L. Golden,et al.  Optimisation , 1982, IEEE Trans. Syst. Man Cybern..

[10]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[11]  W. B. Bearhart,et al.  Acceleration schemes for the method of alternating projections , 1989 .

[12]  Alvaro R. De Pierro,et al.  On the convergence of Han's method for convex programming with quadratic objective , 1991, Math. Program..

[13]  Guy Pierra,et al.  Decomposition through formalization in a product space , 1984, Math. Program..

[14]  Guanghui Lan,et al.  iteration-complexity for cone programming , 2008 .

[15]  A. Kruger.  About Regularity of Collections of Sets , 2006 .

[16]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[17]  R. Dykstra,et al.  A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces , 1986 .

[18]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[19]  Heinz H. Bauschke,et al.  Extrapolation algorithm for affine-convex feasibility problems , 2006, Numerical Algorithms.

[20]  C. Pang.  Accelerating the alternating projection algorithm for the case of affine subspaces using supporting hyperplanes , 2014, 1406.4012.

[21]  Defeng Sun,et al.  A Quadratically Convergent Newton Method for Computing the Nearest Correlation Matrix , 2006, SIAM J. Matrix Anal. Appl..

[22]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[23]  Hein Hundal,et al.  The rate of convergence of dykstra's cyclic projections algorithm: The polyhedral case , 1994 .

[24]  Z.-Q. Luo,et al.  Error bounds and convergence analysis of feasible descent methods: a general approach , 1993, Ann. Oper. Res..

[25]  Shih-Ping Han,et al.  A successive projection method , 1988, Math. Program..

[26]  M. Raydan,et al.  Alternating Projection Methods , 2011 .

[27]  Marcos Raydan,et al.  An acceleration scheme for Dykstra’s algorithm , 2016, Comput. Optim. Appl..

[28]  F. Deutsch.  Accelerating the Convergence of the Method of Alternating Projections Via a Line Search: a Brief Survey , 2001 .

[29]  Jérôme Malick,et al.  A Dual Approach to Semidefinite Least-Squares Problems , 2004, SIAM J. Matrix Anal. Appl..

[30]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[31]  Nicholas J. Higham,et al.  Anderson acceleration of the alternating projections method for computing the nearest correlation matrix , 2016, Numerical Algorithms.

[32]  Boris Polyak,et al.  The method of projections for finding the common point of convex sets , 1967 .

[33]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[34]  Amir Beck,et al.  On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes , 2015, SIAM J. Optim..

[35]  R. Mathar,et al.  A cyclic projection algorithm via duality , 1989 .

[36]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[37]  Heinz H. Bauschke,et al.  Accelerating the convergence of the method of alternating projections , 2003 .

[38]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..