Nonconvex set intersection problems: From projection methods to the Newton method for super-regular sets

The problem of finding a point in the intersection of closed sets can be solved by the method of alternating projections and its variants. It was shown in earlier papers that for convex sets, the strategy of using quadratic programming (QP) to project onto the intersection of supporting halfspaces generated earlier by the projection process can lead to an algorithm that converges multiple-term superlinearly. The main contributions of this paper are to show that this strategy can be effective for super-regular sets, which are structured nonconvex sets introduced by Lewis, Luke and Malick. Manifolds should be approximated by hyperplanes rather than halfspaces. We prove the linear convergence of this strategy, followed by proving that superlinear and quadratic convergence can be obtained when the problem is similar to the setting of the Newton method. We also show an algorithm that converges at an arbitrarily fast linear rate if halfspaces from older iterations are used to construct the QP.

[1]  Ubaldo M. García-Palomares.  A superlinearly convergent projection algorithm for solving the convex inequality problem , 1998, Oper. Res. Lett..

[2]  Karin Schwab,et al.  Best Approximation In Inner Product Spaces , 2016 .

[3]  Donald Goldfarb,et al.  Efficient primal algorithms for strictly convex quadratic programs , 1986 .

[4]  Heinz H. Bauschke,et al.  On the convergence of von Neumann's alternating projection algorithm for two sets , 1993 .

[5]  Karolos M. Grigoriadis,et al.  Low-order control design for LMI problems using alternating projection methods , 1996, Autom..

[6]  Ubaldo M. García-Palomares,et al.  Superlinear Rate of Convergence and Optimal Acceleration Schemes in the Solution of Convex Inequality Problems , 2001 .

[7]  Moody T. Chu,et al.  Constructing a Hermitian Matrix from Its Diagonal Entries and Eigenvalues , 1995, SIAM J. Matrix Anal. Appl..

[8]  Robert W. Heath,et al.  Designing structured tight frames via an alternating projection method , 2005, IEEE Transactions on Information Theory.

[9]  S. Marchesini,et al.  Alternating projection, ptychographic imaging and phase synchronization , 2014, 1402.0550.

[10]  J. Borwein,et al.  Techniques of variational analysis , 2005 .

[11]  Jean Charles Gilbert,et al.  Numerical Optimization: Theoretical and Practical Aspects , 2003 .

[12]  Boris Polyak,et al.  The method of projections for finding the common point of convex sets , 1967 .

[13]  C. Pang.  Finitely convergent algorithm for nonconvex inequality problems , 2014, 1405.7280.

[14]  A. Ioffe.  Metric regularity and subdifferential calculus , 2000 .

[15]  M. Raydan,et al.  Alternating Projection Methods , 2011 .

[16]  A. Pierro,et al.  A finitely convergent “row-action” method for the convex feasibility problem , 1988 .

[17]  Wei Hong Yang,et al.  Regularities and their relations to error bounds , 2004, Math. Program..

[18]  M. Chu,et al.  On the Least Squares Solution of Inverse Eigenvalue Problems , 1996 .

[19]  Robert Orsi.  Numerical Methods for Solving Inverse Eigenvalue Problems for Nonnegative Matrices , 2006, SIAM J. Matrix Anal. Appl..

[20]  D. Mayne,et al.  On the finite solution of nonlinear inequalities , 1979 .

[21]  øöö Blockinø.  Phase retrieval, error reduction algorithm, and Fienup variants: A view from convex optimization , 2002 .

[22]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[23]  Marcos Raydan,et al.  Unconstrained Optimization Techniques for the Acceleration of Alternating Projection Methods , 2011 .

[24]  E. Polak.  Method of Successive Projections for Finding a Common Point of Sets in Metric Spaces , 1990 .

[25]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[26]  John B. Moore,et al.  A Newton-like method for solving rank constrained linear matrix inequalities , 2006, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[27]  Heinz H. Bauschke,et al.  Accelerating the convergence of the method of alternating projections , 2003 .

[28]  Adrian S. Lewis,et al.  Alternating Projections on Manifolds , 2008, Math. Oper. Res..

[29]  Guy Pierra,et al.  Decomposition through formalization in a product space , 1984, Math. Program..

[30]  Heinz H. Bauschke,et al.  On the Finite Convergence of a Projected Cutter Method , 2014, J. Optim. Theory Appl..

[31]  David Q. Mayne,et al.  Solving nonlinear inequalities in a finite number of iterations , 1981 .

[32]  N. Maratos,et al.  Exact penalty function algorithms for finite dimensional and control optimization problems , 1978 .

[33]  A. Kruger.  About Regularity of Collections of Sets , 2006 .

[34]  Adrian S. Lewis,et al.  Local Linear Convergence for Alternating and Averaged Nonconvex Projections , 2009, Found. Comput. Math..

[35]  F. Deutsch.  Best approximation in inner product spaces , 2001 .

[36]  Heinz H. Bauschke,et al.  Strong conical hull intersection property, bounded linear regularity, Jameson’s property (G), and error bounds in convex optimization , 1999, Math. Program..

[37]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[38]  W. B. Bearhart,et al.  Acceleration schemes for the method of alternating projections , 1989 .

[39]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[40]  D. Russell Luke,et al.  Nonconvex Notions of Regularity and Convergence of Fundamental Algorithms for Feasibility Problems , 2012, SIAM J. Optim..

[41]  K. Grigoriadis,et al.  Alternating projection algorithms for linear matrix inequalities problems with rank constraints , 1999 .

[42]  Robert Orsi,et al.  Generalized pole placement via static output feedback: A methodology based on projections , 2006, Autom..

[43]  Masao Fukushima,et al.  A finitely convergent algorithm for convex inequalities , 1982 .