Random projection algorithms for convex set intersection problems

The focus of this paper is on the set intersection problem for closed convex sets admitting projection operation in a closed form. The objective is to investigate algorithms that would converge (in some sense) if and only if the problem has a solution. To do so, we view the set intersection problem as a stochastic optimization problem of minimizing the “average” residual error of the set collection. We consider a stochastic gradient method as a main tool for investigating the properties of the stochastic optimization problem. We show that the stochastic optimization problem has a solution if and only if the stochastic gradient method is convergent almost surely. We then consider a special case of the method, namely the random projection method, and we analyze its convergence. We show that a solution of the intersection problem exists if and only if the random projection method exhibits certain convergence behavior almost surely. In addition, we provide convergence rate results for the expected residual error.

[1]  Frank Deutsch,et al.  The rate of convergence for the cyclic projections algorithm I: Angles between convex sets , 2006, J. Approx. Theory.

[2]  Patrick L. Combettes,et al.  A Convex Programming Algorithm for Noisy Discrete Tomography , 2007 .

[3]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[4]  Angelia Nedic,et al.  Incremental Stochastic Subgradient Algorithms for Convex Optimization , 2008, SIAM J. Optim..

[5]  Boris Polyak,et al.  The method of projections for finding the common point of convex sets , 1967 .

[6]  Zvi Drezner.  Technical Note - The Nested Ball Principle for the Relaxation Method , 1983, Oper. Res..

[7]  Giuseppe Carlo Calafiore,et al.  Stochastic algorithms for exact and approximate feasibility of robust LMIs , 2001, IEEE Trans. Autom. Control..

[8]  K. Kiwiel.  Block-iterative surrogate projection methods for convex feasibility problems , 1995 .

[9]  Jean-Louis Goffin,et al.  The Relaxation Method for Solving Systems of Linear Inequalities , 1980, Math. Oper. Res..

[10]  Patrick L. Combettes,et al.  Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections , 1997, IEEE Trans. Image Process..

[11]  F. Deutsch.  Rate of Convergence of the Method of Alternating Projections , 1984 .

[12]  Heinz H. Bauschke,et al.  Hybrid projection-reflection method for phase retrieval. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[13]  Boris Polyak.  Random Algorithms for Solving Convex Inequalities , 2001 .

[14]  S. Agmon.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[15]  V. Borkar.  Stochastic Approximation: A Dynamical Systems Viewpoint , 2008 .

[16]  M. Ferris,et al.  Weak sharp minima in mathematical programming , 1993 .

[17]  John N. Tsitsiklis,et al.  Gradient Convergence in Gradient methods with Errors , 1999, SIAM J. Optim..

[18]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[19]  Giuseppe Carlo Calafiore,et al.  Random Convex Programs , 2010, SIAM J. Optim..

[20]  I. J. Schoenberg,et al.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[21]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[22]  Heinz H. Bauschke.  Projection Algorithms: Results and Open Problems , 2001 .

[23]  D. Bertsekas.  Gradient convergence in gradient methods , 1997 .

[24]  Eduardo F. Camacho,et al.  Randomized Strategies for Probabilistic Solutions of Uncertain Feasibility and Optimization Problems , 2009, IEEE Transactions on Automatic Control.

[25]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .