Randomized projection methods for convex feasibility problems: conditioning and convergence rates

Finding a point in the intersection of a collection of closed convex sets, that is the convex feasibility problem, represents the main modeling strategy for many computational problems. In this paper we analyze new stochastic reformulations of the convex feasibility problem in order to facilitate the development of new algorithmic schemes. We also analyze the conditioning problem parameters using certain (linear) regularity assumptions on the individual convex sets. Then, we introduce a general random projection algorithmic framework, which extends to the random settings many existing projection schemes, designed for the general convex feasibility problem. Our general random projection algorithm allows to project simultaneously on several sets, thus providing great flexibility in matching the implementation of the algorithm on the parallel architecture at hand. Based on the conditioning parameters, besides the asymptotic convergence results, we also derive explicit sublinear and linear convergence rates for this general algorithmic framework.

[1]  P. L. Combettes,et al.  The Convex Feasibility Problem in Image Recovery , 1996 .

[2]  Yongyi Yang,et al.  Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics , 1998 .

[3]  Yair Censor,et al.  Averaging Strings of Sequential Iterations for Convex Feasibility Problems , 2001 .

[4]  R. Vershynin,et al.  A Randomized Kaczmarz Algorithm with Exponential Convergence , 2007, math/0702226.

[5]  Dennis L. Parker,et al.  POCS-BASED RECONSTRUCTION FOR SENSITIVITY ENCODED MRI , 2004 .

[6]  Heinz H. Bauschke,et al.  On cluster points of alternating projections , 2013, 1307.2712.

[7]  Boris Polyak,et al.  The method of projections for finding the common point of convex sets , 1967 .

[8]  Gaurav Sharma.  Set theoretic estimation for problems in subtractive color , 2000 .

[9]  H. Stark,et al.  Wide-band smart antenna design using vector space projection methods , 2004, IEEE Transactions on Antennas and Propagation.

[10]  Gabor T. Herman,et al.  Fundamentals of Computerized Tomography: Image Reconstruction from Projections , 2009, Advances in Pattern Recognition.

[11]  Wei Chen,et al.  A fast algorithm for solving a linear feasibility problem with application to Intensity-Modulated Radiation Therapy. , 2008, Linear algebra and its applications.

[12]  Ion Necoara,et al.  Nonasymptotic convergence of stochastic proximal point methods for constrained convex optimization , 2017, J. Mach. Learn. Res..

[13]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[14]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[15]  Marc Teboulle,et al.  A Log-Quadratic Projection Method for Convex Feasibility Problems , 2001 .

[16]  Chris R. Johnson,et al.  POCSENSE: POCS‐based reconstruction for sensitivity encoded magnetic resonance imaging , 2004, Magnetic resonance in medicine.

[17]  Gene H. Golub,et al.  Matrix computations , 1983 .

[18]  Peter Richtárik,et al.  Randomized Iterative Methods for Linear Systems , 2015, SIAM J. Matrix Anal. Appl..

[19]  Frank Deutsch,et al.  The rate of convergence for the cyclic projections algorithm I: Angles between convex sets , 2006, J. Approx. Theory.

[20]  Hong Yan,et al.  POCS-based blocking artifacts suppression using a smoothness constraint set with explicit region modeling , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Richard G. Baraniuk,et al.  Multiple wavelet basis image denoising using Besov ball projections , 2004, IEEE Signal Processing Letters.

[22]  Patrick L. Combettes,et al.  On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints , 2009, Computational Optimization and Applications.

[23]  Adrian S. Lewis,et al.  Randomized Methods for Linear Constraints: Convergence Rates and Conditioning , 2008, Math. Oper. Res..

[24]  Angelia Nedic,et al.  Random algorithms for convex minimization problems , 2011, Math. Program..

[25]  Yair Censor,et al.  Proximity Function Minimization Using Multiple Bregman Projections, with Applications to Split Feasibility and Kullback–Leibler Distance Minimization , 2001, Ann. Oper. Res..

[26]  P. L. Combettes,et al.  Hilbertian convex feasibility problem: Convergence of projection methods , 1997 .

[27]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[28]  S. Agmon.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[29]  I. Necoara,et al.  Nonasymptotic convergence of stochastic proximal point algorithms for constrained convex optimization , 2017, 1706.06297.

[30]  Marc Teboulle,et al.  Convergence rate analysis and error bounds for projection algorithms in convex feasibility problems , 2003, Optim. Methods Softw..

[31]  Alfred O. Hero,et al.  Energy-based sensor network source localization via projection onto convex sets , 2005, IEEE Transactions on Signal Processing.

[32]  Martin Vetterli,et al.  Demosaicking by Alternating Projections: Theory and Fast One-Step Implementation , 2010, IEEE Transactions on Image Processing.

[33]  Marc Teboulle,et al.  A conditional gradient method with linear rate of convergence for solving convex linear systems , 2004, Math. Methods Oper. Res..

[34]  M. Ferris,et al.  Weak sharp minima in mathematical programming , 1993 .

[35]  Angelia Nedic,et al.  Random projection algorithms for convex set intersection problems , 2010, 49th IEEE Conference on Decision and Control (CDC).

[36]  I. Yamada,et al.  Pairwise Optimal Weight Realization—Acceleration Technique for Set-Theoretic Adaptive Parallel Subgradient Projection Algorithm , 2006, IEEE Transactions on Signal Processing.

[37]  Marc Teboulle,et al.  A Linearly Convergent Algorithm for Solving a Class of Nonconvex/Affine Feasibility Problems , 2011, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.