Two-Subspace Projection Method for Coherent Overdetermined Systems

We present a Projection onto Convex Sets (POCS) type algorithm for solving systems of linear equations. POCS methods have found many applications ranging from computer tomography to digital signal and image processing. The Kaczmarz method is one of the most popular solvers for overdetermined systems of linear equations due to its speed and simplicity. Here we introduce and analyze an extension of the Kaczmarz method that iteratively projects the estimate onto a solution space given by two randomly selected rows. We show that this projection algorithm provides exponential convergence to the solution in expectation. The convergence rate improves upon that of the standard randomized Kaczmarz method when the system has correlated rows. Experimental results confirm that in this case our method significantly outperforms the randomized Kaczmarz method.

[1]  Y. Censor,et al.  Strong underrelaxation in Kaczmarz's method for inconsistent systems , 1983 .

[2]  F. Natterer.  The Mathematics of Computerized Tomography , 1986 .

[3]  M. Hanke,et al.  On the acceleration of Kaczmarz's method for inconsistent linear systems , 1990 .

[4]  Hans G. Feichtinger,et al.  New variants of the POCS method using affine subspaces of finite codimension with applications to irregular sampling , 1992, Other Conferences.

[5]  Gabor T. Herman,et al.  Algebraic reconstruction techniques can be made computationally efficient [positron emission tomography application] , 1993, IEEE Trans. Medical Imaging.

[6]  Hein Hundal,et al.  The Rate of Convergence for the Method of Alternating Projections, II , 1997 .

[7]  A. Galántai.  On the rate of convergence of the alternating projection method in finite dimensional spaces , 2005 .

[8]  Thomas Strohmer,et al.  A Randomized Solver for Linear Systems with Exponential Convergence , 2006, APPROX-RANDOM.

[9]  R. Vershynin,et al.  A Randomized Kaczmarz Algorithm with Exponential Convergence , 2007, math/0702226.

[10]  D. Needell.  Randomized Kaczmarz solver for noisy linear systems , 2009, 0902.0958.

[11]  Adrian S. Lewis,et al.  Randomized Methods for Linear Constraints: Convergence Rates and Conditioning , 2008, Math. Oper. Res..

[12]  C. Badea,et al.  The rate of convergence in the method of alternating projections , 2010, 1006.2047.

[13]  Per Christian Hansen,et al.  Semi-convergence and relaxation parameters for a class of SIRT algorithms , 2010 .

[14]  S. Muthukrishnan,et al.  Faster least squares approximation , 2007, Numerische Mathematik.

[15]  Deanna Needell,et al.  Paved with Good Intentions: Analysis of a Randomized Block Kaczmarz Method , 2012, ArXiv.

[16]  Christopher Ré,et al.  Toward a Noncommutative Arithmetic-geometric Mean Inequality: Conjectures, Case-studies, and Consequences , 2012, COLT.

[17]  Xuemei Chen,et al.  Almost Sure Convergence of the Kaczmarz Algorithm with Random Measurements , 2012 .

[18]  B. Recht,et al.  Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences , 2012, 1202.4184.