Almost Sure Convergence of the Kaczmarz Algorithm with Random Measurements

The Kaczmarz algorithm is an iterative method for reconstructing a signal x∈ℝd from an overcomplete collection of linear measurements yn=〈x,φn〉, n≥1. We prove quantitative bounds on the rate of almost sure exponential convergence in the Kaczmarz algorithm for suitable classes of random measurement vectors $\{\varphi_{n}\}_{n=1}^{\infty} \subset {\mathbb {R}}^{d}$. Refined convergence results are given for the special case when each φn has i.i.d. Gaussian entries and, more generally, when each φn/∥φn∥ is uniformly distributed on $\mathbb{S}^{d-1}$. This work on almost sure convergence complements the mean squared error analysis of Strohmer and Vershynin for randomized versions of the Kaczmarz algorithm.

[1]  W. Rudin.  Real and complex analysis , 1968 .

[2]  J. Milnor.  Hyperbolic geometry: The first 150 years , 1982 .

[3]  F. Natterer.  The Mathematics of Computerized Tomography , 1986 .

[4]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[5]  Henry Stark,et al.  Image recovery: Theory and application , 1987 .

[6]  S. Kaczmarz.  Approximate solution of systems of linear equations , 1993 .

[7]  E. Saff,et al.  Logarithmic Potentials with External Fields , 1997 .

[8]  On the Geometrical Convergence of Gibbs Sampler inRd , 1998 .

[9]  On the Kaczmarz algorithm of approximation in infinite dimensional spaces , 2001 .

[10]  Sundeep Rangan,et al.  Recursive consistent estimation with bounded noise , 2001, IEEE Trans. Inf. Theory.

[11]  Kaczmarz algorithm in Hilbert space and tight frames , 2005, math/0509236.

[12]  Ryszard Szwarc,et al.  Kaczmarz algorithm in Hilbert space , 2005 .

[13]  Thomas Strohmer,et al.  A Randomized Solver for Linear Systems with Exponential Convergence , 2006, APPROX-RANDOM.

[14]  Suvrit Sra,et al.  Row-Action Methods for Compressed Sensing , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[15]  Mihalis Yannakakis,et al.  Small Approximate Pareto Sets for Bi-objective Shortest Paths and Other Problems , 2007, APPROX-RANDOM.

[16]  R. Vershynin,et al.  A Randomized Kaczmarz Algorithm with Exponential Convergence , 2007, math/0702226.

[17]  D. Needell.  Randomized Kaczmarz solver for noisy linear systems , 2009, 0902.0958.

[18]  R. Vershynin,et al.  Comments on the Randomized Kaczmarz Method , 2009 .

[19]  Y. Censor,et al.  A Note on the Behavior of the Randomized Kaczmarz Algorithm of Strohmer and Vershynin , 2009, The journal of fourier analysis and applications.

[20]  Adrian S. Lewis,et al.  Randomized Methods for Linear Constraints: Convergence Rates and Conditioning , 2008, Math. Oper. Res..

[21]  Alexander M. Powell,et al.  Mean squared error bounds for the Rangan–Goyal soft thresholding algorithm , 2010 .

[22]  Yonina C. Eldar,et al.  Acceleration of randomized Kaczmarz method via the Johnson–Lindenstrauss Lemma , 2010, Numerical Algorithms.

[23]  Peter G. Casazza,et al.  Finite Frames: Theory and Applications , 2012 .

[24]  M. Ehler,et al.  Probabilistic Frames: An Overview , 2011, 1108.2169.