Stabilization of Stochastic Iterative Methods for Singular and Nearly Singular Linear Systems

We consider linear systems of equations, Ax = b, with an emphasis on the case where A is singular. Under certain conditions, necessary as well as sufficient, linear deterministic iterative methods generate sequences {xk} that converge to a solution as long as there exists at least one solution. This convergence property can be impaired when these methods are implemented with stochastic simulation, as is often done in important classes of large-scale problems. We introduce additional conditions and novel algorithmic stabilization schemes under which {xk} converges to a solution when A is singular and may also be used with substantial benefit when A is nearly singular.

[1]  R. A. Leibler,et al.  Matrix inversion by a Monte Carlo method , 1950 .

[2]  W. Wasow.  A note on the inversion of matrices by random walks , 1952 .

[3]  Richard Bellman,et al.  Introduction to Matrix Analysis , 1972 .

[4]  H. Heinrich.  R. Bellman, Introduction to Matrix Analysis. XX + 328 S. London 1960. McGraw-Hill. Preis geb. 77s. 6d , 1961 .

[5]  H. Keller.  On the Solution of Singular and Semidefinite Linear Systems by Iteration , 1965 .

[6]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[7]  B. Martinet,et al.  R'egularisation d''in'equations variationnelles par approximations successives , 1970 .

[8]  M. Sibony.  Méthodes itératives pour les équations et inéquations aux dérivées partielles non linéaires de type monotone , 1970 .

[9]  J. Halton.  A Retrospective and Prospective Survey of the Monte Carlo Method , 1970 .

[10]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[11]  D. Young.  On the Consistency of Linear Stationary Iterative Methods , 1972 .

[12]  M. A. Krasnoselʹskii,et al.  Approximate Solution of Operator Equations , 1972 .

[13]  G. Stewart.  Introduction to matrix computations , 1973 .

[14]  K. Tanabe.  Characterization of linear stationary iterative processes for solving a singular system of linear equations , 1974 .

[15]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[16]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[17]  C. D. Meyer,et al.  Generalized inverses of linear transformations , 1979 .

[18]  D. Bertsekas,et al.  Projection methods for variational inequalities with application to the traffic assignment problem , 1982 .

[19]  F. R. Gantmakher.  The Theory of Matrices , 1984 .

[20]  I. Marek,et al.  On the solution of singular linear systems of algebraic equations by semiiterative methods , 1988 .

[21]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[22]  Achiya Dax.  The Convergence of Linear Stationary Iterative Processes for Solving Singular Unstructured Systems of Linear Equations , 1990, SIAM Rev..

[23]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[24]  J. L. Goldberg.  Matrix theory with applications , 1991 .

[25]  P. Tseng,et al.  On the convergence of a matrix splitting algorithm for the symmetric monotone linear complementarity problem , 1991 .

[26]  Richard W. Cottle,et al.  Linear Complementarity Problem. , 1992 .

[27]  R. D. Murphy,et al.  Iterative solution of nonlinear equations , 1994 .

[28]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[29]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[30]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[31]  Guanrong Chen,et al.  Approximate Solutions of Operator Equations , 1997 .

[32]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[33]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[34]  Dimitri P. Bertsekas,et al.  Least Squares Policy Evaluation Algorithms with Linear Function Approximation , 2003, Discret. Event Dyn. Syst..

[35]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[36]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[37]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[38]  Steven J. Bradtke,et al.  Linear Least-Squares algorithms for temporal difference learning , 2004, Machine Learning.

[39]  David M. Young,et al.  Applied Iterative Methods , 2004 .

[40]  Justin A. Boyan,et al.  Technical Update: Least-Squares Temporal Difference Learning , 2002, Machine Learning.

[41]  Petros Drineas,et al.  Fast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a Matrix , 2006, SIAM J. Comput..

[42]  S. Muthukrishnan,et al.  Sampling algorithms for l2 regression and applications , 2006, SODA '06.

[43]  Petros Drineas,et al.  Fast Monte Carlo Algorithms for Matrices I: Approximating Matrix Multiplication , 2006, SIAM J. Comput..

[44]  Sean P. Meyn.  Control Techniques for Complex Networks: Workload , 2007 .

[45]  S. M. Moser.  Some expectations of a non-central chi-square distribution with an even number of degrees of freedom , 2007, TENCON 2007 - 2007 IEEE Region 10 Conference.

[46]  R. Vershynin,et al.  A Randomized Kaczmarz Algorithm with Exponential Convergence , 2007, math/0702226.

[47]  Charles L. Byrne,et al.  Applied Iterative Methods , 2007 .

[48]  S. Muthukrishnan,et al.  Relative-Error CUR Matrix Decompositions , 2007, SIAM J. Matrix Anal. Appl..

[49]  D. Cvetkovic-Ilic,et al.  A note on the representation for the Drazin inverse of 2×2 block matrices , 2008 .

[50]  V. Borkar.  Stochastic Approximation: A Dynamical Systems Viewpoint , 2008, Texts and Readings in Mathematics.

[51]  D. Bertsekas,et al.  Journal of Computational and Applied Mathematics Projected Equation Methods for Approximate Solution of Large Linear Systems , 2022 .

[52]  Y. Censor,et al.  A Note on the Behavior of the Randomized Kaczmarz Algorithm of Strohmer and Vershynin , 2009, The journal of fourier analysis and applications.

[53]  Adrian S. Lewis,et al.  Randomized Methods for Linear Constraints: Convergence Rates and Conditioning , 2008, Math. Oper. Res..

[54]  S. Muthukrishnan,et al.  Faster least squares approximation , 2007, Numerische Mathematik.

[55]  D. Bertsekas.  Approximate policy iteration: a survey and some new methods , 2011 .

[56]  Dimitri P. Bertsekas,et al.  Temporal Difference Methods for General Projected Equations , 2011, IEEE Transactions on Automatic Control.

[57]  D. Bertsekas,et al.  On the Convergence of Simulation-Based Iterative Methods for Singular Linear Systems , 2012 .

[58]  D. Bertsekas,et al.  On the convergence of simulation-based iterative methods for solving singular linear systems , 2013 .

[59]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.