Fast and efficient parallel solution of dense linear systems

The most efficient previously known parallel algorithms for the inversion of a nonsingular n × n matrix A or solving a linear system Ax = b over the rational numbers require O(log2n) time and M(n)n processors [provided that M(n) processors suffice in order to multiply two n × n rational matrices in time O(logn)]. Furthermore, the known polylog arithmetic time algorithms for those problems are numerically unstable. In this paper we apply Newton's iteration and initially choose an approximate inverse matrix by following Ben-Israel. This quadratically convergent and numerically stable iterative method takes O(log2n) parallel time using M(n) processors to compute the inverse (within the relative precision 2−nc for a positive constant c) of an n × n rational matrix A with the condition number at most nd for a constant d. This is the optimum processor bound and by a factor of n improvement of the previously known processor bounds for polylogarithmic time matrix inversion. The algorithm does not require to precompute the condition number of the input matrix, but it just converges slower for ill-conditioned input matrices.

[1]  Franco P. Preparata,et al.  An Improved Parallel Processor Bound in Fast Matrix Inversion , 1978, Inf. Process. Lett..

[2]  Victor Y. Pan,et al.  How to Multiply Matrices Faster , 1984, Lecture Notes in Computer Science.

[3]  V. Pan,et al.  Fast and efficient linear programming and linear least-squares computations , 1986 .

[4]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[5]  L. Csanky,et al.  Fast parallel matrix inversion algorithms , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).

[6]  Victor Y. Pan,et al.  Fast and Efficient Parallel Algorithms for the Exact Inversion of Integer Matrices , 1985, FSTTCS.

[7]  H. Hotelling Some New Methods in Matrix Calculation , 1943 .

[8]  H. Hotelling Further Points on Matrix Calculation and Simultaneous Equations , 1943 .

[9]  Leslie G. Valiant,et al.  Fast Parallel Computation of Polynomials Using Few Processors , 1983, SIAM J. Comput..

[10]  Allan Borodin,et al.  The computational complexity of algebraic and numeric problems , 1975, Elsevier computer science library.

[11]  Alston S. Householder,et al.  The Theory of Matrices in Numerical Analysis , 1964 .

[12]  Robert Schreiber,et al.  Computing generalized inverses and eigenvalues of symmetric matrices using sytolic arrays , 1985 .

[13]  Vijay V. Vazirani,et al.  Matching is as easy as matrix inversion , 1987, STOC.

[14]  Victor Y. Pan,et al.  Fast and Efficient Parallel Inversion of Toeplitz and Block Toeplitz Matrices , 1989 .

[15]  G. Stewart,et al.  On the Numerical Properties of an Iterative Method for Computing the Moore–Penrose Generalized Inverse , 1974 .

[16]  James Hardy Wilkinson,et al.  Error Analysis of Direct Methods of Matrix Inversion , 1961, JACM.

[17]  J. H. Wilkinson The algebraic eigenvalue problem , 1966 .

[18]  Victor Y. Pan,et al.  An Improved Newton Iteration for the Generalized Inverse of a Matrix, with Applications , 1991, SIAM J. Sci. Comput..

[19]  Victor Y. Pan,et al.  Complexity of Parallel Matrix Computations , 1987, Theor. Comput. Sci..

[20]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[21]  Victor Y. Pan,et al.  Parallel Evaluation of the Determinant and of the Inverse of a Matrix , 1989, Inf. Process. Lett..

[22]  Victor Y. Pan,et al.  Fast and Efficient Algorithms for Linear Programming and for the Linear Least Squares Problem. , 1985 .

[23]  Gene H. Golub,et al.  Matrix computations , 1983 .

[24]  J. Hopcroft,et al.  Fast parallel matrix and GCD computations , 1982, FOCS 1982.

[25]  Adi Ben-Israel,et al.  On Iterative Computation of Generalized Inverses and Associated Projections , 1966 .

[26]  Joachim von zur Gathen,et al.  Parallel algorithms for algebraic problems , 1983, SIAM J. Comput..

[27]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[28]  A. Bojanczyk Complexity of Solving Linear Systems in Different Models of Computation , 1984 .

[29]  Victor Y. Pan,et al.  Improved processor bounds for combinatorial problems in RNC , 1988, Comb..

[30]  Adi Ben-Israel,et al.  A note on an iterative method for generalized inversion of matrices , 1966 .

[31]  K. Ramachandra,et al.  Vermeidung von Divisionen. , 1973 .

[32]  Victor Y. Pan,et al.  Efficient parallel solution of linear systems , 1985, STOC '85.

[33]  G. Schulz Iterative Berechung der reziproken Matrix , 1933 .

[34]  Joachim von zur Gathen Parallel algorithms for algebraic problems , 1983, STOC '83.

[35]  Adi Ben-Israel,et al.  A Hyperpower Iterative Method for Computing Matrix Products Involving the Generalized Inverse , 1971 .

[36]  Allan Borodin,et al.  Fast parallel matrix and GCD computations , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[37]  Stuart J. Berkowitz,et al.  On Computing the Determinant in Small Parallel Time Using a Small Number of Processors , 1984, Inf. Process. Lett..

[38]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[39]  L. Csanky,et al.  Fast Parallel Matrix Inversion Algorithms , 1976, SIAM J. Comput..

[40]  H. Keller,et al.  Analysis of Numerical Methods , 1967 .