Upper bounds on the complexity of solving systems of linear equations

AbstractThe article is of the nature of a survey and is devoted to direct (exact) methods of solving systems of linear equations examined from the point of view of their computational complexity. The construction of most of the algorithms is outlined. The paper consists of two parts. Series methods of solving systems of linear equations are examined in the first part. It includes the algorithms of Gauss and of Konoval'tsev, Strassen's algorithm and its modifications, the YunGustavson results for Toeplitz systems, etc. The second part is devoted to parallel methods of solving systems of linear equations. Examined here are the parallelization of the Gauss algorithm, the results of Hyafil and Kung on complexity estimate of the parallel solution of triangular systems, Csanky's results based on the parallelization of Leverrier's method, Hyabil's general result on the parallelization of a straight-line program for computing polynomials, Stone's algorithm for the parallel solving of tridiagonal systems. Several new bounds are derived. In particular, if a pair of (n×n) -matrices can be multiplied sequentially by a straight-line program of complexity O(nd), then it is possible to solve an arbitrary system of m linear equations in n unknowns on p processors with the complexity $$0\left( {\frac{{max(m, n)min(m, n)^{\alpha - 1} }}{p} + min(m, n)log_2 max(m, n)} \right),$$ , and to solve a triangular system of sizen with the complexity $$0\left( {\frac{{n^2 }}{p} + \frac{n}{{p^{1/\alpha } }}log_2^{1 - \tfrac{1}{\alpha }} n + log_2^2 n} \right).$$

[1]  Don Coppersmith,et al.  On the Asymptotic Complexity of Matrix Multiplication , 1982, SIAM J. Comput..

[2]  V. N. Faddeeva,et al.  Parallel computations in linear algebra , 1977 .

[3]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[4]  Jr. Samuel Ellis Orcutt.  Computer organization and algorithms for very-high speed computations. , 1975 .

[5]  D. Heller.  A Survey of Parallel Algorithms in Numerical Linear Algebra. , 1978 .

[6]  Laurent Hyafil.  On the Parallel Evaluation of Multivariate Polynomials , 1979, SIAM J. Comput..

[7]  Harold S. Stone,et al.  An Efficient Parallel Algorithm for the Solution of a Tridiagonal Linear System of Equations , 1973, JACM.

[8]  Marshall C. Pease.  Matrix Inversion Using Parallel Processing , 1967, JACM.

[9]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[10]  Grazia Lotti,et al.  O(n2.7799) Complexity for n*n Approximate Matrix Multiplication , 1979, Inf. Process. Lett..

[11]  Leslie G. Valiant,et al.  Fast Parallel Computation of Polynomials Using Few Processors , 1983, SIAM J. Comput..

[12]  Allan Borodin,et al.  The computational complexity of algebraic and numeric problems , 1975, Elsevier computer science library.

[13]  Leslie G. Valiant,et al.  Fast Parallel Computation of Polynomials Using Few Processes , 1981, MFCS.

[14]  J. Hopcroft,et al.  Triangular Factorization and Inversion by Fast Matrix Multiplication , 1974 .

[15]  Leslie G. Valiant.  Computing Multivariate Polynomials in Parallel , 1980, Inf. Process. Lett..

[16]  Omar Wing,et al.  A Computation Model of Parallel Solution of Linear Equations , 1980, IEEE Transactions on Computers.

[17]  Le Verrier,et al.  Mémoire sur les variations séculaires des éléments des orbites : pour les sept planètes principales, Mercure, Vénus, la Terre, Mars, Jupiter, Saturne et Uranus , 1976 .

[18]  David J. Kuck,et al.  Time and Parallel Processor Bounds for Linear Recurrence Systems , 1975, IEEE Transactions on Computers.

[19]  L. Csanky,et al.  Fast Parallel Matrix Inversion Algorithms , 1976, SIAM J. Comput..

[20]  H. T. Kung,et al.  Parallel Algorithms for Solving Triangular Linear Systems with Small Parallelism , 1974 .

[21]  Walter Baur,et al.  The Complexity of Partial Derivatives , 1983, Theor. Comput. Sci..

[22]  Communications of the Moscow Mathematical Society: on Fast Calculations in Some Problems of Linear Algebra Connected with Toeplitz and Hankel Matrices , 1980 .

[23]  Arnold Schönhage,et al.  Partial and Total Matrix Multiplication , 1981, SIAM J. Comput..

[24]  Franco P. Preparata,et al.  An Improved Parallel Processor Bound in Fast Matrix Inversion , 1978, Inf. Process. Lett..

[25]  W. F. Trench.  An Algorithm for the Inversion of Finite Toeplitz Matrices , 1964 .

[26]  Victor Y. Pan,et al.  Strassen's algorithm is not optimal trilinear technique of aggregating, uniting and canceling for constructing fast algorithms for matrix operations , 1978, 19th Annual Symposium on Foundations of Computer Science (sfcs 1978).

[27]  J. L. Hock,et al.  An exact recursion for the composite nearest‐neighbor degeneracy for a 2×N lattice space , 1984 .

[28]  D. Faddeev,et al.  Computational methods of linear algebra , 1959 .

[29]  E. Bareiss.  Numerical solution of linear equations with Toeplitz and Vector Toeplitz matrices , 1969 .