Parallel Complexity of Computations with General and Toeplitz-Like Matrices Filled with Integers and Extensions

Computations with Toeplitz and Toeplitz-like matrices are fundamental for many areas of algebraic and numerical computing. The list of computational problems reducible to Toeplitz and Toeplitz-like computations includes, in particular, the evaluation of the greatest common divisor (gcd), the least common multiple (lcm), and the resultant of two polynomials, computing Pade approximation and the Berlekamp--Massey recurrence coefficients, as well as numerous problems reducible to these. Transition to Toeplitz and Toeplitz-like computations is currently the basis for the design of the parallel randomized NC (RNC) algorithms for these computational problems. Our main result is in constructing nearly optimal randomized parallel algorithms for Toeplitz and Toeplitz-like computations and, consequently, for numerous related computational problems (including the computational problems listed above), where all the input values are integers and all the output values are computed exactly. This includes randomized parallel algorithms for computing the rank, the determinant, and a basis for the null-space of an n × n Toeplitz or Toeplitz-like matrix A filled with integers, as well as a solution x to a linear system A x= f if the system is consistent. Our algorithms use O((log n) log (n log |A|)) parallel time and O(n log n) processors, each capable of performing (in unit time) an arithmetic operation, a comparision, or a rounding of a rational number to a closest integer. The cost bounds cover the cost of the verification of the correctness of the output. The computations by these algorithms can be performed with the precision of O(n log |A|) bits, which matches the precision required in order to represent the output, except for the rank computation, where the precision of the computation decreases. The algorithms involve either a single random parameter or at most 2n-1 parameters. The cited processor bounds are less by roughly factor n than ones supported by the known algorithms that run in polylogarithmic arithmetic time and do not use rounding to the closest integers. Technically, we first devise new algorithms supporting our old nearly optimal complexity estimates for parallel computations with general matrices filled with integers. Then we decrease dramatically, by roughly factor n1.376, the processor bounds required in these algorithms in the case where the input matrix is Toeplitz-like. Our algorithms exploit and combine some new techniques (which may be of independent interest, e.g., in the study of parallel and sequential computation of recursive factorization of integer matrices) as well as our earlier techniques of variable diagonal (relating to each other several known algebraic and numerical methods), stream contraction, and the truncation of displacement generators in Toeplitz-like computations; our development and application of these techniques may be of independent

[1]  V. Strassen Gaussian elimination is not optimal , 1969 .

[2]  M. Morf,et al.  Inverses of Toeplitz operators, innovations, and orthogonal polynomials , 1975, 1975 IEEE Conference on Decision and Control including the 14th Symposium on Adaptive Processes.

[3]  Victor Y. Pan,et al.  Processor efficient parallel solution of linear systems over an abstract field , 1991, SPAA '91.

[4]  Victor Y. Pan,et al.  A unified superfast algorithm for boundary rational tangential interpolation problems and for inversion and factorization of dense structured matrices , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[5]  Joachim von zur Gathen,et al.  Parallel algorithms for algebraic problems , 1983, SIAM J. Comput..

[6]  Victor Y. Pan,et al.  Optimal (up to polylog factors) sequential and parallel algorithms for approximating complex polynomial zeros , 1995, STOC '95.

[7]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[8]  Gary L. Miller,et al.  Efficient Parallel Evaluation of Straight-Line Code and Arithmetic Circuits , 1988, SIAM J. Comput..

[9]  V. Pan PARAMETRIZATION OF NEWTON'S ITERATION FOR COMPUTATIONS WITH STRUCTURED MATRICES AND APPLICATIONS , 1992 .

[10]  Martin Morf,et al.  Doubling algorithms for Toeplitz and related equations , 1980, ICASSP.

[11]  Robert T. Moenck,et al.  Approximate algorithms to derive exact solutions to systems of linear equations , 1979, EUROSAM.

[12]  David Y. Y. Yun,et al.  Fast Solution of Toeplitz Systems of Equations and Computation of Padé Approximants , 1980, J. Algorithms.

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  Michael Rosen,et al.  A classical introduction to modern number theory , 1982, Graduate texts in mathematics.

[15]  Victor Y. Pan,et al.  The structure of sparse resultant matrices , 1997, ISSAC.

[16]  J. Makhoul,et al.  Linear prediction: A tutorial review , 1975, Proceedings of the IEEE.

[17]  Victor Y. Pan,et al.  Controlled iterative methods for solving polynomial systems , 1998, ISSAC '98.

[18]  Erich Kaltofen,et al.  On Wiedemann's Method of Solving Sparse Linear Systems , 1991, AAECC.

[19]  M. Morf,et al.  Displacement ranks of matrices and linear equations , 1979 .

[20]  L. Csanky,et al.  Fast parallel matrix inversion algorithms , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).

[21]  D. Rose,et al.  Generalized nested dissection , 1977 .

[22]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[23]  Allan Borodin,et al.  Fast parallel matrix and GCD computations , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[24]  B. Anderson,et al.  Asymptotically fast solution of toeplitz and related systems of linear equations , 1980 .

[25]  Joachim von zur Gathen Parallel algorithms for algebraic problems , 1983, STOC '83.

[26]  Elwyn R. Berlekamp,et al.  Algebraic coding theory , 1984, McGraw-Hill series in systems science.

[27]  Victor Y. Pan,et al.  Newton's iteration for structured matrices , 1999 .

[28]  Victor Y. Pan,et al.  Matrix structure, polynomial arithmetic, and erasure-resilient encoding/decoding , 2000, ISSAC.

[29]  M. Morf Fast Algorithms for Multivariable Systems , 1974 .

[30]  Victor Y. Pan,et al.  Fast and Efficient Parallel Solution of Sparse Linear Systems , 1993, SIAM J. Comput..

[31]  John H. Reif,et al.  Work efficient parallel solution of Toeplitz systems and polynomial GCD , 1995, STOC '95.

[32]  Victor Y. Pan,et al.  Fast and Efficient Parallel Algorithms for the Exact Inversion of Integer Matrices , 1985, FSTTCS.

[33]  S. Sitharama Iyengar,et al.  Introduction to parallel algorithms , 1998, Wiley series on parallel and distributed computing.

[34]  Stuart J. Berkowitz,et al.  On Computing the Determinant in Small Parallel Time Using a Small Number of Processors , 1984, Inf. Process. Lett..

[35]  Victor Y. Pan,et al.  Work-Preserving Speed-Up of Parallel Matrix Computations , 1995, SIAM J. Comput..

[36]  Stephen R. Tate,et al.  Optimal size integer division circuits , 1989, STOC '89.

[37]  J. H. Wilkinson The algebraic eigenvalue problem , 1966 .

[38]  R. Cottle Manifestations of the Schur complement , 1974 .

[39]  Thomas Kailath,et al.  Fast reliable algorithms for matrices with structure , 1999 .

[40]  Victor Y. Pan,et al.  Solving a Polynomial Equation: Some History and Recent Progress , 1997, SIAM Rev..

[41]  Victor Y. Pan,et al.  Asymptotic acceleration of solving multivariate polynomial systems of equations , 1998, STOC '98.

[42]  Victor Y. Pan,et al.  Complexity of Computations with Matrices and Polynomials , 1992, SIAM Rev..

[43]  Dario Bini,et al.  Stability of fast algorithms for matrix multiplication , 1980 .

[44]  E Spedicato Computer algorithms for solving linear algebraic equations : the state of the art , 1991 .

[45]  Subrahmanyan Chandrasekhar,et al.  On the radiative equilibrium of a stellar atmosphere , 1944 .

[46]  B. Cuny,et al.  Fast and Efficient Parallel Solution of Dense Linear Systems , 1989 .

[47]  M. J. Quinn,et al.  Parallel Computing: Theory and Practice , 1994 .

[48]  J. Hopcroft,et al.  Fast parallel matrix and GCD computations , 1982, FOCS 1982.

[49]  J. Edmonds Systems of distinct representatives and linear algebra , 1967 .

[50]  Alexander L. Chistov,et al.  Fast parallel calculation of the rank of matrices over a field of arbitrary characteristic , 1985, FCT.

[51]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[52]  Majid Ahmadi,et al.  Digital Filtering in One and Two Dimensions: Design and Applications , 1989 .

[53]  Victor Y. Pan,et al.  The Parallel Computation of Minimum Cost Paths in Graphs by Stream Contraction , 1991, Inf. Process. Lett..

[54]  Bruce Ronald. Musicus,et al.  Levinson and fast Choleski algorithms for Toeplitz and almost Toeplitz matrices , 1988 .

[55]  V.Y. Pan,et al.  Concurrent Iterative Algorithm for Toeplitz-like Linear Systems , 1993, IEEE Trans. Parallel Distributed Syst..

[56]  Joachim von zur Gathen,et al.  Parallel Arithmetic Computations: A Survey , 1986, MFCS.

[57]  V. Pan Optimal and nearly optimal algorithms for approximating polynomial zeros , 1996 .

[58]  Erich Kaltofen,et al.  Analysis of Coppersmith's Block Wiedemann Algorithm for the Parallel Solution of Sparse Linear Systems , 1993, AAECC.

[59]  Victor Y. Pan Decreasing the displacement rank of a matrix , 1993 .

[60]  Frank Thomson Leighton Introduction to parallel algorithms and architectures: arrays , 1992 .

[61]  C. Cullen An Introduction to Numerical Linear Algebra , 1993 .

[62]  F. Leighton,et al.  Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes , 1991 .

[63]  Thomas Kailath,et al.  Least-squares adaptive lattice and transversal filters: A unified geometric theory , 1984, IEEE Trans. Inf. Theory.

[64]  E. Bareiss Sylvester’s identity and multistep integer-preserving Gaussian elimination , 1968 .

[65]  Victor Y. Pan,et al.  Parallel Computation of Polynomial GCD and Some Related Parallel Computations over Abstract Fields , 1996, Theor. Comput. Sci..

[66]  Richard M. Karp,et al.  A Survey of Parallel Algorithms for Shared-Memory Machines , 1988 .

[67]  Victor Y. Pan,et al.  Processor-efficient parallel solution of linear systems. II. The positive characteristic and singular cases , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[68]  Joseph F. Traub,et al.  On Euclid's Algorithm and the Theory of Subresultants , 1971, JACM.

[69]  T. Kailath,et al.  An inverse scattering framework for several problems in signal processing , 1987, IEEE ASSP Magazine.

[70]  Leslie G. Valiant,et al.  Fast Parallel Computation of Polynomials Using Few Processors , 1983, SIAM J. Comput..

[71]  V. Pan On computations with dense structured matrices , 1990 .

[72]  Victor Y. Pan,et al.  Complexity of Parallel Matrix Computations , 1987, Theor. Comput. Sci..

[73]  Victor Y. Pan,et al.  A new approach to fast polynomial interpolation and multipoint evaluation , 1993 .

[74]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[75]  James R. Bunch,et al.  Stability of Methods for Solving Toeplitz Systems of Equations , 1985 .

[76]  Thomas Kailath,et al.  A view of three decades of linear filtering theory , 1974, IEEE Trans. Inf. Theory.

[77]  Victor Y. Pan,et al.  Parallel complexity of tridiagonal symmetric Eigenvalue problem , 1991, SODA '91.

[78]  Thomas Kailath Signal-processing applications of some moment problems , 1987 .

[79]  Joseph JáJá,et al.  An Introduction to Parallel Algorithms , 1992 .

[80]  Victor Y. Pan,et al.  Nearly optimal computations with structured matrices , 2014, Theor. Comput. Sci..

[81]  John H. Reif An O(nlog/sup 3/ n) algorithm for the real root problem , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[82]  W. Gragg,et al.  Superfast solution of real positive definite toeplitz systems , 1988 .

[83]  Victor Y. Pan,et al.  Improved parallel computations with Toeplitz-like and Hankel-like matrices☆☆☆ , 1993 .

[84]  Thomas Kailath,et al.  Fast Gaussian elimination with partial pivoting for matrices with displacement structure , 1995 .

[85]  Michael Ben-Or,et al.  Simple algorithms for approximating all roots of a polynomial with real roots , 1990, J. Complex..

[86]  Georg Heinig,et al.  Inversion of generalized Cauchy matrices and other classes of structured matrices , 1995 .

[87]  F. Hoog A new algorithm for solving Toeplitz systems of equations , 1987 .

[88]  Victor Y. Pan,et al.  Complexity of Algorithms for Linear Systems of Equations , 1991 .

[89]  David Eppstein,et al.  Parallel Algorithmic Techniques for Combinatorial Computation , 1988, ICALP.