Parallel Structured Networks for Solving a Wide Variety of Matrix Algebra Problems

Abstract In this paper, structured networks, which are multilayer feed-forward neural networks with linear neurons and with additional constraints on the weights of neurons, are developed for solving a wide variety of matrix algebra problems, including LU decomposition, matrix inversion, QR factorization, singular value decomposition, symmetric eigenproblem, and Schur decomposition. The basic idea of the structured network approaches is: first, represent a given matrix algebra problem by a structured network so that if the network matches a set of desired patterns, the weights of the linear neurons give the solution to the problem; then, train the structured network to match the desired patterns, and obtain the solution to the problem from the converged weights of neurons. Modified error back-propagation algorithms are developed for training the structured networks. Time complexities of these structured network approaches are analyzed. These approaches are tested for random matrices through Monte Carlo simulations. Finally, a general-purpose structured network is developed which can be programmed to solve all the matrix algebra problems considered in this paper.

[1]  Ahmed H. Sameh,et al.  A multiprocessor algorithm for the symmetric tridiagonal eigenvalue problem , 1985, PPSC.

[2]  Willard L. Miranker,et al.  Fast Hybrid Solution of Algebraic Systems , 1990, SIAM J. Sci. Comput..

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  H. T. Kung,et al.  Numerically Stable Solution of Dense Systems of Linear Equations Using Mesh-Connected Processors , 1984 .

[5]  David J. Evans,et al.  Parallel Multisection Applied to the Eigenvalue Problem , 1983, Comput. J..

[6]  Jerry M. Mendel,et al.  Cumulant-based parameter estimation using structured networks , 1991, IEEE Trans. Neural Networks.

[7]  Christian H. Bischof,et al.  A Parallel QR Factorization Algorithm with Controlled Local Pivoting , 1991, SIAM J. Sci. Comput..

[8]  H. T. Kung.  VLSI Systems and Computations , 1982 .

[9]  D. Heller.  A Survey of Parallel Algorithms in Numerical Linear Algebra. , 1978 .

[10]  J. Mendel,et al.  Matrix computations and equation solving using structured networks and training , 1990, 29th IEEE Conference on Decision and Control.

[11]  Michael W. Berry,et al.  A Multiprocessor Scheme for the Singular Value Decomposition , 1987, PPSC.

[12]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[13]  David J. Kuck,et al.  Parallel Computation of Eigenvalues of Real Matrices , 1971, IFIP Congress.

[14]  Alan George,et al.  QR Factorization of a Dense Matrix on a Hypercube Multiprocessor , 1990, SIAM J. Sci. Comput..

[15]  Jack J. Dongarra,et al.  A fully parallel algorithm for the symmetric eigenvalue problem , 1985, PPSC.

[16]  Jerry M. Mendel,et al.  Structured trainable networks for matrix algebra , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[17]  S. Kung,et al.  VLSI Array processors , 1985, IEEE ASSP Magazine.

[18]  S. Johnsson.  Solving tridiagonal systems on ensemble architectures , 1987 .

[19]  K. Jainandunsing,et al.  A new class of parallel algorithms for solving systems of linear equations , 1989 .

[20]  Jerry M. Mendel,et al.  Three-Dimensional Structured Networks for Matrix Equation Solving , 1991, IEEE Trans. Computers.

[21]  T. Davis,et al.  A nondeterministic parallel algorithm for general unsymmetric sparse lu factorization , 1990 .

[22]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[23]  Marios M. Polycarpou,et al.  Learning and convergence analysis of neural-type structured networks , 1992, IEEE Trans. Neural Networks.

[24]  K. A. Gallivan,et al.  Parallel Algorithms for Dense Linear Algebra Computations , 1990, SIAM Rev..

[25]  A. Bojanczyk.  Complexity of Solving Linear Systems in Different Models of Computation , 1984 .

[26]  Ilse C. F. Ipsen.  Singular Value Decomposition With Systolic Arrays , 1984, Optics & Photonics.

[27]  L. Csanky,et al.  Fast parallel matrix inversion algorithms , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).

[28]  David J. Kuck,et al.  On Stable Parallel Linear System Solvers , 1978, JACM.

[29]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[30]  E. Jessup.  Parallel solution of the symmetric tridiagonal eigenproblem , 1989 .

[31]  S. P. Kumar,et al.  Solving Linear Algebraic Equations on an MIMD Computer , 1983, JACM.