Three-Dimensional Structured Networks for Matrix Equation Solving

Two three-dimensional structured networks are developed for solving linear equations and the Lyapunov equation. The basic idea of the structured network approaches is to first represent a given equation-solving problem by a 3-D structured network so that if the network matches a desired pattern array, the weights of the linear neurons give the solution to the problem: then, train the 3-D structured network to match the desired pattern array using some training algorithms; and finally, obtain the solution to the specific problem from the converged weights of the network. The training algorithms for the two 3-D structured networks are proved to converge exponentially fast to the correct solutions. Simulations were performed to show the detailed convergence behaviors of the 3-D structured networks. >

[1]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[2]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[3]  L. Csanky,et al.  Fast Parallel Matrix Inversion Algorithms , 1976, SIAM J. Comput..

[4]  Joe Brewer,et al.  Kronecker products and matrix calculus in system theory , 1978 .

[5]  David J. Kuck,et al.  On Stable Parallel Linear System Solvers , 1978, JACM.

[6]  D. Heller.  A Survey of Parallel Algorithms in Numerical Linear Algebra. , 1978 .

[7]  H. T. Kung.  VLSI Systems and Computations , 1982 .

[8]  S. P. Kumar,et al.  Solving Linear Algebraic Equations on an MIMD Computer , 1983, JACM.

[9]  David J. Evans,et al.  Parallel Multisection Applied to the Eigenvalue Problem , 1983, Comput. J..

[10]  H. T. Kung,et al.  Numerically Stable Solution of Dense Systems of Linear Equations Using Mesh-Connected Processors , 1984 .

[11]  Adam W. Bojanczyk,et al.  Optimal Asynchronous Newton Method for the Solution of Nonlinear Equations , 1984, JACM.

[12]  A. Bojanczyk.  Complexity of Solving Linear Systems in Different Models of Computation , 1984 .

[13]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[14]  S. Kung,et al.  VLSI Array processors , 1985, IEEE ASSP Magazine.

[15]  Jack J. Dongarra,et al.  A fully parallel algorithm for the symmetric eigenvalue problem , 1985, PPSC.

[16]  Ahmed H. Sameh,et al.  A multiprocessor algorithm for the symmetric tridiagonal eigenvalue problem , 1985, PPSC.

[17]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[18]  Elizabeth R. Jessup,et al.  A Divide and Conquer Algorithm for Computing the Singular Value Decomposition , 1987, SIAM Conference on Parallel Processing for Scientific Computing.

[19]  S. Johnsson.  Solving tridiagonal systems on ensemble architectures , 1987 .

[20]  Michael W. Berry,et al.  A Multiprocessor Scheme for the Singular Value Decomposition , 1987, PPSC.

[21]  E. Jessup.  Parallel solution of the symmetric tridiagonal eigenproblem , 1989 .

[22]  Jerry M. Mendel,et al.  Structured trainable networks for matrix algebra , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[23]  Nelson Morgan,et al.  Artificial neural networks: electronic implementations—an introduction , 1990 .

[24]  J. Mendel,et al.  Matrix computations and equation solving using structured networks and training , 1990, 29th IEEE Conference on Decision and Control.

[25]  Ulrich Rückert,et al.  VLSI Design of Neural Networks , 1990 .

[26]  Jerry M. Mendel,et al.  Cumulant-based parameter estimation using structured networks , 1991, IEEE Trans. Neural Networks.

[27]  Marios M. Polycarpou,et al.  Learning and convergence analysis of neural-type structured networks , 1992, IEEE Trans. Neural Networks.

[28]  Jerry M. Mendel,et al.  Parallel Structured Networks for Solving a Wide Variety of Matrix Algebra Problems , 1992, J. Parallel Distributed Comput..