Linear Least Least Simplified Neural Networks for Solving Squares Problems in Real Time Squares and Total

In this paper a new class of simplified low-cost analog artificial neural networks with on chip adaptive learning algorithms are proposed for solving linear systems of algebraic equations in real time. The proposed learning algorithms for linear least squares (LS), total least squares (TLS) and data least squares (DLS) problems can be considered as modifications and extensions of well known algorithms: the row-action projection-Kaczmarz algorithm [25] andlor the LMS (Adaline) Widrow-Hoff algorithms [21]. The algorithms can be applied to any problem which can be formulated as a linear regression problem. The correctness and high performance of the proposed neural networks are illustrated by extensive computer simulation results.

[1]  Dianne P. O’LEARYt.  ROBUST REGRESSION COMPUTATION USING ITERATIVELY REWEIGHTED LEAST SQUARES * , 2022 .

[2]  Malur K. Sundareshan,et al.  Exponential stability and a systematic synthesis of a neural network for quadratic minimization , 1991, Neural Networks.

[3]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[4]  Jerry M. Mendel,et al.  Parallel Structured Networks for Solving a Wide Variety of Matrix Algebra Problems , 1992, J. Parallel Distributed Comput..

[5]  Willard L. Miranker,et al.  New techniques for fast hybrid solutions of systems of equations , 1989 .

[6]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[7]  Sabine Van Huffel,et al.  Algebraic connections between the least squares and total least squares problems , 1989 .

[8]  D. O’Leary.  Robust regression computation computation using iteratively reweighted least squares , 1990 .

[9]  A. Cichocki,et al.  Neural networks for solving systems of linear equations and related problems , 1992 .

[10]  M. Hanke,et al.  On the acceleration of Kaczmarz's method for inconsistent linear systems , 1990 .

[11]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[12]  Musheng Wei,et al.  Algebraic relations between the total least squares and least squares problems with more than one solution , 1992 .

[13]  C. Jennison,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[14]  Mauro Forti,et al.  Neural network for adaptive FIR filtering , 1990 .

[15]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[16]  Andrzej Cichocki,et al.  Neural networks for linear inverse problems with incomplete data especially in applications to signal and image reconstruction , 1995, Neurocomputing.

[17]  C.R.K. Marrian,et al.  A neural net approach to discrete Hartley and Fourier transforms , 1989 .

[18]  K. S. Arun,et al.  A Unitarily Constrained Total Least Squares Problem in Signal Processing , 1992, SIAM J. Matrix Anal. Appl..

[19]  Joos Vandewalle,et al.  A neural net approach to discrete hartley and fourier-transforms - comment , 1992 .

[20]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[21]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[22]  Andrzej Cichocki,et al.  MOS Switched-Capacitor and Continuous-Time Integrated Circuits and Systems: Analysis and Design , 1989 .

[23]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[24]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[25]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[27]  Eric M. Dowling,et al.  The Data Least Squares Problem and Channel Equalization , 1993, IEEE Trans. Signal Process..

[28]  Sabine Van Huffel.  On the Significance of Nongeneric Total Least Squares Problems , 1992, SIAM J. Matrix Anal. Appl..

[29]  G. Saridis.  Parameter estimation: Principles and problems , 1983, Proceedings of the IEEE.

[30]  Stuart Haber,et al.  A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks , 1991 .

[31]  Richard J. Mammone.  Computational methods of signal recovery and recognition , 1992 .

[32]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .