Exponential stability and a systematic synthesis of a neural network for quadratic minimization

Abstract A continuous-time network with piecewise linear neuron input-output characteristics is proposed for optimization applications. Certain qualitative properties of the network of fundamental importance in these applications, such as the uniqueness of equilibrium conditions and the global exponential stability with any arbitrarily prescribed degree of this equilibrium, are analytically investigated. Deriving guidance from the obtained analytical results, a systematic synthesis procedure is outlined for identifying the network parameters and the bias inputs to employ the neural network for efficiently solving the important class of optimization problems where the objective is to minimize a specified quadratic function in the decision variables. For demonstrating the versatility of the solution procedure, three illustrative applications, namely synthesis of a class of spatial filters popularly employed in image recognition, the design of an associative memory by a master-slave formulation and the estimation of parameters of a linear system by a least squares procedure are outlined and the superiority of the present approach over the existing results is indicated. Some of the present results concerning the characterization of the network equilibrium conditions and the network scaling for confining the equilibrium to desired operational ranges are of basic interest and are useful in other applications of the neural network besides the specific applications to solve optimization problems discussed in this paper.

[1]  D. Casasent,et al.  Correlation synthetic discriminant functions. , 1986, Applied optics.

[2]  B. K. Jenkins,et al.  Image restoration using a neural network , 1988, IEEE Trans. Acoust. Speech Signal Process..

[3]  J. Farrell,et al.  Qualitative analysis of neural networks , 1989 .

[4]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[5]  D. Casasent,et al.  Minimum average correlation energy filters. , 1987, Applied optics.

[6]  Robert M. Farber,et al.  Programming a massively parallel, computation universal system: Static behavior , 1987 .

[7]  Fernando J. Pineda,et al.  Dynamics and architecture for neural computation , 1988, J. Complex..

[8]  Abhijit Mahalanobis,et al.  Unified framework for the synthesis of synthetic discriminant functions with reduced noise variance and sharp correlation structure , 1990 .

[9]  B. V. Vijaya Kumar,et al.  Minimum-variance synthetic discriminant functions , 1986 .

[10]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[12]  S. Grossberg,et al.  Some physiological and biochemical consequences of psychological postulates. , 1968, Proceedings of the National Academy of Sciences of the United States of America.

[13]  C. A. Desoer,et al.  Nonlinear Systems Analysis , 1978 .

[14]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[15]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[16]  Wolfgang Hahn,et al.  Stability of Motion , 1967 .

[17]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[18]  D Casasent,et al.  Unified synthetic discriminant function computational formulation. , 1984, Applied optics.

[19]  Malur K. Sundareshan,et al.  Systematic design of associative memory networks: equilibrium confinement, exponential stability and gradient descent learning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[20]  M. Fiedler.  Special matrices and their applications in numerical mathematics , 1986 .

[21]  Ronald A. Iltis,et al.  Neural solution to the multitarget tracking data association problem , 1989 .

[22]  James A. Anderson,et al.  Cognitive and psychological computation with neural models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[23]  Karl Johan Åström,et al.  Adaptive Control , 1989, Embedded Digital Control with Microcontrollers.

[24]  Jerry M. Mendel,et al.  Lessons in digital estimation theory , 1986 .

[25]  Massimo A. Sivilotti,et al.  Real-time visual computations using analog CMOS processing arrays , 1987 .

[26]  J. Goodman,et al.  Neural networks for computation: number representations and programming complexity. , 1986, Applied optics.

[27]  Michael W. Marcellin,et al.  Predictive trellis coded quantization of speech , 1990, IEEE Trans. Acoust. Speech Signal Process..

[28]  Nikitas J. Dimopoulos,et al.  A study of the asymptotic behavior of neural networks , 1989 .

[29]  Wolfgang Porod,et al.  Qualitative analysis and synthesis of a class of neural networks , 1988 .

[30]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.