Analysis and applications of general classes of dynamic neural networks

Research interest in neural networks has grown over the past few years in the hope that they may offer more efficient alternatives to conventional algorithms. Generally speaking, along the path from research to development two main issues arise, namely (i) qualitative behavior of the systems, and (ii) training rules. Qualitative analysis of first order networks has been carried out by Cohen and Grossberg, among others. Widrow, Rumelhart, Hopfield, and others have proposed various training rules for different network structures. In this thesis results pertaining to training as well as to qualitative analysis of neural networks are presented. In some cases they represent generalizations of existing results, and in other cases they introduce entirely novel concepts. First, a new technique for training neural networks based on optimal control theory is presented. This method is different from many existing rules in that it places very few constraints on the order or architecture of the network. The method yields an optimal weight matrix that is a function of time. The optimal control technique is applied to train the weights in an associative memory. For this problem, a common weight rule is the outer product rule, introduced by Hopfield. By considering special cases of the performance index, optimal rules for the problem are derived, and encouraging simulation results are presented. Still addressing the issue of neural network training, the optimal control technique above is applied to determine the weights in a Probabilistic Cellular Automaton (PCA) for pattern recognition. Two ways of determining the weights in this structure are examined, and simulation results are presented for some simple examples. Finally, a qualitative analysis of a class of arbitrary order dynamic neural networks is presented. Such networks at steady state can give rise to polynomial threshold functions (Bruck 1989). Other applications for such networks include higher order associative memories and nonlinear programming. All these applications place certain constraints on the nature of the equilibrium points of the neural network. The analysis characterizes these equilibrium points.