ANASA-a stochastic reinforcement algorithm for real-valued neural computation

This paper introduces ANASA (adaptive neural algorithm of stochastic activation), a new, efficient, reinforcement learning algorithm for training neural units and networks with continuous output. The proposed method employs concepts, found in self-organizing neural networks theory and in reinforcement estimator learning algorithms, to extract and exploit information relative to previous input pattern presentations. In addition, it uses an adaptive learning rate function and a self-adjusting stochastic activation to accelerate the learning process. A form of optimal performance of the ANASA algorithm is proved (under a set of assumptions) via strong convergence theorems and concepts. Experimentally, the new algorithm yields results, which are superior compared to existing associative reinforcement learning methods in terms of accuracy and convergence rates. The rapid convergence rate of ANASA is demonstrated in a simple learning task, when it is used as a single neural unit, and in mathematical function modeling problems, when it is used to train various multilayered neural networks.

[1]  Bart Kosko,et al.  Unsupervised learning in noise , 1990, International 1989 Joint Conference on Neural Networks.

[2]  Z.-P. Lo,et al.  Analysis of a learning algorithm for neural network classifiers , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[3]  V. Gullapalli,et al.  Associative reinforcement learning of real-valued functions , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[4]  D. Gorse,et al.  AN ANALYSIS OF NOISY RAM AND NEURAL NETS , 1989 .

[5]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[6]  R. S. Nutter,et al.  An extended back-propagation learning algorithm by using heterogeneous processing units , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[7]  Kumpati S. Narendra,et al.  Learning Automata - A Survey , 1974, IEEE Trans. Syst. Man Cybern..

[8]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[9]  John G. Taylor,et al.  A continuous input RAM-based stochastic neural model , 1991, Neural Networks.

[10]  E. G. Gladyshev.  On Stochastic Approximation , 1965 .

[11]  Athanasios V. Vasilakos,et al.  Ergodic discretized estimator learning automata with high accuracy and high adaptation rate for nonstationary environments , 1992, Neurocomputing.

[12]  Athanasios V. Vasilakos,et al.  A new approach to the design of reinforcement schemes for learning automata: Stochastic estimator learning algorithm , 1995, Neurocomputing.

[13]  A G Barto,et al.  Learning by statistical cooperation of self-interested neuron-like computing elements. , 1985, Human neurobiology.

[14]  Snehasis Mukhopadhyay,et al.  Associative learning in random environments using neural networks , 1991, IEEE Trans. Neural Networks.

[15]  R. Szabo,et al.  A fast learning algorithm for neural network applications , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[16]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  V. Gullapalli,et al.  Acquiring robot skills via reinforcement learning , 1994, IEEE Control Systems.

[18]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[19]  Taek Mu Kwon.  Gaussian perceptron: experimental results , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[20]  T. L. Ruchti,et al.  Gray layer technology: incorporating a priori knowledge into feedforward artificial neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[21]  E Harth,et al.  Alopex: a stochastic method for determining visual receptive fields. , 1974, Vision research.