Analysis of associative reinforcement learning in neural networks using iterated function systems

A mathematical theory of associative reinforcement learning in neural networks is developed in terms of random iterated function systems (IFSs), which are finite sets of random maps on metric spaces. In particular, the stochastic search for an associative mapping that maximizes the expected pay-off arising from reinforcement is formulated as a random IFS on weight-space. The dynamical evolution of the weights is described by a Markov process. If this process is ergodic then the limiting behavior of the system is described by an invariant probability measure on weight space that can have a fractal-like structure. A class of associative reinforcement learning algorithms is constructed that is an extension of the nonassociative schemes used in stochastic automata theory. The issue of generalization is discussed within the IFS framework and related to the stochastic and possibly fractal nature of the learning process. >

[1]  Kumpati S. Narendra,et al.  Learning Automata - A Survey , 1974, IEEE Trans. Syst. Man Cybern..

[2]  Haim Sompolinsky,et al.  Learning from Examples in a Single-Layer Neural Network , 1990 .

[3]  M. Norman.  Some convergence theorems for stochastic learning models with distance diminishing operators , 1968 .

[4]  P. Erdös.  On a Family of Symmetric Bernoulli Convolutions , 1939 .

[5]  S. Karlin.  Some random walks arising in learning models. I. , 1953 .

[6]  Paul C. Bressloff,et al.  Neural networks, learning automata and iterated function systems , 1991 .

[7]  Naftali Tishby,et al.  Consistent inference of probabilities in layered networks: predictions and generalizations , 1989, International 1989 Joint Conference on Neural Networks.

[8]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[9]  S. Lakshmivarahan,et al.  Learning Algorithms Theory and Applications , 1981 .

[10]  K. Falconer.  The geometry of fractal sets: Contents , 1985 .

[11]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  Michael F. Barnsley,et al.  A better way to compress images , 1988 .

[13]  J. Elton.  An ergodic theorem for iterated maps , 1987, Ergodic Theory and Dynamical Systems.

[14]  E. Gardner.  The space of interactions in neural network models , 1988 .

[15]  Györgyi,et al.  Inference of a rule by a neural network with thermal noise. , 1990, Physical review letters.

[16]  A G Barto,et al.  Learning by statistical cooperation of self-interested neuron-like computing elements. , 1985, Human neurobiology.

[17]  Sara A. Solla,et al.  Learning contiguity with layered neural networks , 1988, Neural Networks.

[18]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[19]  Lawrence D. Jackel,et al.  Large Automatic Learning, Rule Extraction, and Generalization , 1987, Complex Syst..

[20]  Jaroslav Stark.  Iterated Function Systems as neural networks , 1991, Neural Networks.

[21]  W. Rudin.  Real and complex analysis , 1968 .