Varieties of learning automata: an overview

Automata models of learning systems introduced in the 1960s were popularized as learning automata (LA) in a survey paper by Narendra and Thathachar (1974). Since then, there have been many fundamental advances in the theory as well as applications of these learning models. In the past few years, the structure of LA, has been modified in several directions to suit different applications. Concepts such as parameterized learning automata (PLA), generalized learning,automata (GLA), and continuous action-set learning automata (CALA) have been proposed, analyzed, and applied to solve many significant learning problems. Furthermore, groups of LA forming teams and feedforward networks have been shown to converge to desired solutions under appropriate learning algorithms. Modules of LA have been used for parallel operation with consequent increase in speed of convergence. All of these concepts and results are relatively new and are scattered in technical literature. An attempt has been made in this paper to bring together the main ideas involved in a unified framework and provide pointers to relevant references.

[1]  M. L. Tsetlin.  On the Behavior of Finite Automata in Random Media , 1961 .

[2]  B. Chandrasekaran,et al.  On Expediency and Convergence in Variable-Structure Automata , 1968, IEEE Trans. Syst. Sci. Cybern..

[3]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[4]  M. L. Tsetlin,et al.  Automaton theory and modeling of biological systems , 1973 .

[5]  Kumpati S. Narendra,et al.  Adaptation and learning in automatic systems , 1974 .

[6]  Kumpati S. Narendra,et al.  Learning Automata - A Survey , 1974, IEEE Trans. Syst. Man Cybern..

[7]  S. Lakshmivarahan,et al.  Learning Algorithms Theory and Applications , 1981 .

[8]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  M. A. L. THATHACHAR,et al.  A new approach to the design of reinforcement schemes for learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  B. Johnoommen.  Absorbing and Ergodic Discretized Two-Action Learning Automata , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  Mandayam A. L. Thathachar,et al.  Relaxation Labeling with Learning Automata , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  P. S. Sastry,et al.  Estimator Algorithms for Learning Automata , 1986 .

[13]  King-Sun Fu,et al.  Learning Control Systems-Review and Outlook , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[15]  C. Hwang,et al.  Diffusion for global optimization in R n , 1987 .

[16]  Mandayam A. L. Thathachar,et al.  Learning Optimal Discriminant Functions through a Cooperative Game of Automata , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[18]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[19]  B. John Oommen,et al.  Stochastic Automata Solutions to the Object Partitioning Problem , 1991, Comput. J..

[20]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[21]  B. John Oommen,et al.  Discretized estimator learning automata , 1992, IEEE Trans. Syst. Man Cybern..

[22]  P. Mars,et al.  Games of stochastic learning automata and adaptive signal processing , 1993, IEEE Trans. Syst. Man Cybern..

[23]  Kaddour Najim,et al.  Learning Automata: Theory and Applications , 1994 .

[24]  Georgios I. Papadimitriou.  A New Approach to the Design of Reinforcement Schemes for Learning Automata: Stochastic Estimator Learning Algorithms , 1994, IEEE Trans. Knowl. Data Eng..

[25]  V. V. Phansalkar,et al.  Decentralized Learning of Nash Equilibria in Multi-Person Stochastic Games With Incomplete Information , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[26]  P. S. Sastry,et al.  Continuous action set learning automata for stochastic optimization , 1994 .

[27]  Mandayam A. L. Thathachar,et al.  Local and Global Optimization Algorithms for Generalized Learning Automata , 1995, Neural Computation.

[28]  Q. H. Wu.  Learning coordinated control of power systems using interconnected learning automata , 1995 .

[29]  Mandayam A. L. Thathachar,et al.  Convergence of teams and hierarchies of learning automata in connectionist systems , 1995, IEEE Trans. Syst. Man Cybern..

[30]  Mandayam A. L. Thathachar,et al.  Learning the global maximum with parameterized learning automata , 1995, IEEE Trans. Neural Networks.

[31]  Georgios I. Papadimitriou,et al.  Learning automata-based receiver conflict avoidance algorithms for WDM broadcast-and-select star networks , 1996, TNET.

[32]  B. John Oommen,et al.  Graph Partitioning Using Learning Automata , 1996, IEEE Trans. Computers.

[33]  Kaddour Najim,et al.  Adaptive selection of the optimal order of linear regression models using learning automata , 1996, Int. J. Syst. Sci..

[34]  M T Arvind.  Stochastic Learning Algorithms With Improved Speed Performance , 1996 .

[35]  B. John Oommen,et al.  String taxonomy using learning automata , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[36]  Kaddour Najim,et al.  Learning automata and stochastic optimization , 1997 .

[37]  Mandayam A. L. Thathachar,et al.  Parallel algorithms for modules of learning automata , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[38]  Pushkin Kachroo,et al.  Multiple stochastic learning automata for vehicle path control in an automated highway system , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[39]  K. Rajaraman,et al.  Stochastic optimization over continuous and discrete variables with applications to concept learning under noise , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[40]  P. Shanti Sastry,et al.  New algorithms for learning and pruning oblique decision trees , 1999, IEEE Trans. Syst. Man Cybern. Part C.

[41]  M. Thathachar,et al.  Learning automata algorithms for pattern classification , 1999 .

[42]  C. Vasseur,et al.  A strategy for controlling nonlinear systems using a learning automaton , 2000, Autom..

[43]  Mohammad S. Obaidat,et al.  Fast learning automata for high-speed real-time applications , 2000, ICECS 2000. 7th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.00EX445).

[44]  Sudeep Sarkar,et al.  Modeling Parameter Space Behavior of Vision Systems Using Bayesian Networks , 2000, Comput. Vis. Image Underst..

[45]  Georgios I. Papadimitriou,et al.  On the use of stochastic estimator learning automata for dynamic channel allocation in broadcast networks , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[46]  Sudeep Sarkar,et al.  Supervised Learning of Large Perceptual Organization: Graph Spectral Partitioning and Learning Automata , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  P. S. Sastry,et al.  Adaptive Stochastic Algorithms for Pattern Classification , 2001 .

[48]  Mohammad S. Obaidat,et al.  An efficient adaptive bus arbitration scheme for scalable shared-medium ATM switch , 2001, Comput. Commun..

[49]  B. John Oommen,et al.  Continuous and discretized pursuit learning schemes: various algorithms and their comparison , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[50]  Sankar K. Pal,et al.  Pattern Recognition: From Classical to Modern Approaches , 2001 .

[51]  P. S. Nagendra Rao,et al.  A reinforcement learning approach to automatic generation control , 2002 .

[52]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.