Recent advances in Learning Automata systems

This paper presents an overview of the field of Stochastic Learning Automata (LA), and concentrates, in particular, on the most recent advances in the field. LA, which have been studied for more than three decades, are probabilistic finite state machines that can model how biological systems can learn. The structure of such a machine can be fixed, or it can be changing with time. A LA can also be implemented by using action probability updating rules which may or may not depend on estimates from the Environment being investigated. While, traditionally, these updating rules have worked with the continuous probability space, we1 will explain how LA can be designed by discretizing the probability space. The paper2 will describe the design and analysis of both continuous and discretized LA, and will highlight the subtle differences between the corresponding learning machines, their convergence properties, and their learning capabilities. The paper will then discuss the most recent developments such as the Generalized Thathachar-Sastry estimator scheme. The paper also includes a comprehensive list of the applications in which LA have proven their powerful potential.

[1]  P. S. Sastry,et al.  Estimator Algorithms for Learning Automata , 1986 .

[2]  B.J. Oommen,et al.  Parameter learning from stochastic teachers and stochastic compulsive liars , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Harley Bornbach,et al.  An introduction to mathematical learning theory , 1967 .

[4]  Peter J. Diggle,et al.  SPLANCS: spatial point pattern analysis code in S-Plus , 1993 .

[5]  Mohammad S. Obaidat,et al.  Learning automata-based bus arbitration for shared-medium ATM switches , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Dimitris Bertsimas,et al.  Stochastic and Dynamic Vehicle Routing in the Euclidean Plane with Multiple Capacitated Vehicles , 1993, Oper. Res..

[7]  M. L. Tsetlin,et al.  Automaton theory and modeling of biological systems , 1973 .

[8]  Athanasios V. Vasilakos,et al.  Ergodic discretized estimator learning automata with high accuracy and high adaptation rate for nonstationary environments , 1992, Neurocomputing.

[9]  Athanasios V. Vasilakos,et al.  The Use of Reinforcement Learning Algorithms in Traffic Control of High Speed Networks , 2002, Advances in Computational Intelligence and Learning.

[10]  B. John Oommen,et al.  Discretized estimator learning automata , 1992, IEEE Trans. Syst. Man Cybern..

[11]  Joseph P. Cusumano,et al.  A stochastic interrogation method for experimental measurements of global dynamics and basin evolution: Application to a two-well oscillator , 1995 .

[12]  M. Thathachar,et al.  Networks of Learning Automata: Techniques for Online Stochastic Optimization , 2003 .

[13]  Mohammad S. Obaidat,et al.  Guest editorial learning automata: theory, paradigms, and applications , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[14]  Athanasios V. Vasilakos,et al.  Ergodic discretized estimator learning automata with high accuracy and high adaptation rate for nonstationary environments , 1990, [1990] Proceedings of the 2nd International IEEE Conference on Tools for Artificial Intelligence.

[15]  Adrian Baddeley,et al.  spatstat: An R Package for Analyzing Spatial Point Patterns , 2005 .

[16]  B. John Oommen,et al.  The asymptotic optimality of discretized linear reward-inaction learning automata , 1984, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  Carson C. Chow,et al.  Aperiodic stochastic resonance in excitable systems. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[18]  Mohammad Reza Meybodi,et al.  New Learning Automata Based Algorithms for Adaptation of Backpropagation Algorithm Parameters , 2002, Int. J. Neural Syst..

[19]  B. John Oommen,et al.  Continuous Learning Automata Solutions to the Capacity Assignment Problem , 2000, IEEE Trans. Computers.

[20]  Athanasios V. Vasilakos,et al.  The use of learning algorithms in ATM networks call admission control problem: a methodology , 1995, Proceedings of 20th Conference on Local Computer Networks.

[21]  Kaddour Najim,et al.  Learning Automata: Theory and Applications , 1994 .

[22]  Witold Pedrycz,et al.  Optimizing QoS routing in hierarchical ATM networks using computational intelligence techniques , 2003, IEEE Trans. Syst. Man Cybern. Part C.

[23]  Athanasios V. Vasilakos,et al.  On the use of a stochastic estimator learning algorithm to the ATM routing problem: a methodology , 1998, Comput. Commun..

[24]  George E. Ferris,et al.  An Introduction to Mathematical Learning Theory , 1966 .

[25]  Georgios I. Papadimitriou,et al.  Learning-automata-based TDMA protocols for broadcast communication systems with bursty traffic , 2000, IEEE Communications Letters.

[26]  Hannes Hartenstein,et al.  Stochastic Properties of the Random Waypoint Mobility Model , 2004, Wirel. Networks.

[27]  B. John Oommen,et al.  GPSPA: a new adaptive algorithm for maintaining shortest path routing trees in stochastic networks , 2004, Int. J. Commun. Syst..

[28]  M. D. Paola,et al.  Digital simulation of wind field velocity , 1998 .

[29]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[30]  B. John Oommen,et al.  Discretized pursuit learning automata , 1990, IEEE Trans. Syst. Man Cybern..

[31]  Dirk Ifenthaler,et al.  Stochastic Models of Learning , 2012 .

[32]  Franciszek Seredynski,et al.  Distributed scheduling using simple learning machines , 1998, Eur. J. Oper. Res..

[33]  Athanasios V. Vasilakos,et al.  LB-SELA: Rate-Based Access Control for ATM Networks , 1998, Comput. Networks.

[34]  David B. Cooper,et al.  Automatic Finding of Main Roads in Aerial Images by Using Geometric-Stochastic Models and Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  B. John Oommen,et al.  Generalized pursuit learning schemes: new families of continuous and discretized learning automata , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[36]  P. S. Sastry,et al.  Continuous action set learning automata for stochastic optimization , 1994 .

[37]  B. John Oommen,et al.  Graph Partitioning Using Learning Automata , 1996, IEEE Trans. Computers.

[38]  B. John Oommen,et al.  Continuous and discretized pursuit learning schemes: various algorithms and their comparison , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[39]  S. Lakshmivarahan,et al.  Learning Algorithms Theory and Applications , 1981 .

[40]  M. L. Tsetlin.  On the Behavior of Finite Automata in Random Media , 1961 .

[41]  Pushkin Kachroo,et al.  Simulation study of multiple intelligent vehicle control using stochastic learning automata , 1997 .

[42]  Mohammad Reza Meybodi,et al.  Applying continuous action reinforcement learning automata(CARLA) to global training of hidden Markov models , 2004, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004..

[43]  B. John Oommen,et al.  Discretized learning automata solutions to the capacity assignment problem for prioritized networks , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[44]  M. Brandeau,et al.  An overview of representative problems in location research , 1989 .

[45]  B. John Oommen,et al.  Absorbing and Ergodic Discretized Two-Action Learning Automata , 1986, IEEE Trans. Syst. Man Cybern..

[46]  Robert L. Cook,et al.  Stochastic sampling in computer graphics , 1988, TOGS.

[47]  Kaddour Najim,et al.  Learning automata and stochastic optimization , 1997 .