Networks of Learning Automata: Techniques for Online Stochastic Optimization

1. Introduction.- 1.1 Machine Intelligence and Learning.- 1.2 Learning Automata.- 1.3 The Finite Action Learning Automaton (FALA).- 1.3.1 The Automaton.- 1.3.2 The Random Environment.- 1.3.3 Operation of FALA.- 1.4 Some Classical Learning Algorithms.- 1.4.1 Linear Reward-Inaction (LR?I) Algorithm.- 1.4.2 Other Linear Algorithms.- 1.4.3 Estimator Algorithms.- 1.4.4 Simulation Results.- 1.5 The Discretized Probability FALA.- 1.5.1 DLR?I Algorithm.- 1.5.2 Discretized Pursuit Algorithm.- 1.6 The Continuous Action Learning Automaton (CALA).- 1.6.1 Analysis of the Algorithm.- 1.6.2 Simulation Results.- 1.6.3 Another Continuous Action Automaton.- 1.7 The Generalized Learning Automaton (GLA).- 1.7.1 Learning Algorithm.- 1.7.2 An Example.- 1.8 The Parameterized Learning Automaton (PLA).- 1.8.1 Learning Algorithm.- 1.9 Multiautomata Systems.- 1.10 Supplementary Remarks.- 2. Games of Learning Automata.- 2.1 Introduction.- 2.2 A Multiple Payoff Stochastic Game of Automata.- 2.2.1 The Learning Algorithm.- 2.3 Analysis of the Automata Game Algorithm.- 2.3.1 Analysis of the Approximating ODE.- 2.4 Game with Common Payoff.- 2.5 Games of FALA.- 2.5.1 Common Payoff Games of FALA.- 2.5.2 Pursuit Algorithm for a Team of FALA.- 2.5.3 Other Types of Games.- 2.6 Common Payoff Games of CALA.- 2.6.1 Stochastic Approximation Algorithms and CALA.- 2.7 Applications.- 2.7.1 System Identification.- 2.7.2 Learning Conjunctive Concepts.- 2.8 Discussion.- 2.9 Supplementary Remarks.- 3. Feedforward Networks.- 3.1 Introduction.- 3.2 Networks of FALA.- 3.3 The Learning Model.- 3.3.1 G-Environment.- 3.3.2 The Network.- 3.3.3 Network Operation.- 3.4 The Learning Algorithm.- 3.5 Analysis.- 3.6 Extensions.- 3.6.1 Other Network Structures.- 3.6.2 Other Learning Algorithms.- 3.7 Convergence to the Global Maximum.- 3.7.1 The Network.- 3.7.2 The Global Learning Algorithm.- 3.7.3 Analysis of the Global Algorithm.- 3.8 Networks of GLA.- 3.9 Discussion.- 3.10 Supplementary Remarks.- 4. Learning Automata for Pattern Classification.- 4.1 Introduction.- 4.2 Pattern Recognition.- 4.3 Common Payoff Game of Automata for PR.- 4.3.1 Pattern Classification with FALA.- 4.3.2 Pattern Classification with CALA.- 4.3.3 Simulations.- 4.4 Automata Network for Pattern Recognition.- 4.4.1 Simulations.- 4.4.2 Network of Automata for Learning Global Maximum.- 4.5 Decision Tree Classifiers.- 4.5.1 Learning Decision Trees using GLA and CALA.- 4.5.2 Learning Piece-wise Linear Functions.- 4.6 Discussion.- 4.7 Supplementary Remarks.- 5. Parallel Operation of Learning Automata.- 5.1 Introduction.- 5.2 Parallel Operation of FALA.- 5.2.1 Analysis.- 5.2.2 ?-optimality.- 5.2.3 Speed of Convergence and Module Size.- 5.2.4 Simulation Studies.- 5.3 Parallel Operation of CALA.- 5.4 Parallel Pursuit Algorithm.- 5.4.1 Simulation Studies.- 5.5 General Procedure.- 5.6 Parallel Operation of Games of FALA.- 5.6.1 Analysis.- 5.6.2 Common Payoff Game.- 5.7 Parallel Operation of Networks of FALA.- 5.7.1 Analysis.- 5.7.2 Modules of Parameterized Learning Automata (PLA).- 5.7.3 Modules of Generalized Learning Automata (GLA).- 5.7.4 Pattern Classification Example.- 5.8 Discussion.- 5.9 Supplementary Remarks.- 6. Some Recent Applications.- 6.1 Introduction.- 6.2 Supervised Learning of Perceptual Organization in Computer Vision.- 6.3 Distributed Control of Broadcast Communication Networks.- 6.4O ther Applications.- 6.5 Discussion.- Epilogue.- Appendices.- A The ODE Approach to Analysis of Learning Algorithms.- A.I Introduction.- A.2 Derivation of the ODE Approximation.- A.2.1 Assumptions.- A.2.2 Analysis.- A.3 Approximating ODEs for Some Automata Algorithms.- A.3.2 The CALA Algorithm.- A.3.3 Automata Team Algorithms.- A.4 Relaxing the Assumptions.- B Proofs of Convergence for Pursuit Algorithm.- B.1 Proof of Theorem 1.1.- B.2 Proof of Theorem 5.7.- C Weak Convergence and SDE Approximations.- C.I Introduction.- C.2 Weak Convergence.- C.3 Convergence to SDE.- C.3.1 Application to Global Algorithms.- C.4 Convergence to ODE.- References.