A comparison between ATNoSFERES and Learning Classifier Systems on non-Markov problems

The automated design of the controller of software agents embedded in an environment is an important class of problems addressed in information sciences. In that class of problems, the case where agents face perceptual aliasing problems is particularly difficult. Within the evolutionary and adaptive approaches to controller design, there are several families of systems capable of dealing with such problems. This paper is devoted to a comparison of the way perceptual aliasing problems are solved by one family, namely Learning Classifier Systems, compared to our own model, ATNoSFERES. We present this model based on an indirect encoding Genetic Algorithm which builds Augmented Transition Network controllers, and we compare it with different Learning Classifier Systems, namely XCSM (a memory-based extension of the most studied Learning Classifier System, XCS) and ACS (an Anticipatory Learning Classifier System). To solve perceptual aliasing problems, the first uses an explicit internal state management mechanism while the second uses a rule-chaining mechanism. To carry out our comparison, we apply these systems to three different benchmark experiments. Our results raise a discussion of the respective properties of the mechanisms used by XCSM and ACS. Then we show that ATNoSFERES is endowed with enough expressive power to represent the mechanisms used by the other two systems to deal with perceptual aliasing problems. We conclude that ATNoSFERES provides a powerful framework to deal with such problems.

[1]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[2]  Olivier Sigaud,et al.  Contribution au problème de la sélection de l'action en environnement partiellement observable , 1999 .

[3]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[4]  Lawrence J. Fogel,et al.  Intelligence Through Simulated Evolution: Forty Years of Evolutionary Programming , 1999 .

[5]  Olivier Sigaud,et al.  A Comparison Between ATNoSFERES And XCSM , 2002, GECCO.

[6]  Olivier Sigaud,et al.  Combining latent learning with dynamic programming in the modular anticipatory classifier system , 2005, Eur. J. Oper. Res..

[7]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[8]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[9]  Stewart W. Wilson,et al.  Toward Optimal Classifier System Performance in Non-Markov Environments , 2000, Evolutionary Computation.

[10]  Conor Ryan,et al.  Grammatical Evolution: A Steady State approach , 2008 .

[11]  Ron Sun,et al.  Multi-Agent Reinforcement Learning with Bidding for Segmenting Action Sequences , 2000 .

[12]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[13]  Jean-Arcady Meyer,et al.  Evolution and development of neural controllers for locomotion, gradient-following, and obstacle-avoidance in artificial insects , 1998, IEEE Trans. Neural Networks.

[14]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[15]  Wolfgang Stolzmann,et al.  An Introduction to Anticipatory Classifier Systems , 1999, Learning Classifier Systems.

[16]  Pier Luca Lanzi,et al.  Learning classifier systems from a reinforcement learning perspective , 2002, Soft Comput..

[17]  Sébastien Picault,et al.  Stack-Based Gene Expression , 2002 .

[18]  Shlomo Zilberstein,et al.  Finite-memory control of partially observable systems , 1998 .

[19]  Riccardo Poli,et al.  Schema Theory for Genetic Programming with One-Point Crossover and Point Mutation , 1997, Evolutionary Computation.

[20]  Olivier Sigaud,et al.  An Experimental Comparison Between ATNoSFERES and ACS , 2005, IWLCS.

[21]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[22]  Kee-Eung Kim,et al.  Learning Finite-State Controllers for Partially Observable Environments , 1999, UAI.

[23]  Long Lin,et al.  Memory Approaches to Reinforcement Learning in Non-Markovian Domains , 1992 .

[24]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[25]  David E. Goldberg,et al.  A Critical Review of Classifier Systems , 1989, ICGA.

[26]  Jürgen Schmidhuber,et al.  HQ-Learning , 1997, Adapt. Behav..

[27]  John R. Koza.  Proceedings of the 1st annual conference on genetic programming , 1996 .

[28]  Olivier Sigaud,et al.  Further Comparison between ATNoSFERES and XCSM , 2002, IWLCS.

[29]  John J. Grefenstette,et al.  Proceedings of the 1st International Conference on Genetic Algorithms , 1985 .

[30]  Dario Floreano,et al.  From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior , 2000, Journal of Cognitive Neuroscience.

[31]  Robert E. Smith,et al.  Memory Exploitation in Learning Classifier Systems , 1994, Evolutionary Computation.

[32]  Olivier Sigaud,et al.  YACS: a new learning classifier system using anticipation , 2002, Soft Comput..

[33]  Alexis Drogoul,et al.  ATNoSFERES : a Model for Evolutive Agent Behaviors , 2001 .

[34]  Eugene Galanter,et al.  Handbook of mathematical psychology: I. , 1963 .

[35]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[36]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[37]  Stewart W. Wilson.  ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[38]  George G. Robertson,et al.  A tale of two classifier systems , 1988, Machine Learning.

[39]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[40]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[41]  A. Lindenmayer.  Mathematical models for cellular interactions in development. I. Filaments with one-sided inputs. , 1968, Journal of theoretical biology.

[42]  Larry Bull,et al.  A zeroth level corporate classifier system , 1999 .

[43]  Pier Luca Lanzi,et al.  A Roadmap to the Last Decade of Learning Classifier System Research , 1999, Learning Classifier Systems.

[44]  Huebner,et al.  Proceedings of the First Annual Conference of the Wharton School of Finance and Commerce , 2022 .

[45]  Sébastien Picault,et al.  Ethogenetics and the Evolutionary Design of Agents Behaviors , 2001 .

[46]  Claude Lattaud,et al.  Anticipatory Classifier System Using Behavioral Sequences in Non-Markov Environments , 2002, IWLCS.

[47]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[48]  Lashon B. Booker,et al.  Proceedings of the fourth international conference on Genetic algorithms , 1991 .

[49]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[50]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[51]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[52]  Jean-Arcady Meyer,et al.  Adaptive Behavior , 2005 .

[53]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[54]  George A. Miller,et al.  Introduction to the Formal Analysis of Natural Languages , 1968 .

[55]  Martin V. Butz,et al.  Anticipatory Learning Classifier Systems , 2002, Genetic Algorithms and Evolutionary Computation.

[56]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[57]  William A. Woods,et al.  Computational Linguistics Transition Network Grammars for Natural Language Analysis , 2022 .

[58]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[59]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[60]  Charles H. Moore,et al.  Forth - a language for interactive computing , 1970 .

[61]  Larry Bull,et al.  Learning Classifier Systems , 2002, Annual Conference on Genetic and Evolutionary Computation.

[62]  Lee Spector,et al.  Evolving Graphs and Networks with Edge Encoding: Preliminary Report , 1996 .

[63]  Stewart W. Wilson,et al.  Learning Classifier Systems, From Foundations to Applications , 2000 .

[64]  John J. Grefenstette,et al.  Lamarckian Learning in Multi-Agent Environments , 1991, ICGA.

[65]  John R. Koza,et al.  Automated Design of Both the Topology and Sizing of Analog Electrical Circuits Using Genetic Programming , 1996 .

[66]  Larry Bull,et al.  An accuracy based corporate classifier system , 2002, Soft Comput..

[67]  Dave Cliff,et al.  Adding Temporary Memory to ZCS , 1994, Adapt. Behav..

[68]  A. Lindenmayer.  Mathematical models for cellular interactions in development. II. Simple and branching filaments with two-sided inputs. , 1968, Journal of theoretical biology.

[69]  Pier Luca Lanzi.  An Analysis of the Memory Mechanism of XCSM , 2007 .

[70]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[71]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .