An Experimental Comparison Between ATNoSFERES and ACS

After two papers comparing ATNoSFERES with XCSM, a Learning Classifier System with internal states, this paper is devoted to a comparison between ATNoSFERES and ACS (an Anticipatory Learning Classifier System). As previously, we focus on the way perceptual aliazing problems encountered in non-Markov environments are solved with both kinds of systems. We shortly present ATNoSFERES, a framework based on an indirect encoding Genetic Algorithm which builds finite-state automata controllers, and we compare it with ACS through two benchmark experiments. The comparison shows that the difference in performance between both system depends on the environment. This raises a discussion of the adequacy of both adaptive mechanisms to particular subclasses of non-Markov problems. Furthermore, since ACS converges much faster than ATNoSFERES, we discuss the need to introduce learning capabilities in our model. As a conclusion, we advocate for the need of more experimental comparisons between different systems in the Learning Classifier System community.

[1]  Shlomo Zilberstein,et al.  Finite-memory control of partially observable systems , 1998 .

[2]  John J. Grefenstette,et al.  Lamarckian Learning in Multi-Agent Environments , 1991, ICGA.

[3]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[4]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[5]  William A. Woods,et al.  Computational Linguistics Transition Network Grammars for Natural Language Analysis , 2022 .

[6]  Robert E. Smith,et al.  Memory Exploitation in Learning Classifier Systems , 1994, Evolutionary Computation.

[7]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[8]  Olivier Sigaud,et al.  YACS: a new learning classifier system using anticipation , 2002, Soft Comput..

[9]  George G. Robertson,et al.  A tale of two classifier systems , 1988, Machine Learning.

[10]  Olivier Sigaud,et al.  Further Comparison between ATNoSFERES and XCSM , 2002, IWLCS.

[11]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[12]  Dave Cliff,et al.  Adding Temporary Memory to ZCS , 1994, Adapt. Behav..

[13]  Martin V. Butz,et al.  Latent Learning and Action Planning in Robots with Anticipatory Classifier Systems , 1999, Learning Classifier Systems.

[14]  Pier Luca Lanzi.  An Analysis of the Memory Mechanism of XCSM , 2007 .

[15]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[16]  Stewart W. Wilson,et al.  Toward Optimal Classifier System Performance in Non-Markov Environments , 2000, Evolutionary Computation.

[17]  Olivier Sigaud,et al.  A Comparison Between ATNoSFERES And XCSM , 2002, GECCO.

[18]  Olivier Sigaud,et al.  Combining latent learning with dynamic programming in the modular anticipatory classifier system , 2005, Eur. J. Oper. Res..

[19]  Kee-Eung Kim,et al.  Learning Finite-State Controllers for Partially Observable Environments , 1999, UAI.

[20]  Long Lin,et al.  Memory Approaches to Reinforcement Learning in Non-Markovian Domains , 1992 .

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  Alexis Drogoul,et al.  ATNoSFERES : a Model for Evolutive Agent Behaviors , 2001 .

[23]  Claude Lattaud,et al.  Anticipatory Classifier System Using Behavioral Sequences in Non-Markov Environments , 2002, IWLCS.

[24]  Larry Bull,et al.  A Corporate XCS , 1999, Learning Classifier Systems.

[25]  Larry Bull,et al.  A zeroth level corporate classifier system , 1999 .

[26]  Jürgen Schmidhuber,et al.  HQ-Learning , 1997, Adapt. Behav..

[27]  Pier Luca Lanzi,et al.  Learning classifier systems from a reinforcement learning perspective , 2002, Soft Comput..