论文信息 - Adding memory to XCS

Adding memory to XCS

We add internal memory to XCS (eXtended Classifier System). We then test this version of XCS with internal memory, named XCSM, in non-Markovian environments with two and four aliasing states. The experimental results show that XCSM can easily converge to optimal solutions in simple environments; moreover, XCSM's performance is very stable with respect to the size of the internal memory involved in learning. However, the results we present evidence that in more complex non-Markovian environments, XCSM may fail to evolve an optimal solution. Our results suggest that this happens because the exploration strategies currently employed with XCS are not adequate to guarantee the convergence to an optimal policy with XCSM in complex non-Markovian environments.

Pier Luca Lanzi

[1] Chris Watkins,et al. Learning from delayed rewards , 1989 .

[2] Stewart W. Wilson. ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[3] Bernard Widrow,et al. Adaptive switching circuits , 1988 .

[4] Andrew W. Moore,et al. Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[5] Stewart W. Wilson. Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[6] Dave Cliff,et al. Adding Temporary Memory to ZCS , 1994, Adapt. Behav..

[7] Pier Luca Lanzi. Solving Problems in Partially Observable Environments with Classiier Systems (experiments on Adding Memory to Xcs) Solving Problems in Partially Observable Environments with Classiier Systems (experiments on Adding Memory to Xcs) , 2007 .

[8] Pier Luca Lanzi,et al. A Study of the Generalization Capabilities of XCS , 1997, ICGA.

[9] John H. Holland,et al. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .