YACS: Combining Dynamic Programming with Generalization in Classifier Systems

This paper describes our work on the use of anticipation in Learning Classifier Systems (LCS) applied to Markov problems. We present YACS1, a new kind of Anticipatory Classifier System. It calls upon classifiers with a [Condition], an [Action] and an [Effect] part. As in the traditional LCS framework, the classifier discovery process relies on a selection and a creation mechanism. As in the Anticipatory Classifier System (ACS), YACS looks for classifiers which anticipate well rather than for classifiers which propose an optimal action. The creation mechanism does not rely on classical genetic operators but on a specialization operator, which is explicitly driven by experience. Likewise, the action qualities of the classifiers are not computed by a classical bucket-brigade algorithm, but by a variety of the value iteration algorithm that takes advantage of the effect part of the classifiers. This paper presents the latent learning process of YACS. The description of the reinforcement learning process is focussed on the problem induced by the joint use of generalization and dynamic programming methods.