State space construction of reinforcement learning agents based upon anticipated sensory changes

We propose herein a new incremental state construction method which consists of Fritzke's growing neural gas algorithm and a class management mechanism of GNG units. The GNG algorithm condenses sensory inputs and learns which areas are frequently sensed. The CMM yields a new state based upon the anticipated behaviors of the agent, i.e., a couple of actions by an agent and the resultant change in sensory inputs. Computational simulations on the mountain-car task confirm the effectiveness of the proposed method.

[1]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[4]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.