论文信息 - The Thing We Tried That Worked : Utile Distinctions for Relational Reinforcement Learning

The Thing We Tried That Worked : Utile Distinctions for Relational Reinforcement Learning

This paper introduces a relational function approximation technique based on McCallum’s UTree algorithm (McCallum, 1995). We have extended the original approach to handle relational observations using an attribute graph of observable objects and relationships (McGovern et al., 2003). Furthermore, we address the inherent challenges that arise with a relational representation. We use stochastic sampling to manage the search space (Srinivasan, 1999), and sampling to address issues of autocorrelation (Jensen and Neville, 2002). We prevent the algorithm from growing an overly large and complex tree by incorporating Iterative Tree Induction’s approach (Utgoff, 1995). We compare Relational UTree’s performance with similar relational learning methods (Finney et al., 2002) (Driessens et al., 2001).

Will Dabney | A. McGovern

[1] J. Schmee. Applied Statistics—A Handbook of Techniques , 1984 .

[2] Andrew W. Moore,et al. Memory-Based Reinforcement Learning: Efficient Computation with Prioritized Sweeping , 1992, NIPS.

[3] Andrew McCallum,et al. Reinforcement learning with selective perception and hidden state , 1996 .

[4] Leslie Pack Kaelbling,et al. Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[5] Kurt Driessens,et al. Speeding Up Relational Reinforcement Learning through the Use of an Incremental First Order Decision Tree Learner , 2001, ECML.

[6] Tim Oates,et al. The Thing that we Tried Didn't Work very Well: Deictic Representation in Reinforcement Learning , 2002, UAI.

[7] Jennifer Neville,et al. Linkage and Autocorrelation Cause Feature Selection Bias in Relational Learning , 2002, ICML.

[8] David D. Jensen,et al. Exploiting relational structure to understand publication patterns in high-energy physics , 2003, SKDD.

[9] Paul E. Utgoff,et al. Decision Tree Induction Based on Efficient Tree Restructuring , 1997, Machine Learning.

[10] Richard S. Sutton,et al. Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11] De,et al. Relational Reinforcement Learning , 2001, Encyclopedia of Machine Learning and Data Mining.