论文信息 - Learning in non-stationary Partially Observable Markov Decision Processes

Learning in non-stationary Partially Observable Markov Decision Processes

We study the problem of finding an optimal policy for a Partially Observable Markov Decision Process (POMDP) when the model is not perfectly known and may change over time. We present the algorithm MEDUSA+, which incrementally improves a POMDP model using selected queries, while still optimizing the reward. Empirical results show the response of the algorithm to changes in the parameters of a model: the changes are learned quickly and the agent still accumulates high reward throughout the process.

Doina Precup | R. Jaulmes

[1] Lonnie Chrisman,et al. Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach , 1992, AAAI.

[2] David A. Cohn,et al. Active Learning with Statistical Models , 1996, NIPS.

[3] Leslie Pack Kaelbling,et al. Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[4] Andrew McCallum,et al. Reinforcement learning with selective perception and hidden state , 1996 .

[5] Leslie Pack Kaelbling,et al. Learning Topological Maps with Weak Local Odometric Information , 1997, IJCAI.

[6] Leslie Pack Kaelbling,et al. Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[7] David Andre,et al. Model based Bayesian Exploration , 1999, UAI.

[8] Joelle Pineau,et al. Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[9] Peter Stone,et al. Learning Predictive State Representations , 2003, ICML.

[10] Nikos A. Vlassis,et al. Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[11] Joelle Pineau,et al. Active Learning in Partially Observable Markov Decision Processes , 2005, ECML.