论文信息 - United We Stand: Population Based Methods for Solving Unknown POMDPs

United We Stand: Population Based Methods for Solving Unknown POMDPs

Solving large unknown POMDPs is an open research problem. Policy search is one solution method that is attractive as it scales in the size of the policy, which is typically much simpler than the environment. We present a global search algorithm capable of finding good policies for POMDPs that are substantially larger than previously reported results. Our algorithm is general; we show it can be used with, and improves the performance of, existing local search techniques such as gradient ascent. Sharing information between the members of the population is the key to our algorithm and we show it results in better performance than equivalent parallel searches that do not share information. Unlike previous work our algorithm does not require the size of the policy to be known in advance.

Noel Welsh | Jeremy Wyatt | J. Wyatt | Noel Welsh

[1] Joelle Pineau,et al. Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[2] Andrew W. Moore,et al. Direct Policy Search using Paired Statistical Tests , 2001, ICML.

[3] C. D. Gelatt,et al. Optimization by Simulated Annealing , 1983, Science.

[4] Pat Langley,et al. Editorial: On Machine Learning , 1986, Machine Learning.

[5] Andrew McCallum,et al. Reinforcement learning with selective perception and hidden state , 1996 .

[6] Douglas Aberdeen,et al. Policy-Gradient Algorithms for Partially Observable Markov Decision Processes , 2003 .

[7] Carl E. Rasmussen,et al. Factorial Hidden Markov Models , 1997 .

[8] Michael L. Littman,et al. Algorithms for Sequential Decision Making , 1996 .

[9] Katia P. Sycara,et al. Evolutionary Search, Stochastic Policies with Memory, and Reinforcement Learning with Hidden State , 2001, ICML.

[10] Leslie Pack Kaelbling,et al. Learning Policies with External Memory , 1999, ICML.