An Uncertainty-Based Belief Selection Method for POMDP Value Iteration

Partially Observable Markov Decision Process (POMDP) provides a probabilistic model for decision making under uncertainty. Point-based value iteration algorithms are effective approximate algorithms to solve POMDP problems. Belief selection is a key step of point-based algorithm. In this paper we provide a belief selection method based on the uncertainty of belief point. The algorithm first computes the uncertainties of the belief points that could be reached, and then selects the belief points that have lower uncertainties and whose distances to the current belief set are larger than a threshold. The experimental results indicate that this method is effective to gain an approximate long-term discounted reward using fewer belief states than the other point-based algorithms.