Generating Exponentially Smaller POMDP Models Using Conditionally Irrelevant Variable Abstraction

The state of a POMDP can often be factored into a tuple of n state variables. The corresponding flat model, with size exponential in n, may be intractably large. We present a novel method called conditionally irrelevant variable abstraction (CIVA) for losslessly compressing the factored model, which is then expanded into an exponentially smaller flat model in a representation compatible with many existing POMDP solvers. We applied CIVA to previously intractable problems from a robotic exploration domain. We were able to abstract, expand, and approximately solve POMDPs that had up to 1024 states in the uncompressed flat representation.

[1]  Trey Smith,et al.  Probabilistic planning for robotic exploration , 2007 .

[2]  Reid G. Simmons,et al.  Point-Based POMDP Algorithms: Improved Analysis and Implementation , 2005, UAI.

[3]  Nikos A. Vlassis,et al.  Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[4]  Trey Smith,et al.  Life in the Atacama -- Year 2: Geologic Reconnaissance Through Long-Range Roving and Implications on the Search for Life , 2005 .

[5]  Joelle Pineau,et al.  POMDP Planning for Robust Robot Control , 2005, ISRR.

[6]  Craig Boutilier,et al.  VDCBPI: an Approximate Scalable Algorithm for Large POMDPs , 2004, NIPS.

[7]  Robert Givan,et al.  Equivalence notions and model minimization in Markov decision processes , 2003, Artif. Intell..

[8]  Tara Estlin,et al.  Rover traverse science for increased mission science return , 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652).

[9]  Joelle Pineau,et al.  Policy-contingent abstraction for robust robot control , 2002, UAI.

[10]  Zhengzhu Feng,et al.  Approximate Planning for Factored POMDPs , 2001 .

[11]  Carlos Guestrin,et al.  Multiagent Planning with Factored MDPs , 2001, NIPS.

[12]  Craig Boutilier,et al.  Stochastic dynamic programming with factored representations , 2000, Artif. Intell..

[13]  Zhengzhu Feng,et al.  Dynamic Programming for POMDPs Using a Factored State Representation , 2000, AIPS.

[14]  Jesse Hoey,et al.  APRICODD: Approximate Policy Construction Using Decision Diagrams , 2000, NIPS.

[15]  David A. McAllester,et al.  Approximate Planning for Factored POMDPs using Belief State Simplification , 1999, UAI.

[16]  Ann E. Nicholson,et al.  Dynamic Non-uniform Abstractions for Approximate Planning in Large Structured Stochastic Domains , 1998, PRICAI.

[17]  Craig Boutilier,et al.  Using Abstractions for Decision-Theoretic Planning with Time Constraints , 1994, AAAI.