论文信息 - Building Incomplete but Accurate Models

Building Incomplete but Accurate Models

We consider an agent that seeks to make abstract predictions about the world by only distinguishing between certain features of observations. Making accurate abstract predictions of this form may not require a fully detailed model of the world, though in general it will require that the agent make some finer distinctions than those that interest it. We assume the agent has a partition of the observation space of a dynamical system induced by the features of interest. The goal of this paper is to find a minimal refinement of that partition such that a model of the refined system will make accurate predictions with respect to the features of interest. We provide algorithms and worst-case bounds on the difficulty of performing this task. Our results apply generally to all discrete dynamical systems.

[1] Leslie Pack Kaelbling,et al. Acting Optimally in Partially Observable Stochastic Domains , 1994, AAAI.

[2] Andrew McCallum,et al. Reinforcement learning with selective perception and hidden state , 1996 .

[3] Jesse Hoey,et al. Solving POMDPs with Continuous or Large Discrete Observation Spaces , 2005, IJCAI.

[4] Richard S. Sutton,et al. Predictive Representations of State , 2001, NIPS.

[5] Michael R. James,et al. Predictive State Representations: A New Theory for Modeling Dynamical Systems , 2004, UAI.

[6] G. Monahan. State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[7] A. Cassandra,et al. Exact and approximate algorithms for partially observable markov decision processes , 1998 .

[8] Alicia P. Wolfe,et al. Decision Tree Methods for Finding Reusable MDP Homomorphisms , 2006, AAAI.

[9] Vishal Soni,et al. Relational Knowledge with Predictive State Representations , 2007, IJCAI.

[10] George E. Monahan,et al. A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 2007 .

[11] Vishal Soni,et al. Abstraction in Predictive State Representations , 2007, AAAI.

[12] A. Barto,et al. An algebraic approach to abstraction in reinforcement learning , 2004 .