Approximate predictive state representations

Predictive state representations (PSRs) are models that represent the state of a dynamical system as a set of predictions about future events. The existing work with PSRs focuses on trying to learn exact models, an approach that cannot scale to complex dynamical systems. In contrast, our work takes the first steps in developing a theory of approximate PSRs. We examine the consequences of using an approximate predictive state representation, bounding the error of the approximate state under certain conditions. We also introduce factored PSRs, a class of PSRs with a particular approximate state representation. We show that the class of factored PSRs allow one to tune the degree of approximation by trading off accuracy for compactness. We demonstrate this trade-off empirically on some example systems, using factored PSRs that were learned from data.

[1]  Andrew McCallum,et al.  A comparison of event models for naive bayes text classification , 1998, AAAI 1998.

[2]  Xavier Boyen,et al.  Tractable Inference for Complex Stochastic Processes , 1998, UAI.

[3]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[4]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[5]  R. Sandberg,et al.  Capturing whole-genome characteristics in short sequences using a naïve Bayesian classifier. , 2001, Genome research.

[6]  Daphne Koller,et al.  Multi-Agent Influence Diagrams for Representing and Solving Games , 2001, IJCAI.

[7]  Michael R. James,et al.  Learning and discovery of predictive state representations in dynamical systems with reset , 2004, ICML.

[8]  Michael R. James,et al.  Predictive State Representations: A New Theory for Modeling Dynamical Systems , 2004, UAI.

[9]  Edmund H. Durfee,et al.  Graphical models in local, asymmetric multi-agent Markov decision processes , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[10]  Michael R. James,et al.  Learning predictive state representations in dynamical systems without reset , 2005, ICML.

[11]  Peter Wagner,et al.  Validating microscopic traffic flow models , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[12]  Michael I. Jordan,et al.  A graphical model for predicting protein molecular function , 2006, ICML '06.

[13]  Benjamin Coifman,et al.  Highway Traffic Data Sensitivity Analysis , 2007 .

[14]  Vishal Soni,et al.  Relational Knowledge with Predictive State Representations , 2007, IJCAI.