Predictive state representations (PSRs) use predictions of a set of tests to represent the state of controlled dynamical systems. One reason why this representation is exciting as an alternative to partially observable Markov decision processes (POMDPs) is that PSR models of dynamical systems may be much more compact than POMDP models. Empirical work on PSRs to date has focused on linear PSRs, which have not allowed for compression relative to POMDPs. We introduce a new notion of tests which allows us to define a new type of PSR that is nonlinear in general and allows for exponential compression in some deterministic dynamical systems. These new tests, called e-tests, are related to the tests used by Rivest and Schapire [1] in their work with the diversity representation, but our PSR avoids some of the pitfalls of their representation—in particular, its potential to be exponentially larger than the equivalent POMDP.
[1]
Richard S. Sutton,et al.
Predictive Representations of State
,
2001,
NIPS.
[2]
W. Lovejoy.
A survey of algorithmic methods for partially observed Markov decision processes
,
1991
.
[3]
Herbert Jaeger,et al.
Observable Operator Models for Discrete Stochastic Time Series
,
2000,
Neural Computation.
[4]
Michael L. Littman,et al.
Algorithms for Sequential Decision Making
,
1996
.
[5]
Ronald L. Rivest,et al.
Diversity-based inference of finite automata
,
1994,
28th Annual Symposium on Foundations of Computer Science (sfcs 1987).
[6]
Peter Stone,et al.
Learning Predictive State Representations
,
2003,
ICML.