Semantically Interpretable Predictive State Representation

Predictive State Representations (PSRs) allow modeling of dynamical systems directly in observables and without relying on latent variable representations. However, it is often hard to attribute semantic meaning to PSR representations. In this paper, we present the idea of introducing prior information to PSR learning (P-PSRs) in order to learn representations which are more suitable for generalization, planning, and interpretation. For this, we learn an embedding of test features such that belief points of similar semantic share the same region of a subspace. The resulting spacial relationship facilitates generalization and semantical interpretation of the representation. We demonstrate how our approach can handle biased training data and allows feature selection such that the resulting representation emphasizes observables that relate to the planning task. We show that our P-PSRs result in qualitatively meaningful representations and present quantitative results that indicate improved suitability for planning.