Optimal Value of Information in Dynamic Bayesian Networks

Decision-making based on probabilistic reasoning often involves selecting a subset of expensive observations, that best predict the system state. Krause and Guestrin described two problems of non-myopically selecting observations in graphical models to optimize the value of information (VoI), namely, selection of an optimal subset of observations, and generation of an optimal conditional observation plan. They showed that these problems are intractable in general, but gave polynomial-time dynamic programming algorithms, called VoIDP, for chain graphical models. In this paper, we consider the general setting of Dynamic Bayesian Networks (DBNs), and formulate these problems in terms of finding optimal policies in Markov Decision Processes (MDPs). The time complexities of the resulting algorithms are exponential in general, but polynomial for chain models. Given a chain model, our algorithms compute the same subset, or plan, as VoIDP. Interestingly, despite their generality, our algorithms have significantly better time complexities for chain models compared to VoIDP. We also present an outline of how to use our framework to formulate an approximate, nonmyopic VoI optimization technique, with absolute a posteriori guarantees on approximation, that can handle arbitrary DBNs efficiently.