The Factored Frontier Algorithm for Approximate Inference in DBNs

The Factored Frontier (FF) algorithm is a simple approximate inference algorithm for Dynamic Bayesian Networks (DBNs). It is very similar to the fully factorized version of the Boyen-Koller (BK) algorithm, but instead of doing an exact update at every step followed by marginalisation (projection), it always works with factored distributions. Hence it can be applied to models for which the exact update step is intractable. We show that FF is equivalent to (one iteration of) loopy belief propagation (LBP) on the original DBN, and that BK is equivalent (to one iteration of) LBP on a DBN where we cluster some of the nodes. We then show empirically that by iterating more than once, LBP can improve on the accuracy of both FF and BK. We compare these algorithms on two real-world DBNs: the first is a model of a water treatment plant, and the second is a coupled HMM, used to model freeway traffic.

[1]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[2]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[3]  Uue Kjjrull.  Triangulation of Graphs { Algorithms Giving Small Total State Space Triangulation of Graphs { Algorithms Giving Small Total State Space , 1990 .

[4]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[5]  Ross D. Shachter,et al.  Fusion and Propagation with Multiple Observations in Belief Networks , 1991, Artif. Intell..

[6]  Uffe Kjærulff,et al.  A Computational Scheme for Reasoning in Dynamic Probabilistic Networks , 1992, UAI.

[7]  Rina Dechter,et al.  Mini-Buckets: A General Scheme for Generating Approximations in Automated Reasoning , 1997, IJCAI.

[8]  Michael I. Jordan,et al.  Probabilistic Independence Networks for Hidden Markov Probability Models , 1997, Neural Computation.

[9]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[10]  Xavier Boyen,et al.  Tractable Inference for Complex Stochastic Processes , 1998, UAI.

[11]  Xavier Boyen,et al.  Approximate Learning of Dynamic Models , 1998, NIPS.

[12]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[13]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[14]  Michael I. Jordan,et al.  Probabilistic Networks and Expert Systems , 1999 .

[15]  B. Frey.  Turbo Factor Analysis , 1999 .

[16]  Yair Weiss,et al.  Correctness of Local Probability Propagation in Graphical Models with Loops , 2000, Neural Computation.

[17]  Jaimyoung Kwon.  Modeling Freeway Traffic with Coupled HMMs , 2000 .

[18]  William T. Freeman,et al.  On the fixed points of the max-product algorithm , 2000 .

[19]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[20]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[21]  Tom Minka,et al.  A family of algorithms for approximate Bayesian inference , 2001 .