Extending Factor Graphs so as to Unify Directed and Undirected Graphical Models

The two most popular types of graphical model are Bayesian networks (BNs) and Markov random fields (MRFs). These types of model offer complementary properties in model construction, expressing conditional independencies, expressing arbitrary factorizations of joint distributions, and formulating messagepassing inference algorithms. We show how the notation and semantics of factor graphs (a relatively new type of graphical model) can be extended so as to combine the strengths of BNs and MRFs. Every BN or MRF can be easily converted to a factor graph that expresses the same conditional independencies, expresses the same factorization of the joint distribution, and can be used for probabilistic inference through application of a single, simple message-passing algorithm. We describe a modified "Bayes-ball" algorithm for establishing conditional independence in factor graphs, and we show that factor graphs form a strict superset of BNs and MRFs. In particular, we give an example of a commonly-used model fragment, whose independencies cannot be represented in a BN or an MRF, but can be represented in a factor graph. For readers who use chain graphs, we describe a further extension of factor graphs that enables them to represent properties of chain graphs.