Probabilistic inference in graphical models

1 INTRODUCTION A " graphical model " is a type of probabilistic network that has roots in several different framework provides a clean mathematical formalism that has made it possible to understand the relationships among a wide variety of network-based approaches to computation, and in particular to understand many neural network algorithms and architectures as instances of a broader probabilistic methodology. Graphical models use graphs to represent and manipulate joint probability distributions. The graph underlying a graphical model may be directed, in which case the model is often referred to as a belief network or a Bayesian network (see BAYESIAN NETWORKS), or the graph may be undirected, in which case the model is generally referred to as a Markov random field. A graphical model has both a structural component—encoded by the pattern of edges in the graph—and a parametric component—encoded by numerical " potentials " associated with sets of edges in the graph. The relationship between these components underlies the computational machinery associated with graphical models. In particular, general inference algorithms allow statistical quantities (such as likelihoods and conditional probabilities) and information-theoretic quantities (such as mutual information and conditional entropies) to be computed efficiently. These algorithms are the subject of the current article. Learning algorithms build on these inference algorithms and allow parameters and structures to be estimated from data (see GRAPHICAL MODELS, PARAMETER LEARNING and GRAPHICAL MODELS, STRUCTURE LEARNING).