Stratified exponential families: Graphical models and model selection

We describe a hierarchy of exponential families which is useful for distinguishing types of graphical models. Undirected graphical models with no hidden variables are linear exponential families (LEFs). Directed acyclic graphical (DAG) models and chain graphs with no hidden variables, including DAG models with several families of local distributions, are curved exponential families (CEFs). Graphical models with hidden variables are what we term stratified exponential families (SEFs). A SEF is a finite union of CEFs of various dimensions satisfying some regularity conditions. We also show that this hierarchy of exponential families is noncollapsing with respect to graphical models by providing a graphical model which is a CEF but not a LEF and a graphical model that is a SEF but not a CEF. Finally, we show how to compute the dimension of a stratified exponential family. These results are discussed in the context of model selection of graphical models.

[1]  J. Munkres,et al.  Calculus on Manifolds , 1965 .

[2]  O. Barndorff-Nielsen.  Information And Exponential Families , 1970 .

[3]  L. A. Goodman.  Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[4]  B. Efron.  THE GEOMETRY OF EXPONENTIAL FAMILIES , 1978 .

[5]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[6]  J. V. Santen,et al.  How many parameters can a model have and still be testable , 1985 .

[7]  D. Sattinger,et al.  Calculus on Manifolds , 1986 .

[8]  Max Henrion,et al.  Some Practical Issues in Constructing Belief Networks , 1987, UAI.

[9]  D. Haughton.  On the Choice of a Model to Fit Data from an Exponential Family , 1988 .

[10]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[11]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[12]  Steen Andreassen,et al.  A munin network for the median nerve - a case study on loops , 1989, Appl. Artif. Intell..

[13]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[14]  J. Risler,et al.  Real algebraic and semi-algebraic sets , 1990 .

[15]  N L Harris.  Probabilistic belief networks for genetic counseling. , 1990, Computer methods and programs in biomedicine.

[16]  D. Heckerman,et al.  ,81. Introduction , 2022 .

[17]  Selman Akbulut,et al.  Topology of Real Algebraic Sets , 1991 .

[18]  W. Bruce Croft,et al.  Evaluation of an inference network-based retrieval model , 1991, TOIS.

[19]  David J. Spiegelhalter,et al.  Bayesian networks for patient monitoring , 1992, Artif. Intell. Medicine.

[20]  Kim L. Boyer,et al.  Integration, Inference, and Management of Spatial Information Using Bayesian Networks: Perceptual Organization , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Valmir Carneiro Barbosa,et al.  A Bayesian-Network Approach to Lexical Disambiguation , 1993, Cogn. Sci..

[22]  David Heckerman,et al.  Learning Gaussian Networks , 1994, UAI.

[23]  David Heckerman,et al.  Decision-theoretic troubleshooting , 1995, CACM.

[24]  Robert M. Fung,et al.  Applying Bayesian networks to information retrieval , 1995, CACM.

[25]  David Heckerman,et al.  Asymptotic Model Selection for Directed Networks with Hidden Variables , 1996, UAI.

[26]  David J. C. MacKay,et al.  Bayesian neural network model for austenite formation in steels , 1996 .

[27]  A. H. Murphy,et al.  Hailfinder: A Bayesian system for forecasting severe weather , 1996 .

[28]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[29]  David Madigan,et al.  An Alternative Markov Property for Chain Graphs , 1996, UAI.

[30]  V. F Kumar,et al.  Image Interpretation Using Bayesian Networks , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  David Heckerman,et al.  Causal independence for probability assessment and inference using Bayesian networks , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[32]  R. Kass,et al.  Geometrical Foundations of Asymptotic Inference , 1997 .

[33]  Christopher Meek,et al.  The dimensionality of mixed ancestral graphs , 1997 .

[34]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[35]  David Heckerman,et al.  Structure and Parameter Learning for Causal Independence and Causal Interaction Models , 1997, UAI.

[36]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[37]  Adrian E. Raftery,et al.  How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis , 1998, Comput. J..

[38]  Jim Q. Smith,et al.  On the Geometry of Bayesian Graphical Models with Hidden Variables , 1998, UAI.

[39]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[40]  Dan Geiger,et al.  Graphical Models and Exponential Families , 1998, UAI.

[41]  Michael I. Jordan.  Graphical Models , 2003 .

[42]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[43]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[44]  Ted Chang.  Geometrical foundations of asymptotic inference , 2002 .

[45]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.