Learning the Structure of Probabilistic Graphical Models with an Extended Cascading Indian Buffet Process

This paper presents an extension of the cascading Indian buffet process (CIBP) intended to learning arbitrary directed acyclic graph structures as opposed to the CIBP, which is limited to purely layered structures. The extended cascading Indian buffet process (eCIBP) essentially consists in adding an extra sampling step to the CIBP to generate connections between nonconsecutive layers. In the context of graphical model structure learning, the proposed approach allows learning structures having an unbounded number of hidden random variables and automatically selecting the model complexity. We evaluated the extended process on multivariate density estimation and structure identification tasks by measuring the structure complexity and predictive performance. The results suggest the extension leads to extracting simpler graphs without scarifying predictive precision.

[1]  Thomas L. Griffiths,et al.  A Non-Parametric Bayesian Method for Inferring Hidden Causes , 2006, UAI.

[2]  Thomas L. Griffiths,et al.  The Indian Buffet Process: An Introduction and Review , 2011, J. Mach. Learn. Res..

[3]  J. Pitman.  Exchangeable and partially exchangeable random partitions , 1995 .

[4]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .

[5]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[6]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[7]  Nir Friedman,et al.  Being Bayesian about Network Structure , 2000, UAI.

[8]  Fernando Pérez-Cruz,et al.  Kullback-Leibler divergence estimation of continuous distributions , 2008, 2008 IEEE International Symposium on Information Theory.

[9]  N. Hjort.  Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[10]  Michael I. Jordan.  Hierarchical Models , Nested Models and Completely Random Measures , 2010 .

[11]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[12]  P. Müller,et al.  Approximatereference priors in the presence of latent structures , 2010 .

[13]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[14]  Michael I. Jordan,et al.  Hierarchical Beta Processes and the Indian Buffet Process , 2007, AISTATS.

[15]  Lawrence Carin,et al.  A Stick-Breaking Construction of the Beta Process , 2010, ICML.

[16]  Lawrence Carin,et al.  Nonparametric factor analysis with beta process priors , 2009, ICML '09.

[17]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[18]  Etienne Barnard,et al.  Maximum leave-one-out likelihood for kernel density estimation , 2010 .

[19]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[20]  Ryan P. Adams,et al.  Learning the Structure of Deep Sparse Graphical Models , 2009, AISTATS.

[21]  David B. Dunson,et al.  The Hierarchical Beta Process for Convolutional Factor Analysis and Deep Learning , 2011, ICML.

[22]  Dilan Görür,et al.  Dirichlet process Gaussian mixture models: choice of the base distribution , 2010 .

[23]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[24]  Michael I. Jordan.  Graphical Models , 2003 .

[25]  Dipak K. Dey,et al.  Frontiers of statistical decision making and Bayesian analysis : in honor of James O. Berger , 2010 .

[26]  Jessika Weiss,et al.  Graphical Models In Applied Multivariate Statistics , 2016 .

[27]  Brendan J. Frey,et al.  Continuous Sigmoidal Belief Networks Trained using Slice Sampling , 1996, NIPS.

[28]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[29]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[30]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.