A conditional independence algorithm for learning undirected graphical models

When it comes to learning graphical models from data, approaches based on conditional independence tests are among the most popular methods. Since Bayesian networks dominate research in this field, these methods usually refer to directed graphs, and thus have to determine not only the set of edges, but also their direction. At least for a certain kind of possibilistic graphical models, however, undirected graphs are a much more natural basis. Hence, in this area, algorithms for learning undirected graphs are desirable, especially, since first learning a directed graph and then transforming it into an undirected one wastes resources and computation time. In this paper I present a general algorithm for learning undirected graphical models, which is strongly inspired by the well-known Cheng-Bell-Liu algorithm for learning Bayesian networks from data. Its main advantage is that it needs fewer conditional independence tests, while it achieves results of comparable quality.

[1]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[2]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[3]  David A. Bell,et al.  Learning Bayesian networks from data: An information-theory based approach , 2002, Artif. Intell..

[4]  Moninder Singh,et al.  An Algorithm for the Construction of Bayesian Network Structures from Data , 1993, UAI.

[5]  Harald Steck,et al.  Constraint-based structural learning in Bayesian networks using finite data sets , 2001 .

[6]  Franz Pernkopf,et al.  Detection of surface defects on raw steel blocks using Bayesian network classifiers , 2004, Pattern Analysis and Applications.

[7]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[8]  Michael I. Jordan.  Graphical Models , 2003 .

[9]  Wray L. Buntine.  Operations for Learning with Graphical Models , 1994, J. Artif. Intell. Res..

[10]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[11]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[12]  Henry Tirri,et al.  On Discriminative Bayesian Network Classifiers and Logistic Regression , 2005, Machine Learning.

[13]  Zwi Altman,et al.  Automated Diagnosis for UMTS Networks Using Bayesian Network Approach , 2008, IEEE Transactions on Vehicular Technology.

[14]  Peter J. F. Lucas,et al.  A dynamic Bayesian network for diagnosing ventilator-associated pneumonia in ICU patients , 2009, Expert Syst. Appl..

[15]  José Manuel Gutiérrez,et al.  Expert Systems and Probabiistic Network Models , 1996 .

[16]  Christian Borgelt,et al.  Graphical models - methods for data analysis and mining , 2002 .

[17]  Christian Borgelt,et al.  Efficient maximum projection of database-induced multivariate possibility distributions , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[18]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[19]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[20]  S. Moral,et al.  Independence in uncertainty theories and its applications to learning belief networks , 2000 .

[21]  Simon King,et al.  Articulatory feature recognition using dynamic Bayesian networks , 2007, Comput. Speech Lang..

[22]  Tom M. Mitchell,et al.  Bayesian Network Learning with Parameter Constraints , 2006, J. Mach. Learn. Res..

[23]  L. Williams,et al.  Contents , 2020, Ophthalmology (Rochester, Minn.).

[24]  Pedro M. Domingos,et al.  Learning Bayesian network classifiers by maximizing conditional likelihood , 2004, ICML.

[25]  Jörg Gebhardt,et al.  Learning from data: possibilistic graphical models , 2000 .

[26]  Satoru Miyano,et al.  Dynamic Bayesian network and nonparametric regression for nonlinear modeling of gene networks from time series gene expression data. , 2004 .

[27]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[28]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[29]  Gladys Castillo,et al.  Adaptive learning algorithms for Bayesian network classifiers , 2008, AI Commun..

[30]  Sung-Jung Cho,et al.  Bayesian network modeling of strokes and their relationships for on-line handwriting recognition , 2004, Pattern Recognit..

[31]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[32]  C. Borgelt,et al.  Evaluation measures for learning probabilistic and possibilistic networks , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[33]  José A. Gámez,et al.  Advances in Bayesian networks , 2004 .

[34]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[35]  María S. Pérez-Hernández,et al.  Bayesian network multi-classifiers for protein secondary structure prediction , 2004, Artif. Intell. Medicine.

[36]  Henry Schneiderman,et al.  Learning a restricted Bayesian network for object detection , 2004, CVPR 2004.