Discriminative Scoring of Bayesian Network Classifiers: a Comparative Study

We consider the problem of scoring Bayesian Network Classifiers (BNCs) on the basis of the conditional loglikelihood (CLL). Currently, optimization is usually performed in BN parameter space, but for perfect graphs (such as Naive Bayes, TANs and FANs) a mapping to an equivalent Logistic Regression (LR) model is possible, and optimization can be performed in LR parameter space. We perform an empirical comparison of the efficiency of scoring in BN parameter space, and in LR parameter space using two different mappings. For each parameterization, we study two popular optimization methods: conjugate gradient, and BFGS. Efficiency of scoring is compared on simulated data and data sets from the UCI Machine Learning repository.

[1]  Bin Shen,et al.  Structural Extension to Logistic Regression: Discriminative Parameter Learning of Belief Net Classifiers , 2002, Machine Learning.

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  J. A. Anderson,et al.  7 Logistic discrimination , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[4]  Pedro Larrañaga,et al.  Discriminative Learning of Bayesian Network Classifiers via the TM Algorithm , 2005, ECSQARU.

[5]  Thomas P. Minka,et al.  Algorithms for maximum-likelihood logistic regression , 2003 .

[6]  W. Press,et al.  Numerical Recipes in Fortran: The Art of Scientific Computing.@@@Numerical Recipes in C: The Art of Scientific Computing. , 1994 .

[7]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields for Relational Learning , 2007 .

[8]  J. Nash.  Compact Numerical Methods for Computers , 2018 .

[9]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[10]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[11]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[12]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[13]  Henry Tirri,et al.  On Discriminative Bayesian Network Classifiers and Logistic Regression , 2005, Machine Learning.

[14]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[15]  Franz Pernkopf,et al.  Discriminative versus generative parameter and structure learning of Bayesian network classifiers , 2005, ICML.

[16]  Brian D. Ripley,et al.  Modern Applied Statistics with S Fourth edition , 2002 .