Parameter learning in hybrid Bayesian networks using prior knowledge

Mixtures of truncated basis functions have been recently proposed as a generalisation of mixtures of truncated exponentials and mixtures of polynomials for modelling univariate and conditional distributions in hybrid Bayesian networks. In this paper we analyse the problem of learning the parameters of marginal and conditional MoTBF densities when both prior knowledge and data are available. Incorporating prior knowledge provide a valuable tool for obtaining useful models, especially in domains of applications where data are costly or scarce, and prior knowledge is available from practitioners. We explore scenarios where the prior knowledge can be expressed as an MoTBF density that is afterwards combined with another MoTBF density estimated from the available data. The resulting model remains within the MoTBF class which is a convenient property from the point of view of inference in hybrid Bayesian networks. The performance of the proposed method is tested in a series of experiments carried out over synthetic and real data.

[1]  Serafín Moral,et al.  Mixtures of Truncated Exponentials in Hybrid Bayesian Networks , 2001, ECSQARU.

[2]  Kaplan,et al.  ‘Combining Probability Distributions from Experts in Risk Analysis’ , 2000, Risk analysis : an official publication of the Society for Risk Analysis.

[3]  Prakash P. Shenoy,et al.  Inference in hybrid Bayesian networks using mixtures of polynomials , 2011, Int. J. Approx. Reason..

[4]  S. Lauritzen.  Propagation of Probabilities, Means, and Variances in Mixed Graphical Association Models , 1992 .

[5]  Rafael Rumí,et al.  Mixtures of truncated basis functions , 2012, Int. J. Approx. Reason..

[6]  Antonio Salmerón,et al.  Learning Bayesian Networks for Regression from Incomplete Databases , 2010, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[7]  Serafín Moral,et al.  Approximating Conditional MTE Distributions by Means of Mixed Trees , 2003, ECSQARU.

[8]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[9]  Rafael Rumí,et al.  Hybrid Bayesian network classifiers: Application to species distribution models , 2010, Environ. Model. Softw..

[10]  Antonio Salmerón,et al.  Incorporating Prior Knowledge when Learning Mixtures of Truncated Basis Functions from Data , 2013, SCAI.

[11]  Nevin Lianwen Zhang,et al.  Exploiting Causal Independence in Bayesian Network Inference , 1996, J. Artif. Intell. Res..

[12]  David Heckerman,et al.  Bayesian Networks for Data Mining , 2004, Data Mining and Knowledge Discovery.

[13]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[14]  Rafael Rumí,et al.  Naive Bayes Classifier with Mixtures of Polynomials , 2015, ICPRAM.

[15]  José A. Gámez,et al.  Mixture of truncated exponentials in supervised classification: Case study for the naive bayes and averaged one-dependence estimators classifiers , 2011, 2011 11th International Conference on Intelligent Systems Design and Applications.

[16]  Antonio Salmerón,et al.  Learning mixtures of truncated basis functions from data , 2014, Int. J. Approx. Reason..

[17]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[18]  Tzu-Tsung Wong.  Alternative prior assumptions for improving the performance of naïve Bayesian classifiers , 2008, Data Mining and Knowledge Discovery.

[19]  Takafumi Kanamori,et al.  Improving Logitboost with prior knowledge , 2013, Inf. Fusion.

[20]  Concha Bielza,et al.  Learning mixtures of polynomials from data using B-spline interpolation , 2012 .

[21]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[22]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[23]  Concha Bielza,et al.  Learning mixtures of polynomials of multidimensional probability densities from data using B-spline interpolation , 2014, Int. J. Approx. Reason..

[24]  Rafael Rumí,et al.  Supervised Classification Using Hybrid Probabilistic Decision Graphs , 2014, Probabilistic Graphical Models.

[25]  Prakash P. Shenoy,et al.  Axioms for probability and belief-function proagation , 1990, UAI.

[26]  Carmelo Rodríguez,et al.  Selective Naive Bayes for Regression Based on Mixtures of Truncated Exponentials , 2007, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[27]  Antonio Salmerón,et al.  On Using the PC Algorithm for Learning Continuous Bayesian Networks: An Experimental Analysis , 2013, CAEPIA.

[28]  José A. Gámez,et al.  Data clustering using hidden variables in hybrid Bayesian networks , 2014, Progress in Artificial Intelligence.

[29]  Serafín Moral,et al.  Estimating mixtures of truncated exponentials in hybrid bayesian networks , 2006 .