Hypergraphs as a mean of discovering the dependence structure of a discrete multivariate probability distribution

Most everyday reasoning and decision making is based on uncertain premises. The premises or attributes, which we must take into consideration, are random variables, therefore we often have to deal with a high dimensional multivariate random vector. A multivariate random vector can be represented graphically as a Markov network. Usually the structure of the Markov network is unknown. In this paper we construct special type of junction trees, in order to obtain good approximations of the real probability distribution. These junction trees are capable of revealing some of the conditional independences of the network. We have already introduced the concept of the t-cherry junction tree (E. Kovács and T. Szántai in Proceedings of the IFIP/IIASA//GAMM Workshop on Coping with Uncertainty, 2010), based on the t-cherry tree graph structure. This approximation uses only two and three dimensional marginal probability distributions. Now we use k-th order t-cherry trees, also called simplex multitrees to introduce the concept of the k-th order t-cherry junction tree. We prove that the k-th order t-cherry junction tree gives the best approximation among the family of k-width junction trees. Then we give a method which starting from a k-th order t-cherry junction tree constructs a (k+1)-th order t-cherry junction tree which gives at least as good approximation. In the last part we present some numerical results and some possible applications.

[1]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[2]  I. Csiszár $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[3]  András Prékopa,et al.  Probability Bounds with Cherry Trees , 2001, Math. Oper. Res..

[4]  Luis M. de Campos,et al.  A new approach for learning belief networks using independence criteria , 2000, Int. J. Approx. Reason..

[5]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[6]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[7]  Tamás Szántai,et al.  Probability bounds given by hypercherry trees , 2002, Optim. Methods Softw..

[8]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[9]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[10]  Jie Cheng,et al.  An Algorithm for Bayesian Belief Network Construction from Data , 2004 .

[11]  Luis M. de Campos,et al.  An Algorithm for Finding Minimum d-Separating Sets in Belief Networks , 1996, UAI.

[12]  Tamás Szántai,et al.  On the Approximation of a Discrete Multivariate Probability Distribution Using the New Concept of t -Cherry Junction Tree , 2010 .

[13]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[14]  Mahadev Satyanarayanan,et al.  Coping with uncertainty , 2003, IEEE Pervasive Computing.

[15]  J. Bukszár Upper bounds for the probability of a union by multitrees , 2001 .

[16]  Tamás Szántai,et al.  Application Of t-Cherry Junction Trees in Pattern Recognition , 2010 .

[17]  Marek Makowski,et al.  Coping with Uncertainty. Modeling and Policy Issues , 2006 .