On the Approximation of a Discrete Multivariate Probability Distribution Using the New Concept of t -Cherry Junction Tree

Most everyday reasoning and decision making is based on uncertain premises. The premises or attributes, which we must take into consideration, are random variables, so that we often have to deal with a high dimensional discrete multivariate random vector. We are going to construct an approximation of a high dimensional probability distribution that is based on the dependence structure between the random variables and on a special clustering of the graph describing this structure. Our method uses just one-, two- and three-dimensional marginal probability distributions. We give a formula that expresses how well the constructed approximation fits to the real probability distribution. We then prove that every time there exists a probability distribution constructed this way, that fits to reality at least as well as the approximation constructed from the Chow–Liu dependence tree. In the last part we give some examples that show how efficient is our approximation in application areas like pattern recognition and feature selection.

[1]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[2]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[3]  Tamás Szántai,et al.  Probability bounds given by hypercherry trees , 2002, Optim. Methods Softw..

[4]  E. B. Andersen,et al.  Information Science and Statistics , 1986 .

[5]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[6]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[7]  I. Csiszár $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[8]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[9]  Adnan Darwiche,et al.  Inference in belief networks: A procedural guide , 1996, Int. J. Approx. Reason..

[10]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[11]  András Prékopa,et al.  Probability Bounds with Cherry Trees , 2001, Math. Oper. Res..

[12]  Frank Hutter,et al.  Incremental Thin Junction Trees for Dynamic Bayesian networks , 2004 .

[13]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[14]  Jie Cheng,et al.  An Algorithm for Bayesian Belief Network Construction from Data , 2004 .

[15]  Luis M. de Campos,et al.  An Algorithm for Finding Minimum d-Separating Sets in Belief Networks , 1996, UAI.