Building a bayesian factor tree from examples

A criterion based on mutual information among variables is proposed for building a bayesian tree from a finite number of examples. The factor graph, in Forney-style form, can be used as an associative memory that performs probabilistic inference in data fusion applications. The procedure is explained with the aid of a fully-described example.

[1]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[2]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[3]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Process. Mag..

[4]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[5]  Sergio Benedetto,et al.  Principles of Digital Transmission: With Wireless Applications , 1999 .

[6]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[7]  Nir Friedman,et al.  Learning Module Networks , 2002, J. Mach. Learn. Res..

[8]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[9]  Frank R. Kschischang,et al.  A general computation rule for lossy summaries/messages with examples from equalization , 2006, ArXiv.

[10]  G. Bradski.  Graphical Models: Foundations of Neural Computation , 2003 .

[11]  J. Hawkins,et al.  On Intelligence , 2004 .

[12]  Francesco Palmieri.  Notes on Factor Graphs , 2008, WIRN.

[13]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[14]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[15]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[16]  Li Ping,et al.  The Factor Graph Approach to Model-Based Signal Processing , 2007, Proceedings of the IEEE.

[17]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[18]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[19]  Antony K. Haynes.  Learning Hidden Structure from Data: A Method for Marginalizing Joint Distributions Using Minimum Cross-Correlation Error. , 1997 .

[20]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).