Collective mining of Bayesian networks from distributed heterogeneous data

We present a collective approach to learning a Bayesian network from distributed heterogeneous data. In this approach, we first learn a local Bayesian network at each site using the local data. Then each site identifies the observations that are most likely to be evidence of coupling between local and non-local variables and transmits a subset of these observations to a central site. Another Bayesian network is learnt at the central site using the data transmitted from the local site. The local and central Bayesian networks are combined to obtain a collective Bayesian network, which models the entire data. Experimental results and theoretical justification that demonstrate the feasibility of our approach are presented.

[1]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[2]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[3]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[4]  David J. Spiegelhalter,et al.  Sequential updating of conditional probabilities on directed graphical structures , 1990, Networks.

[5]  Piero P. Bonissone,et al.  Proceedings of the Fourth Annual Conference on Uncertainty in Artificial Intelligence , 1990, UAI 1990.

[6]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[7]  Naoki Abe,et al.  Polynomial learnability of probabilistic concepts with respect to the Kullback-Leibler divergence , 1991, COLT '91.

[8]  Eugene Charniak,et al.  Bayesian Networks without Tears , 1991, AI Mag..

[9]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[10]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[11]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[12]  Remco R. Bouckaert,et al.  Properties of Bayesian Belief Network Learning Algorithms , 1994, UAI.

[13]  Gregory M. Provan,et al.  A Comparison of Induction Algorithms for Selective and non-Selective Bayesian Classifiers , 1995, ICML.

[14]  S. Lauritzen.  The EM algorithm for graphical association models with missing data , 1995 .

[15]  Bo Thiesson,et al.  Accelerated Quantification of Bayesian Networks with Incomplete Data , 1995, KDD.

[16]  Kazuo J. Ezawa,et al.  Fraud/Uncollectible Debt Detection Using a Bayesian Network Based Learning System: A Rare Binary Outcome with Mixed Data Structures , 1995, UAI.

[17]  David Heckerman,et al.  Learning Bayesian Networks: A Unification for Discrete and Gaussian Domains , 1995, UAI.

[18]  Michael J. Pazzani,et al.  Syskill & Webert: Identifying Interesting Web Sites , 1996, AAAI/IAAI, Vol. 1.

[19]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[20]  David Maxwell Chickering,et al.  Efficient Approximations for the Marginal Likelihood of Incomplete Data Given a Bayesian Network , 1996, UAI.

[21]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[22]  Michael J. Pazzani,et al.  Revising User Profiles: The Search for Interesting Web Sites , 1996 .

[23]  Nir Friedman,et al.  On the Sample Complexity of Learning Bayesian Networks , 1996, UAI.

[24]  Roger L. King,et al.  Supporting Information Infrastructure for Distributed, Heterogeneous Knowledge Discovery , 1996 .

[25]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[26]  Raj Bhatnagar,et al.  Pattern Discovery in Distributed Databases , 1997, AAAI/IAAI.

[27]  Michael I. Jordan,et al.  Estimating Dependency Structure as a Hidden Variable , 1997, NIPS.

[28]  Moninder Singh,et al.  Learning Bayesian Networks from Incomplete Data , 1997, AAAI/IAAI.

[29]  Nir Friedman,et al.  Sequential Update of Bayesian Network Structure , 1997, UAI.

[30]  Eric Bauer,et al.  Update Rules for Parameter Estimation in Bayesian Networks , 1997, UAI.

[31]  Kenji Yamanishi,et al.  Distributed cooperative Bayesian learning strategies , 1997, COLT '97.

[32]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[33]  Nir Friedman,et al.  Learning Belief Networks in the Presence of Missing Values and Hidden Variables , 1997, ICML.

[34]  Bo Thiesson,et al.  Learning Mixtures of Bayesian Networks , 1997, UAI 1997.

[35]  Wai Lam,et al.  Distributed data mining of probabilistic knowledge , 1997, Proceedings of 17th International Conference on Distributed Computing Systems.

[36]  David Windridge,et al.  NEW HORIZONS FROM MULTI-WAVELENGTH SKY SURVEYS , 1998 .

[37]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1998, Learning in Graphical Models.

[38]  Geoffrey Zweig,et al.  Speech Recognition with Dynamic Bayesian Networks , 1998, AAAI/IAAI.

[39]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[40]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[41]  Daniel Billsus,et al.  Learning Probabilistic User Models , 1998 .

[42]  Hillol Kargupta,et al.  Collective, Hierarchical Clustering from Distributed, Heterogeneous Data , 1999, Large-Scale Parallel Data Mining.

[43]  Daryl E. Hershberger,et al.  Collective Data Mining: a New Perspective toward Distributed Data Mining Advances in Distributed Data Mining Book , 1999 .

[44]  Srinivasan Parthasarathy,et al.  Clustering Distributed Homogeneous Datasets , 2000, PKDD.

[45]  Zoran Obradovic,et al.  Distributed clustering and local regression for knowledge discovery in multiple spatial databases , 2000, ESANN.

[46]  Mehmet Sayal,et al.  A Distributed Clustering Algorithm for Web-Based Access Patterns , 2000 .

[47]  Robert L. Grossman,et al.  A Framework for Finding Distributed Data Mining Strategies That are Intermediate Between Centralized , 2000 .

[48]  Philip K. Chan,et al.  Advances in Distributed and Parallel Knowledge Discovery , 2000 .

[49]  Hillol Kargupta,et al.  Collective Principal Component Analysis from Distributed, Heterogeneous Data , 2000, PKDD.

[50]  Craig Boutilier,et al.  Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence , 2000 .

[51]  Rong Chen,et al.  Distributed Web mining using Bayesian networks from multiple data streams , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[52]  Hillol Kargupta,et al.  Distributed Clustering Using Collective Principal Component Analysis , 2001, Knowledge and Information Systems.

[53]  Rong Chen,et al.  An approach to online Bayesian learning from multiple data streams , 2001 .

[54]  Viviane Crestana Jensen,et al.  Mining decentralized data repositories. , 2001 .

[55]  Kagan Tumer,et al.  Robust Order Statistics Based Ensembles for Distributed Data Mining , 2001 .

[56]  Hillol Kargupta,et al.  Toward ubiquitous Mining of distributed Data , 2001 .

[57]  Hillol Kargupta,et al.  Toward ubiquitous mining of distributed data , 2001, SPIE Defense + Commercial Sensing.

[58]  Hillol Kargupta,et al.  Distributed Multivariate Regression Using Wavelet-Based Collective Data Mining , 2001, J. Parallel Distributed Comput..

[59]  Salvatore J. Stolfo,et al.  Cost Complexity-Based Pruning of Ensemble Classifiers , 2001, Knowledge and Information Systems.

[60]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[61]  Rong Chen,et al.  A new algorithm for learning parameters of a Bayesian network from distributed data , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[62]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[63]  Hillol Kargupta,et al.  Distributed Data Mining: Algorithms, Systems, and Applications , 2003 .

[64]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[65]  Eleonora Riva Sanseverino,et al.  Distributed, Collaborative Data Analysis from Heterogeneous Sites Using a Scalable Evolutionary Technique , 2001, Applied Intelligence.

[66]  Rong Chen,et al.  Collective Mining of Bayesian Networks from Distributed Heterogeneous Data , 2004, Knowl. Inf. Syst..

[67]  Michael J. Pazzani,et al.  Learning and Revising User Profiles: The Identification of Interesting Web Sites , 1997, Machine Learning.

[68]  Foster J. Provost,et al.  Inductive policy: The pragmatics of bias selection , 1995, Machine Learning.

[69]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[70]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[71]  Sanjoy Dasgupta,et al.  The Sample Complexity of Learning Fixed-Structure Bayesian Networks , 1997, Machine Learning.