An Empirical Investigation of the K2 Metric

The K2 metric is a well-known evaluation measure (or scoring function) for learning Bayesian networks from data [7]. It is derived by assuming uniform prior distributions on the values of an attribute for each possible instantiation of its parent attributes. This assumption introduces a tendency to select simpler network structures. In this paper we modify the K2 metric in three different ways, introducing a parameter by which the strength of this tendency can be controlled. Our experiments with the ALARM network [2] and the BOBLO network [17] suggest that--somewhat contrary to our expectations--a slightly stronger tendency towards simpler structures may lead to even better results.

[1]  Igor Kononenko,et al.  On Biases in Estimating Multi-Valued Attributes , 1995, IJCAI.

[2]  Rudolf Kruse,et al.  Uncertainty and vagueness in knowledge based systems: numerical methods , 1991, Artificial intelligence.

[3]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[4]  Kristian G. Olesen,et al.  HUGIN - A Shell for Building Bayesian Belief Universes for Expert Systems , 1989, IJCAI.

[5]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[6]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[7]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[8]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[9]  Rudolf Kruse,et al.  Uncertainty and Vagueness in Knowledge Based Systems , 1991, Artificial Intelligence.

[10]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[11]  C. Borgelt,et al.  Evaluation measures for learning probabilistic and possibilistic networks , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[12]  L. Wehenkel.  On uncertainty measures used for decision tree induction , 1996 .

[13]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[14]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[15]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[16]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[17]  David Heckerman,et al.  Probabilistic similarity networks , 1991, Networks.