Learning Inductive Rules Using Hellinger Measure

Systems for inducing classification rules from databases are valuable tools for assisting in the task of knowledge acquisition for expert systems. This paper presents an information theoretic approach for extracting knowledge from databases in the form of inductive rules using Hellinger measure, an entropy function which is utilized as a criteria for selecting rules generated from databases. In order to reduce the complexity of rule generation, the characteristics of Hellinger measure are analyzed and used to prune the search space of hypothesis. The system is implemented and tested on some well-known machine-learning databases.

[1]  Nada Lavrac,et al.  The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains , 1986, AAAI.

[2]  Brian R. Gaines,et al.  Induction of inference rules for expert systems , 1986 .

[3]  Vasant Dhar,et al.  Abstract-Driven Pattern Discovery in Databases , 1992, IEEE Trans. Knowl. Data Eng..

[4]  Keki B. Irani,et al.  Multi-interval discretization of continuos attributes as pre-processing for classi cation learning , 1993, IJCAI 1993.

[5]  Padhraic Smyth,et al.  An Information Theoretic Approach to Rule Induction from Databases , 1992, IEEE Trans. Knowl. Data Eng..

[6]  Jadzia Cendrowska,et al.  PRISM: An Algorithm for Inducing Modular Rules , 1987, Int. J. Man Mach. Stud..

[7]  Jiawei Han,et al.  Data-Driven Discovery of Quantitative Rules in Relational Databases , 1993, IEEE Trans. Knowl. Data Eng..

[8]  Ryszard S. Michalski,et al.  Data-driven constructive induction in AQ17-PRE: A method and experiments , 1991, [Proceedings] Third International Conference on Tools for Artificial Intelligence - TAI 91.

[9]  R. Beran.  Minimum Hellinger distance estimates for parametric models , 1977 .

[10]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[11]  Gregory Piatetsky-Shapiro,et al.  Discovery, Analysis, and Presentation of Strong Rules , 1991, Knowledge Discovery in Databases.

[12]  Adam Mrózek,et al.  A New Method for Discovering Rules from Examples in Expert Systems , 1992, Int. J. Man Mach. Stud..

[13]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Jianping Zhang,et al.  Selecting Typical Instances in Instance-Based Learning , 1992, ML.

[15]  Maciej Modrzejewski,et al.  Feature Selection Using Rough Sets Theory , 1993, ECML.

[16]  Andrew K. C. Wong,et al.  Statistical Technique for Extracting Classificatory Knowledge from Databases , 1991, Knowledge Discovery in Databases.

[17]  Changhwan Lee,et al.  A Context-Sensitive Discretization of Numeric Attributes for Classification Learning , 1994, ECAI.

[18]  Ronald L. Rivest,et al.  Inferring Decision Trees Using the Minimum Description Length Principle , 1989, Inf. Comput..

[19]  J. Ross Quinlan,et al.  Generating Production Rules from Decision Trees , 1987, IJCAI.