The RISE 2.0 System: A Case Study in Multistrategy Learning

Several well-developed approaches to inductive learning now exist, but each has specific limitations that are hard to overcome. Multi-strategy learning at tempts to tackle this problem by combining multiple methods in one algorithm. This report describes a unification of two widely-used empirical approaches: rule induction and instance-based learning. In the new algorithm, instances are treated as maximally specific rules, and classification is performed using a best-match strategy. Rules are learned by gradually generalizing instances until no improvement in apparent accuracy is obtained. Theoretical analysis shows this approach to be efficient. It is implemented in the RISE 2.0 system. In an extensive empirical study, RISE consistently outperforms state-of-the-art representatives of both its parent approaches (PEBLS and CN2), as well as a decision tree learner (C4.5). Most significantly, in 15 of the domains studied, RISE achieves higher accuracy than the best of PEBLS and CN2, showing that a significant synergy can be obtained by combining multiple empirical methods.

[1]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[2]  Michael J. Pazzani,et al.  Exploring the Decision Forest: An Empirical Investigation of Occam's Razor in Decision Tree Induction , 1993, J. Artif. Intell. Res..

[3]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[4]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[5]  Raymond J. Mooney,et al.  Theory Refinement Combining Analytical and Empirical Methods , 1994, Artif. Intell..

[6]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[7]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[8]  Carla E. Brodley,et al.  Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection , 1993 .

[9]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[10]  Nada Lavrac,et al.  The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains , 1986, AAAI.

[11]  Padhraic Smyth,et al.  A Hybrid Rule-Based/Bayesian Classifier , 1990, ECAI.

[12]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[13]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[14]  Pedro M. Domingos.  The RISE system: conquering without separating , 1994, Proceedings Sixth International Conference on Tools with Artificial Intelligence. TAI 94.

[15]  Tim Niblett,et al.  Constructing Decision Trees in Noisy Domains , 1987, EWSL.

[16]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[17]  Paul E. Utgoff,et al.  Perceptron Trees : A Case Study in ybrid Concept epresentations , 1999 .

[18]  Wray L. Buntine.  Learning Classification Rules Using Bayes , 1989, ML.

[19]  Christopher K. Riesbeck,et al.  Inside Case-Based Reasoning , 1989 .

[20]  J. Ross Quinlan,et al.  Generating Production Rules from Decision Trees , 1987, IJCAI.

[21]  Cullen Schaffer,et al.  A Conservation Law for Generalization Performance , 1994, ICML.

[22]  Ryszard S. Michalski,et al.  A theory and methodology of inductive learning , 1993 .

[23]  Jason Catlett,et al.  Megainduction: A Test Flight , 1991, ML.

[24]  Paul S. Rosenbloom,et al.  Improving Rule-Based Systems Through Case-Based Reasoning , 1991, AAAI.

[25]  J. Ross Quinlan,et al.  Combining Instance-Based and Model-Based Learning , 1993, ICML.