Case Representation Issues for Case-Based Reasoning from Ensemble Research

Ensembles of classifiers will produce lower errors than the member classifiers if there is diversity in the ensemble. One means of producing this diversity in nearest neighbour classifiers is to base the member classifiers on different feature subsets. In this paper we show four examples where this is the case. This has implications for the practice of feature subset selection (an important issue in CBR and data-mining) because it shows that, in some situations, there is no single best feature subset to represent a problem. We show that if diversity is emphasised in the development of the ensemble that the ensemble members appear to be local learners specializing in sub-domains of the problem space. The paper concludes with some proposals on how analysis of ensembles of local learners might provide insight on problem-space decomposition for hierarchical CBR.

[1]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[2]  Tom Heskes,et al.  Input selection based on an ensemble , 2000, Neurocomputing.

[3]  Tin Kam Ho,et al.  Nearest Neighbors in Random Subspaces , 1998, SSPR/SPR.

[4]  L. Darrell Whitley,et al.  Genetic Approach to Feature Selection for Ensemble Creation , 1999, GECCO.

[5]  Ron Kohavi,et al.  The Utility of Feature Weighting in Nearest-Neighbor Algorithms , 1997 .

[6]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  David W. Opitz,et al.  Generating Accurate and Diverse Members of a Neural-Network Ensemble , 1995, NIPS.

[8]  David W. Aha,et al.  Case-Based Learning: Beyond Classification of Feature Vectors , 1997, ECML.

[9]  Ian D. Watson,et al.  An Introduction to Case-Based Reasoning , 1995, UK Workshop on Case-Based Reasoning.

[10]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[11]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Padraig Cunningham,et al.  Déjà Vu: A Hierarchical Case-Based Reasoning System for Software Design , 1992, ECAI.

[13]  Padraig Cunningham,et al.  Using Diversity in Preparing Ensembles of Classifiers Based on Different Feature Subsets to Minimize Generalization Error , 2001, ECML.

[14]  Allen Newell,et al.  Computer science as empirical inquiry: symbols and search , 1976, CACM.

[15]  Padraig Cunningham,et al.  Hierarchical Case-Based Reasoning Integrating Case-Based and Decompositional Problem-Solving Techniques for Plant-Control Software Design , 2001, IEEE Trans. Knowl. Data Eng..

[16]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[17]  David W. Aha,et al.  Feature Weighting for Lazy Learning Algorithms , 1998 .

[18]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[19]  Kagan Tumer,et al.  Error Correlation and Error Reduction in Ensemble Classifiers , 1996, Connect. Sci..

[20]  Alex A. Freitas.  Data Mining with Evolutionary Algorithms: Research Directions - Papers from the AAAI Workshop , 1999 .

[21]  Kevin J. Cherkauer.  Stuffing Mind into Computer: Knowledge and Learning for Intelligent Systems , 1995, Informatica.

[22]  Padraig Cunningham,et al.  Diversity versus Quality in Classification Ensembles Based on Feature Selection , 2000, ECML.

[23]  Padraig Cunningham,et al.  Using Introspective Learning to Improve Retrieval in CBR: A Case Study in Air Traffic Control , 1997, ICCBR.