Combined feature selection and similarity modelling in case-based reasoning using hierarchical memetic algorithm

This paper proposes a new approach to discover knowledge about key features together with their degrees of importance in the context of case-based reasoning. A hierarchical memetic algorithm is designed for this purpose to search for the best feature subsets and similarity models at the same time. The objective of the memetic search is to optimize the possibility distributions derived for individual cases in the case library under a leave-one-out procedure. The information about the importance of selected features is revealed from the magnitudes of parameters of the learned similarity model. The effectiveness of the proposed approach has been shown by evaluation results on the benchmark data sets from the UCI repository and in comparisons with other machine learning techniques.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Tapio Elomaa,et al.  General and Efficient Multisplitting of Numerical Attributes , 1999, Machine Learning.

[3]  David L. Waltz,et al.  Trading MIPS and memory for knowledge engineering , 1992, CACM.

[4]  Ingoo Han,et al.  Global optimization of feature weights and the number of neighbors that combine in a case‐based reasoning system , 2006, Expert Syst. J. Knowl. Eng..

[5]  Shinn-Jang Ho,et al.  Design of accurate classifiers with a compact fuzzy-rule base using an evolutionary scatter partition of feature space , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Yi-Chung Hu,et al.  Finding useful fuzzy concepts for pattern classification using genetic algorithm , 2005, Inf. Sci..

[7]  Padraig Cunningham,et al.  Using Introspective Learning to Improve Retrieval in CBR: A Case Study in Air Traffic Control , 1997, ICCBR.

[8]  Nick Cercone,et al.  Rule-Induction and Case-Based Reasoning: Hybrid Architectures Appear Advantageous , 1999, IEEE Trans. Knowl. Data Eng..

[9]  Andy J. Keane,et al.  Meta-Lamarckian learning in memetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[10]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[12]  Ron Kohavi,et al.  The Utility of Feature Weighting in Nearest-Neighbor Algorithms , 1997 .

[13]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..

[14]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[15]  Peter Funk,et al.  Building similarity metrics reflecting utility in case-based reasoning , 2006, J. Intell. Fuzzy Syst..

[16]  Agnar Aamodt,et al.  Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , 1994, AI Commun..

[17]  Huan Liu,et al.  Consistency-based search in feature selection , 2003, Artif. Intell..

[18]  James Smith,et al.  A tutorial for competent memetic algorithms: model, taxonomy, and design issues , 2005, IEEE Transactions on Evolutionary Computation.

[19]  Karl Branting,et al.  Acquiring Customer Preferences from Return-Set Selections , 2001, ICCBR.

[20]  Zhang Lei,et al.  Designing of classifiers based on immune principles and fuzzy rules , 2008, Inf. Sci..

[21]  Harro Kiendl,et al.  Finding Relevant Process Characteristics with a Method for Data-Based Complexity Reduction , 1999, Fuzzy Days.

[22]  D. Dubois,et al.  Fuzzy set modelling in case‐based reasoning , 1998 .

[23]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[24]  David W. Aha,et al.  Weighting Features , 1995, ICCBR.

[25]  Michael M. Richter,et al.  The Knowledge Contained in Similarity Measures , 1995 .

[26]  R Kahavi,et al.  Wrapper for feature subset selection , 1997 .

[27]  Chun-Nan Hsu,et al.  The ANNIGMA-wrapper approach to fast feature selection for neural nets , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[28]  Francesco Ricci,et al.  Learning a Local Similarity Metric for Case-Based Reasoning , 1995, ICCBR.

[29]  Susan Craw,et al.  Genetic Algorithms to Optimise CBR Retrieval , 2000, EWCBR.

[30]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[31]  Andrew A. Goldenberg,et al.  Development of a systematic methodology of fuzzy logic modeling , 1998, IEEE Trans. Fuzzy Syst..

[32]  Padraig Cunningham,et al.  Improving Recommendation Ranking by Learning Personal Feature Weights , 2004, ECCBR.

[33]  Kezhi Mao,et al.  Feature subset selection for support vector machines through discriminative function pruning analysis , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).