A Novel RFE-SVM-based Feature Selection Approach for Classification

The feature selection for classification is a very active research field in data mining and optimization. Its combinatorial nature requires the development of specific techniques (such as filters, wrappers, genetic algorithms, simulated annealing, and so on) or hybrid approaches combining several optimization methods. In this context, the support vector machine recursive feature elimination (SVM-RFE), is distinguished as one of the most effective methods. However, the RFE-SVM algorithm is a greedy method that only hopes to find the best possible combination for classification. To overcome this limitation, we propose an alternative approach with the aim to combine the RFE-SVM algorithm with local search operators based on operational research and artificial intelligence. To assess the contributions of our approach, we conducted a series of experiments on datasets from UCI Machine Learning Repository. The experimental results which we obtained are very promising and show the contribution of the local search on the classification process. The main conclusion is that the reuse of features previously removed during the RFE-SVM process improves the quality of the final classifier.