GA-based feature subset clustering for combination of multiple nearest neighbors classifiers

Nearest neighbor classifier (NNC) is stable to the change of the training data set while sensitive to the variation of the feature set. The combination of multiple NNCs on different subsets of features may outperform the standard NNC. In this paper, we develop a new method called FC-MNNC based on feature subset clustering for combining multiple NNCs to obtain better performance than that using a single NNC. In this method, GA is used for clustering features to form different feature subsets according to the combination classification accuracy. Multiple NNCs based on the corresponding feature subsets are parallel and independent to classify one pattern. The final decision is aggregated by majority voting rule, which is a simple and efficient technique. To demonstrate the performance of FC-MNNC, we select four UCI databases in our experiments. The proposed FC-MNNC is compared with (i) standard NNC, (ii) feature selection using GA in individual NNC and (iii) feature subset selection using GA in multiple NNCs. The experimental results show that the accuracy of FC-MNNC is better than that of the standard NNC and feature selection using GA in individual classifier. The performance of FC-MNNC is not worse than that of feature subset selection using GA in multiple NNCs. It is also demonstrated that FC-MNNC is robust to irrelevant features.