An evolutionary voting for k-nearest neighbours

We optimize the voting system for the k nearest neighbours.We use evolutionary computation.We study the influence of the closeness of neighbours on the search process.The results are statistically validated. This work presents an evolutionary approach to modify the voting system of the k-nearest neighbours (kNN) rule we called EvoNN. Our approach results in a real-valued vector which provides the optimal relative contribution of the k-nearest neighbours. We compare two possible versions of our algorithm. One of them (EvoNN1) introduces a constraint on the resulted real-valued vector where the greater value is assigned to the nearest neighbour. The second version (EvoNN2) does not include any particular constraint on the order of the weights. We compare both versions with classical kNN and 4 other weighted variants of the kNN on 48 datasets of the UCI repository. Results show that EvoNN1 outperforms EvoNN2 and statistically obtains better results than the rest of the compared methods.

[1]  José Cristóbal Riquelme Santos,et al.  On the evolutionary optimization of k-NN by label-dependent feature weighting , 2012, Pattern Recognit. Lett..

[2]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[3]  José Cristóbal Riquelme Santos,et al.  Automatic environmental quality assessment for mixed-land zones using lidar and intelligent techniques , 2011, Expert Syst. Appl..

[4]  Enrique Vidal,et al.  Learning weighted metrics to minimize nearest-neighbor classification error , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Mengjie Zhang,et al.  Evaluation of particle swarm optimization based centroid classifier with different distance metrics , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[6]  Seoung Bum Kim,et al.  Sequential random k-nearest neighbor feature selection for high-dimensional data , 2015, Expert Syst. Appl..

[7]  Sahibsingh A. Dudani.  The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Ahmed Bouridane,et al.  Simultaneous feature selection and feature weighting using Hybrid Tabu Search/K-nearest neighbor classifier , 2007, Pattern Recognit. Lett..

[9]  Vladimir B. Berikov.  Weighted ensemble of algorithms for complex data clustering , 2014, Pattern Recognit. Lett..

[10]  William F. Punch,et al.  Knowledge discovery in medical and biological datasets using a hybrid Bayes classifier/evolutionary algorithm , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[11]  Fernando Fernández,et al.  Local Feature Weighting in Nearest Prototype Classification , 2008, IEEE Transactions on Neural Networks.

[12]  Feng Zhao,et al.  Robust Local Feature Weighting Hard C-Means Clustering Algorithm , 2011, IScIDE.

[13]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[14]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[15]  Wlodzislaw Duch,et al.  LVQ algorithm with instance weighting for generation of prototype-based rules , 2011, Neural Networks.

[16]  Sang-goo Lee,et al.  Reversed CF: A fast collaborative filtering algorithm using a k-nearest neighbor graph , 2015, Expert Syst. Appl..

[17]  Jianping Gou,et al.  A Novel Weighted Voting for K-Nearest Neighbor Rule , 2011, J. Comput..

[18]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[19]  Ahmed Al-Ani,et al.  Optimizing the k-NN metric weights using differential evolution , 2010, 2010 International Conference on Multimedia Computing and Information Technology (MCIT).

[20]  James A. Rodger,et al.  A fuzzy nearest neighbor neural network statistical model for predicting demand for natural gas and energy cost savings in public buildings , 2014, Expert Syst. Appl..

[21]  Jianping Gou,et al.  A new distance-weighted k-nearest neighbor classifier , 2012 .

[22]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..