A Weighted KNN Algorithm Based on Entropy Method

Aiming at the problem that the classification accuracy of K-nearest neighbor algorithm is not high, this paper proposes a K-nearest neighbor algorithm that uses the weighted entropy method of Extreme value (EEM-KNN algorithm). The entropy method assigns weight to the sample’s feature index, and then introduces the weight of the feature index when calculating the distance between the query sample vector and the training sample vector. The four groups of classification data sets are used as test samples to test the effectiveness of the improved KNN algorithm, it also compares the difference between the improved algorithm and the traditional algorithm under different K values. Algorithms are implemented and tested on the Jupyter Notebook interactive platform. The improved KNN algorithm is verified by experiments, and the classification accuracy is improved.

[1]  Jing-nan Sun,et al.  Entropy method for determination of weight of evaluating indicators in fuzzy synthetic evaluation for water quality assessment. , 2006, Journal of environmental sciences.

[2]  Liu Jin-sheng.  KNN Algorithm Based on Feature Entropy Correlation Difference , 2011 .

[3]  Liu Jin-sheng.  Improved K-nearest neighbor algorithm for feature union entropy , 2011 .

[4]  Ali Ghaffari,et al.  ECG arrhythmia recognition via a neuro-SVM-KNN hybrid classifier with virtual QRS image-based geometrical features , 2012, Expert Syst. Appl..

[5]  Songbo Tan,et al.  An effective refinement strategy for KNN text classifier , 2006, Expert Syst. Appl..

[6]  Chokri Ben Amar,et al.  Classification improvement of local feature vectors over the KNN algorithm , 2011, Multimedia Tools and Applications.

[7]  StinsonG.,et al.  Mapping attributes of Canada’s forests at moderate resolution through kNN and MODIS imagery , 2014 .

[8]  Ren Jiang.  PSO Based Feature Weighting Algorithm for KNN , 2007 .

[9]  Asoke K. Nandi,et al.  Automatic Modulation Classification Using Combination of Genetic Programming and KNN , 2012, IEEE Transactions on Wireless Communications.

[10]  F. Barkhof,et al.  Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (kNN-TTPs)☆ , 2013, NeuroImage: Clinical.

[11]  Suraj Prakash Harsha,et al.  Fault diagnosis of rolling element bearing with intrinsic mode function of acoustic emission data using APF-KNN , 2013, Expert Syst. Appl..

[12]  Xiaoguang Zhou,et al.  An Enhanced Entropy-K-Nearest Neighbor Algorithm Based on Attribute Reduction , 2015 .

[13]  Thomas A. Darden,et al.  Gene selection for sample classification based on gene expression data: study of sensitivity to choice of parameters of the GA/KNN method , 2001, Bioinform..

[14]  Huafeng Ding,et al.  Enhancement of K-nearest neighbor algorithm based on weighted entropy of attribute value , 2012, 2012 5th International Conference on BioMedical Engineering and Informatics.

[15]  Zhou Zhong-mei.  Enhancement of K-nearest neighbor algorithm based on information entropy of attribute value , 2010 .

[16]  Wang Kai-jue.  Improve KNN algorithm based on entropy method , 2009 .