Evolutionary-modified fuzzy nearest-neighbor rule for pattern classification

Proposed a new fuzzy nearest neighbor rule with two new parameters.Proposed two new cost functions working in fuzzy class membership space.Verified the model reliability using experiments on several data sets.Statistically compared with several fuzzy based nearest neighbor rules.Implemented on graphical processing units for the sake of speed up. This paper presents an improved version of the well-established k nearest neighbor (k-NN) and fuzzy NN (FNN), termed the multi-objective genetic-algorithm-modified FNN (MOGA-MFNN). The MFNN design problem is converted into a multi-modal objective maximization problem constrained by four objective functions. Thereafter, the associated parameter set of the MFNN and the feature attributes can be determined optimally and automatically via the non-dominated sorting genetic algorithm II. We introduce two new objective functions termed the Margin-I and Margin-II, which are used to improve the generalization capability of the MFNN for the unknown data, along with two existing performance functions: the geometric mean and the area under the receiver-operated characteristic curve for the training accuracy. Moreover, we proposed a novel data-dependent weight-assignment technique for local class membership functions of the MFNN. The technique enables the MFNN to determine its local neighbors adaptively through the MOGA algorithm. To expedite the classification, the MOGA-MFNN is implemented on a graphical processing unit (GPU), which significantly increases the computation speed. Furthermore, the local class-membership function of the MFNN can be computed in advance, rather than delaying it to the classification stage. This again can improve the classification speed. The MOGA-MFNN is evaluated on 20 datasets obtained from the repository of the University of California, Irvine (UCI). The experiments with rigorous statistical significance tests demonstrate that the proposed method performs competitively with the existing methods.

[1]  Shweta Taneja,et al.  MFZ-KNN — A modified fuzzy based K nearest neighbor algorithm , 2015, 2015 International Conference on Cognitive Computing and Information Processing(CCIP).

[2]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[3]  Khalid Zenkouar,et al.  A new nearest neighbor classification method based on fuzzy set theory and aggregation operators , 2017, Expert Syst. Appl..

[4]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[5]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[6]  Joon H. Han,et al.  A fuzzy K-NN algorithm using weights from the variance of membership values , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[7]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[8]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[9]  Min-Yuan Cheng,et al.  A Swarm-Optimized Fuzzy Instance-based Learning approach for predicting slope collapses in mountain roads , 2015, Knowl. Based Syst..

[10]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[12]  José Salvador Sánchez,et al.  A bias correction function for classification performance assessment in two-class imbalanced problems , 2014, Knowl. Based Syst..

[13]  Zhe Wang,et al.  Gravitational fixed radius nearest neighbor for imbalanced problem , 2015, Knowl. Based Syst..

[14]  Francisco Herrera,et al.  An Interval Valued K-Nearest Neighbors Classifier , 2015, IFSA-EUSFLAT.

[15]  Robert B. Fisher,et al.  Classifying imbalanced data sets using similarity based hierarchical decomposition , 2015, Pattern Recognit..

[16]  Francisco Herrera,et al.  Fuzzy nearest neighbor algorithms: Taxonomy, experimental analysis and prospects , 2014, Inf. Sci..

[17]  Juan José Rodríguez Diez,et al.  Random Balance: Ensembles of variable priors classifiers for imbalanced data , 2015, Knowl. Based Syst..

[18]  Tadashi Shibata,et al.  A Nearest Neighbor Classifier Employing Critical Boundary Vectors for Efficient On-Chip Template Reduction , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[19]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[20]  Francisco Herrera,et al.  IFROWANN: Imbalanced Fuzzy-Rough Ordered Weighted Average Nearest Neighbor Classification , 2015, IEEE Transactions on Fuzzy Systems.

[21]  Dayou Liu,et al.  Design of an Enhanced Fuzzy k-nearest Neighbor Classifier Based Computer Aided Diagnostic System for Thyroid Disease , 2012, Journal of Medical Systems.

[22]  Nenad Tomašev,et al.  Hubness-based fuzzy measures for high-dimensional k-nearest neighbor classification , 2014 .

[23]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[24]  Michel Barlaud,et al.  Fast k nearest neighbor search using GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[25]  Xindong Wu,et al.  The Top Ten Algorithms in Data Mining , 2009 .

[26]  Euntai Kim,et al.  Proposing a GPU based modified fuzzy nearest neighbor rule for traffic sign detection , 2015, 2015 15th International Conference on Control, Automation and Systems (ICCAS).

[27]  Gang Wang,et al.  A novel bankruptcy prediction model based on an adaptive fuzzy k-nearest neighbor method , 2011, Knowl. Based Syst..

[28]  Michael C. Mozer,et al.  Optimizing Classifier Performance via an Approximation to the Wilcoxon-Mann-Whitney Statistic , 2003, ICML.

[29]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[30]  Paul Scheunders,et al.  Genetic feature selection combined with composite fuzzy nearest neighbor classifiers for hyperspectral satellite imagery , 2002, Pattern Recognit. Lett..

[31]  Eli Shechtman,et al.  In defense of Nearest-Neighbor based image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  J. Ross Quinlan,et al.  Simplifying Decision Trees , 1987, Int. J. Man Mach. Stud..

[33]  Frank Chung-Hoon Rhee,et al.  An interval type-2 fuzzy K-nearest neighbor , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[34]  Sushmita Mitra,et al.  Multi-objective optimization of shared nearest neighbor similarity for feature selection , 2015, Appl. Soft Comput..

[35]  Aboul Ella Hassanien,et al.  Cattle classifications system using Fuzzy K- Nearest Neighbor Classifier , 2015, 2015 International Conference on Informatics, Electronics & Vision (ICIEV).

[36]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..