k-Relevance Vectors for Pattern Classification

This study combines two different learning paradigms, k-nearest neighbor (k-NN) rule, as memory-based learning paradigm and relevance vector machines (RVM), as statistical learning paradigm. This combination is performed in kernel space and is called k-relevance vector (k-RV). The purpose is to improve the performance of k-NN rule. The proposed model significantly prunes irrelevant attributes. We also introduced a new parameter, responsible for early stopping of iterations in RVM. We show that the new parameter improves the classification accuracy of k-RV. Intensive experiments are conducted on several classification datasets from University of California Irvine (UCI) repository and two real datasets from computer vision domain. The performance of k-RV is highly competitive compared to a few state-of-the-arts in terms of classification accuracy.

[1]  Gürsel Serpen,et al.  Host-based misuse intrusion detection using PCA feature extraction and kNN classification algorithms , 2018, Intell. Data Anal..

[2]  David J. C. MacKay,et al.  The Evidence Framework Applied to Classification Networks , 1992, Neural Computation.

[3]  Andrew Beng Jin Teoh,et al.  A new sparse model for traffic sign classification using soft histogram of oriented gradients , 2017, Appl. Soft Comput..

[4]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[5]  Andrew Beng Jin Teoh,et al.  Online Heterogeneous Face Recognition Based on Total-Error-Rate Minimization , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[6]  D. N. Tibarewala,et al.  Classification of lower limb motor imagery using K Nearest Neighbor and Naïve-Bayesian classifier , 2016, 2016 3rd International Conference on Recent Advances in Information Technology (RAIT).

[7]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[8]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[9]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[10]  Chih-Fong Tsai,et al.  CANN: An intrusion detection system based on combining cluster centers and nearest neighbors , 2015, Knowl. Based Syst..

[11]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[12]  Lin Zhu,et al.  A graph-based semi-supervised k nearest-neighbor method for nonlinear manifold distributed data classification , 2016, Inf. Sci..

[13]  Robert Ivor John,et al.  A method of learning weighted similarity function to improve the performance of nearest neighbor , 2009, Inf. Sci..

[14]  Chi-Man Vong,et al.  Sparse Bayesian Extreme Learning Machine for Multi-classification , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Gerhard Tutz,et al.  Improved methods for the imputation of missing data by nearest neighbor methods , 2015, Comput. Stat. Data Anal..

[16]  A comparative study of deep learning architectures on melanoma detection. , 2019 .

[17]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Farhood Rismanchian,et al.  Proposing a Localized Relevance Vector Machine for Pattern Classification , 2019, ArXiv.

[19]  Francisco Herrera,et al.  kNN-IS: An Iterative Spark-based design of the k-Nearest Neighbors classifier for big data , 2017, Knowl. Based Syst..

[20]  David Zhang,et al.  On kernel difference-weighted k-nearest neighbor classification , 2008, Pattern Analysis and Applications.

[21]  Lorenzo Rosasco,et al.  Nonparametric sparsity and regularization , 2012, J. Mach. Learn. Res..

[22]  Harikumar Rajaguru,et al.  Performance Analysis of KNN Classifier with Various Distance Metrics Method for MRI Images , 2019 .

[23]  Sara Hosseinzadeh Kassani,et al.  Introducing a hybrid model of DEA and data mining in evaluating efficiency. Case study: Bank Branches , 2018, ArXiv.

[24]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[25]  J. Ross Quinlan,et al.  Simplifying Decision Trees , 1987, Int. J. Man Mach. Stud..

[26]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[27]  Prachi Surlakar,et al.  Comparative Analysis of K-Means and K-Nearest Neighbor Image Segmentation Techniques , 2016, 2016 IEEE 6th International Conference on Advanced Computing (IACC).

[28]  Ying Ma,et al.  A hybrid method based on extreme learning machine and k-nearest neighbor for cloud classification of ground-based visible cloud image , 2015, Neurocomputing.

[29]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[30]  Mirko Perkusich,et al.  Early diagnosis of gastrointestinal cancer by using case-based and rule-based reasoning , 2016, Expert Syst. Appl..

[31]  Nada Lavrac,et al.  Selected techniques for data mining in medicine , 1999, Artif. Intell. Medicine.

[32]  Kar-Ann Toh,et al.  Benchmarking a reduced multivariate polynomial pattern classifier , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[34]  Julio López,et al.  Redefining nearest neighbor classification in high-dimensional settings , 2018, Pattern Recognit. Lett..

[35]  B. Gopinath,et al.  Development of an Automated Medical Diagnosis System for Classifying Thyroid Tumor Cells using Multiple Classifier Fusion , 2015, Technology in cancer research & treatment.

[36]  Masashi Sugiyama,et al.  Target Neighbor Consistent Feature Weighting for Nearest Neighbor Classification , 2011, NIPS.

[37]  Gongde Guo,et al.  Nearest neighbor classification of categorical data by attributes weighting , 2015, Expert Syst. Appl..

[38]  Taeho Jo,et al.  Using K Nearest Neighbors for text segmentation with feature similarity , 2017, 2017 International Conference on Communication, Control, Computing and Electronics Engineering (ICCCCEE).

[39]  Carl E. Rasmussen,et al.  Healing the relevance vector machine through augmentation , 2005, ICML.

[40]  Euntai Kim,et al.  Proposing a GPU based modified fuzzy nearest neighbor rule for traffic sign detection , 2015, 2015 15th International Conference on Control, Automation and Systems (ICCAS).