A kernelized non-parametric classifier based on feature ranking in anisotropic Gaussian kernel

Abstract Non-parametric methods make no assumptions about the form of data distribution and estimate it directly from the data. Kernel density estimation is a non-parametric method which estimates the probability density function of an unknown distribution. To estimate the density using a kernel estimator, it is necessary to have a bandwidth selection procedure. This paper proposes a kernelized non-parametric classifier based on feature ranking in anisotropic Gaussian kernel (KNR-AGK) and focuses on the selection of different bandwidths in kernel density estimation. KNR-AGK uses the rank of features to learn the parameters of an anisotropic Gaussian kernel and considers these ranks as kernel bandwidths in different dimensions. In the proposed method, the rank of features is also used for feature selection based on filter methods to exclude low-ranked features that have a negative impact on the performance of KNR-AGK. To evaluate the performance of the proposed method, comprehensive experiments are conducted on several benchmark datasets. Experiment results show that the proposed classifier has better performance than Gaussian kernel density estimation based classifier.

[1]  Richard Weber,et al.  Simultaneous feature selection and classification using kernel-penalized support vector machines , 2011, Inf. Sci..

[2]  Lijian Yang,et al.  Spline-backfitted kernel smoothing of partially linear additive model , 2011 .

[3]  Jianping Li,et al.  Multiple-kernel SVM based multiple-task oriented data mining system for gene expression data analysis , 2011, Expert Syst. Appl..

[4]  Dieter Fox,et al.  Object recognition with hierarchical kernel descriptors , 2011, CVPR 2011.

[5]  D. Comaniciu,et al.  The variable bandwidth mean shift and data-driven scale selection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  S. Ramaswamy,et al.  Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma. , 2002, Cancer research.

[7]  Ujjwal Maulik,et al.  Improvement of new automatic differential fuzzy clustering using SVM classifier for microarray analysis , 2011, Expert Syst. Appl..

[8]  Verónica Bolón-Canedo,et al.  A review of feature selection methods on synthetic data , 2013, Knowledge and Information Systems.

[9]  Jenq-Neng Hwang,et al.  Nonparametric multivariate density estimation: a comparative study , 1994, IEEE Trans. Signal Process..

[10]  A. B. M. Shawkat Ali,et al.  Computational intelligence for microarray data and biomedical image analysis for the early diagnosis of breast cancer , 2012, Expert Syst. Appl..

[11]  Hai-Li Liang,et al.  Applying particle swarm optimization to determine the bandwidth parameter in probability density estimation , 2011, 2011 International Conference on Machine Learning and Cybernetics.

[12]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[13]  Alvaro Soto,et al.  Embedded local feature selection within mixture of experts , 2014, Inf. Sci..

[14]  Jens Kersten,et al.  Simultaneous feature selection and Gaussian mixture model estimation for supervised classification problems , 2014, Pattern Recognit..

[15]  Huilin Xiong,et al.  Kernel-based distance metric learning for microarray data classification , 2006, BMC Bioinformatics.

[16]  Xiaolong Wang,et al.  Combining evolutionary information extracted from frequency profiles with sequence-based kernels for protein remote homology detection , 2013, Bioinform..

[17]  Hans-Peter Seidel,et al.  Nonparametric Density Estimation with Adaptive, Anisotropic Kernels for Human Motion Tracking , 2007, Workshop on Human Motion.

[18]  Hemant Ishwaran,et al.  Evaluating Random Forests for Survival Analysis using Prediction Error Curves. , 2012, Journal of statistical software.

[19]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[20]  Daniel J Schaid,et al.  Genomic Similarity and Kernel Methods II: Methods for Genomic Information , 2010, Human Heredity.

[21]  Yuzhen Niu,et al.  Fast Gaussian kernel learning for classification tasks based on specially structured global optimization , 2014, Neural Networks.

[22]  Ron Shamir,et al.  Network-induced Classification Kernels for Gene Expression Profile Analysis , 2012 .

[23]  Nir Friedman,et al.  Tissue classification with gene expression profiles. , 2000 .

[24]  Huan Liu,et al.  Redundancy based feature selection for microarray data , 2004, KDD.

[25]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[27]  Ivor W. Tsang,et al.  A Family of Simple Non-Parametric Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[28]  Anita Bai.  Multiobjective Clustering Using Support Vector Machine: Application to Microarray Cancer Data , 2013, ICACNI.

[29]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[30]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[31]  Alper Ekrem Murat,et al.  A discrete particle swarm optimization method for feature selection in binary classification problems , 2010, Eur. J. Oper. Res..

[32]  Roman Rouzier,et al.  Non-parametric kernel density estimation for the prediction of neoadjuvant chemotherapy outcomes , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[33]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[34]  Nikos Paragios,et al.  Motion-based background subtraction using adaptive kernel density estimation , 2004, CVPR 2004.

[35]  Zhen Liu,et al.  A new feature selection algorithm based on binomial hypothesis testing for spam filtering , 2011, Knowl. Based Syst..

[36]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[37]  Gajendra P S Raghava,et al.  SVM based prediction of RNA‐binding proteins using binding residues and evolutionary information , 2011, Journal of molecular recognition : JMR.

[38]  Hao Helen Zhang.  Variable selection for support vector machines via smoothing spline anova , 2006 .

[39]  Graciela Estévez-Pérez,et al.  Nonparametric Kernel Distribution Function Estimation with kerdiest: An R Package for Bandwidth Choice and Applications , 2012 .

[40]  Dayou Liu,et al.  A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis , 2011, Expert Syst. Appl..

[41]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[42]  Dongyan Zhao,et al.  Two-stage multiple kernel learning with multiclass kernel polarization , 2013, Knowl. Based Syst..

[43]  Jin-Kao Hao,et al.  A Hybrid GA/SVM Approach for Gene Selection and Classification of Microarray Data , 2006, EvoWorkshops.

[44]  Michael K. Ng,et al.  Feature weight estimation for gene selection: a local hyperlinear learning approach , 2014, BMC Bioinformatics.

[45]  Yantao Wei,et al.  Similarity learning for object recognition based on derived kernel , 2012, Neurocomputing.

[46]  Jean Ponce,et al.  A graph-matching kernel for object categorization , 2011, 2011 International Conference on Computer Vision.

[47]  Tai-Yue Wang,et al.  One-against-one fuzzy support vector machine classifier: An approach to text categorization , 2009, Expert Syst. Appl..

[48]  Daniel Q. Naiman,et al.  Simple decision rules for classifying human cancers from gene expression profiles , 2005, Bioinform..

[49]  Yonghong Peng,et al.  A novel feature selection approach for biomedical data classification , 2010, J. Biomed. Informatics.

[50]  William Stafford Noble,et al.  Kernel methods for predicting protein-protein interactions , 2005, ISMB.

[51]  Bernardete Ribeiro,et al.  Distributed Text Classification With an Ensemble Kernel-Based Learning Approach , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[52]  Zhen Ji,et al.  Gabor Wavelet Selection and SVM Classification for Object Recognition , 2009 .

[53]  Xiao-hui Niu,et al.  Predicting DNA binding proteins using support vector machine with hybrid fractal features. , 2014, Journal of theoretical biology.

[54]  Leila Hamdad,et al.  A kernel spatial density estimation allowing for the analysis of spatial clustering. Application to Monsoon Asia Drought Atlas data , 2014, Stochastic Environmental Research and Risk Assessment.

[55]  C. Chandrasekar,et al.  Object Recognition using SVM-KNN based on Geometric Moment Invariant , 2011 .

[56]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[57]  Yong Xu,et al.  Neuro-Fuzzy Ensemble Approach for Microarray Cancer Gene Expression Data Analysis , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[58]  Danijel Skocaj,et al.  Multivariate online kernel density estimation with Gaussian kernels , 2011, Pattern Recognit..

[59]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[60]  Parameswaran Ramachandran,et al.  Adaptive bandwidth kernel density estimation for next-generation sequencing data , 2013, BMC Proceedings.

[61]  K. B. Kulasekera,et al.  Density estimation using asymmetric kernels and Bayes bandwidths with censored data , 2010 .

[62]  Colin Campbell,et al.  Kernel methods: a survey of current techniques , 2002, Neurocomputing.

[63]  Austin H Chen,et al.  The Prediction of Cancer Classification using a Novel Multi-task Support Vector Sample Learning Technique , 2011 .

[64]  Saleh Alsaleem,et al.  Automated Arabic Text Categorization Using SVM and NB , 2011, Int. Arab. J. e Technol..

[65]  Faisal Shafait,et al.  Efficient feature size reduction via predictive forward selection , 2014, Pattern Recognit..

[66]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[67]  Clayton D. Scott,et al.  Robust kernel density estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[68]  Razieh Sheikhpour,et al.  Particle swarm optimization for bandwidth determination and feature selection of kernel density estimation based classifiers in diagnosis of breast cancer , 2016, Appl. Soft Comput..

[69]  Dario Farina,et al.  A non-parametric Bayesian approach for clustering and tracking non-stationarities of neural spikes , 2014, Journal of Neuroscience Methods.

[70]  Haiyan Wang,et al.  Improving accuracy for cancer classification with a new algorithm for genes selection , 2012, BMC Bioinformatics.