Robust SVM for Cost-Sensitive Learning

Although the performance of cost-sensitive support vector machine (CS-SVM) has been demonstrated to approximate to the cost-sensitive Bayes risk, previous CS-SVM methods still suffer from the influence of outlier samples and redundant features. Recently, a few studies have focused on separately solving these two issues by the sparse theory. In this paper, we propose a new robust cost-sensitive support vector machine to simultaneously solve them in a unified framework. To do this, we employ robust statistics and sparse theory, respectively, to take the sample importance and the feature importance into account, for avoiding the influence of outliers and redundant features. Furthermore, we propose a new optimization method to solve the primal problem of our proposed objective function. Experimental results on synthetic and real data sets show that our proposed method outperforms all the comparison methods in terms of cost-sensitive classification.

[1]  Victor S. Sheng,et al.  Thresholding for Making Classifiers Cost-sensitive , 2006, AAAI.

[2]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[3]  Bin Gu,et al.  Chunk incremental learning for cost-sensitive hinge loss support vector machine , 2018, Pattern Recognit..

[4]  Bin Gu,et al.  Bi-Parameter Space Partition for Cost-Sensitive SVM , 2015, IJCAI.

[5]  Feiping Nie,et al.  Multiclass Capped ℓp-Norm SVM for Robust Classifications , 2017, AAAI.

[6]  Heng Tao Shen,et al.  Heterogeneous data fusion for predicting mild cognitive impairment conversion , 2021, Inf. Fusion.

[7]  Maya R. Gupta,et al.  Learning kernels from indefinite similarities , 2009, ICML '09.

[8]  Yonghua Zhu,et al.  Unsupervised Spectral Feature Selection With Dynamic Hyper-Graph Learning , 2022, IEEE Transactions on Knowledge and Data Engineering.

[9]  Xinyu Luo,et al.  Cost-Sensitive Convolution based Neural Networks for Imbalanced Time-Series Classification , 2018, ArXiv.

[10]  Yen-Liang Chen,et al.  Time-constrained cost-sensitive decision tree induction , 2016, Inf. Sci..

[11]  Shichao Zhang,et al.  Cost-sensitive KNN classification , 2020, Neurocomputing.

[12]  Deepak Gupta,et al.  Functional iterative approaches for solving support vector classification problems based on generalized Huber loss , 2019, Neural Computing and Applications.

[13]  Zhi-Hua Zhou,et al.  Sequence-Based Prediction of microRNA-Binding Residues in Proteins Using Cost-Sensitive Laplacian Support Vector Machines , 2013, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[14]  Yu Song,et al.  Multiple Indefinite Kernel Learning for Feature Selection , 2017, IJCAI.

[15]  Xuelong Li,et al.  Graph PCA Hashing for Similarity Search , 2017, IEEE Transactions on Multimedia.

[16]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[17]  Yong Luo,et al.  Cost-Sensitive Feature Selection via F-Measure Optimization Reduction , 2017, AAAI.

[18]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[19]  Nuno Vasconcelos,et al.  Risk minimization, probability elicitation, and cost-sensitive SVMs , 2010, ICML.

[20]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[21]  Zhi-Hua Zhou,et al.  Cost-Sensitive Semi-Supervised Support Vector Machine , 2010, AAAI.

[22]  Zenglin Xu,et al.  Partition level multiview subspace clustering , 2019, Neural Networks.

[23]  Dazhe Zhao,et al.  An Optimized Cost-Sensitive SVM for Imbalanced Data Learning , 2013, PAKDD.

[24]  Xiaofeng Zhu,et al.  Robust SVM with adaptive graph learning , 2019, World Wide Web.

[25]  J. Bourgain,et al.  Toward a unified theory of sparse dimensionality reduction in Euclidean space , 2015 .

[26]  Nuno Vasconcelos,et al.  Cost-Sensitive Support Vector Machines , 2012, Neurocomputing.

[27]  Emilio Carrizosa,et al.  Cost-sensitive Feature Selection for Support Vector Machines , 2019, Comput. Oper. Res..

[28]  Taghi M. Khoshgoftaar,et al.  A Comparative Study of Data Sampling and Cost Sensitive Learning , 2008, 2008 IEEE International Conference on Data Mining Workshops.

[29]  Akiko Takeda,et al.  Robust Cost Sensitive Support Vector Machine , 2015, AISTATS.

[30]  D. Shen,et al.  Joint prediction and time estimation of COVID-19 developing severe symptoms using chest CT scan , 2020, Medical Image Analysis.

[31]  Zhi-Hua Zhou,et al.  ON MULTI‐CLASS COST‐SENSITIVE LEARNING , 2006, Comput. Intell..

[32]  Xiaofeng Zhu,et al.  Unsupervised feature selection by self-paced learning regularization , 2020, Pattern Recognit. Lett..

[33]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[34]  Richard Weber,et al.  Feature selection for high-dimensional class-imbalanced data sets using Support Vector Machines , 2014, Inf. Sci..

[35]  Martin J. Wainwright,et al.  A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers , 2009, NIPS.

[36]  Feiping Nie,et al.  Feature Selection via Scaling Factor Integrated Multi-Class Support Vector Machines , 2017, International Joint Conference on Artificial Intelligence.

[37]  Pedro M. Domingos.  MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[38]  Shichao Zhang,et al.  Spectral clustering via half-quadratic optimization , 2019, World Wide Web.

[39]  Chee Peng Lim,et al.  Feature selection based on brain storm optimization for data classification , 2019, Appl. Soft Comput..

[40]  Raymond H. Chan,et al.  The Equivalence of Half-Quadratic Minimization and the Gradient Linearization Iteration , 2007, IEEE Transactions on Image Processing.

[41]  Stuart Geman,et al.  Statistical methods for tomographic image reconstruction , 1987 .

[42]  Xiaohong Chen,et al.  Solving Indefinite Kernel Support Vector Machine with Difference of Convex Functions Programming , 2017, AAAI.

[43]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[44]  Min Wu,et al.  Adaptive Cost-Sensitive Online Classification , 2018, IEEE Transactions on Knowledge and Data Engineering.

[45]  Jonathan T. Barron,et al.  A General and Adaptive Robust Loss Function , 2017, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Feiping Nie,et al.  Robust and Efficient Linear Discriminant Analysis With L2,1-Norm for Feature Selection , 2020, IEEE Access.

[47]  Feiping Nie,et al.  New primal SVM solver with linear computational cost for big data classifications , 2014, ICML 2014.

[48]  Shichao Zhang,et al.  Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Kenji Fukumizu,et al.  Influence function and robust variant of kernel canonical correlation analysis , 2017, Neurocomputing.