Robust Rescaled Hinge Loss Twin Support Vector Machine for Imbalanced Noisy Classification

Support vector machine (SVM) and twin SVM (TWSVM) are sensitive to the noisy classification, due to the unlimited measures in their losses, especially for imbalanced classification problem. In this paper, by combining the advantages of the correntropy induced loss function (C-Loss) and the hinge loss function (hinge loss), we introduce the rescaled hinge loss function (Rhinge loss), which is a monotonic, bounded, and nonconvex loss, into TWSVM for imbalanced noisy classification, called RTBSVM. We show that the Rhinge loss could approximate the hard margin loss and the hinge loss by adjusting the rescaled parameter, and further, our RTBSVM could improve the stability and performance of TWSVM and it is effective for imbalanced noisy classification. The experimental results show that our method performs better than the compared TWSVMs and robust SVMs on the imbalanced noisy classification.

[1]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[2]  Zheng Cao,et al.  Robust support vector machines based on the rescaled hinge loss function , 2017, Pattern Recognition.

[3]  Zhi-Hua Zhou,et al.  Exploratory Under-Sampling for Class-Imbalance Learning , 2006, Sixth International Conference on Data Mining (ICDM'06).

[4]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[6]  Mila Nikolova,et al.  Analysis of Half-Quadratic Minimization Methods for Signal and Image Recovery , 2005, SIAM J. Sci. Comput..

[7]  Nello Cristianini,et al.  Controlling the Sensitivity of Support Vector Machines , 1999 .

[8]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[9]  Krishna Mohan Buddhiraju,et al.  Classification of Hyperspectral Remote Sensing Images by an Ensemble of Support Vector Machines Under Imbalanced Data , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[10]  Yong Shi,et al.  Ramp loss nonparallel support vector machine for pattern classification , 2015, Knowl. Based Syst..

[11]  Dewei Li,et al.  Multi-view learning based on nonparallel support vector machine , 2018, Knowl. Based Syst..

[12]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[13]  Muhammad Tanveer,et al.  Sparse pinball twin support vector machines , 2019, Appl. Soft Comput..

[14]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[15]  Gustavo E. A. P. A. Batista,et al.  A study of the behavior of several methods for balancing machine learning training data , 2004, SKDD.

[16]  Yitian Xu,et al.  An improved ν-twin bounded support vector machine , 2018, Applied Intelligence.

[17]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[18]  Yuan-Hai Shao,et al.  An efficient weighted Lagrangian twin support vector machine for imbalanced data classification , 2014, Pattern Recognit..

[19]  Yuan-Hai Shao,et al.  Nonparallel hyperplane support vector machine for binary classification problems , 2014, Inf. Sci..

[20]  Yuan-Hai Shao,et al.  A Trace Lasso Regularized Robust Nonparallel Proximal Support Vector Machine for Noisy Classification , 2019, IEEE Access.

[21]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[22]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[23]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[24]  James C. Bezdek,et al.  Some Notes on Alternating Optimization , 2002, AFSS.

[25]  Bernt Schiele,et al.  Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[27]  He Yan,et al.  Least squares twin bounded support vector machines based on L1-norm distance metric for classification , 2018, Pattern Recognit..

[28]  Yanqing Zhang,et al.  SVMs Modeling for Highly Imbalanced Classification , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Yuan-Hai Shao,et al.  Robust Nonparallel Proximal Support Vector Machine With Lp-Norm Regularization , 2018, IEEE Access.

[30]  Suresh Chandra,et al.  Robust Parametric Twin Support Vector Machine for Pattern Classification , 2018, Neural Processing Letters.

[31]  María José del Jesús,et al.  A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets , 2008, Fuzzy Sets Syst..

[32]  William Stafiord Noble,et al.  Support vector machine applications in computational biology , 2004 .

[33]  Euntai Kim,et al.  A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function , 2011, Expert Syst. Appl..

[34]  Yong Shi,et al.  Ramp Loss Linear Programming Nonparallel Support Vector Machine , 2016, ICCS.

[35]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[36]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.

[37]  Bernhard Schölkopf,et al.  Support Vector Machine Applications in Computational Biology , 2004 .

[38]  Shifei Ding,et al.  Twin support vector machines based on fruit fly optimization algorithm , 2016, Int. J. Mach. Learn. Cybern..

[39]  José Carlos Príncipe,et al.  The C-loss function for pattern classification , 2014, Pattern Recognit..

[40]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[41]  James C. Bezdek,et al.  Convergence of Alternating Optimization , 2003, Neural Parallel Sci. Comput..

[42]  Yu Xue,et al.  Weighted linear loss multiple birth support vector machine based on information granulation for multi-class classification , 2017, Pattern Recognit..

[43]  Yuan-Hai Shao,et al.  Weighted linear loss twin support vector machine for large-scale classification , 2015, Knowl. Based Syst..

[44]  Yuan-Hai Shao,et al.  Probabilistic outputs for twin support vector machines , 2012, Knowl. Based Syst..

[45]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Theodore B. Trafalis,et al.  Support vector machine for regression and applications to financial forecasting , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[47]  Yi-Min Huang,et al.  Weighted support vector machine for classification with uneven training class sizes , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[48]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[49]  C. M. Bishop,et al.  Improvements on Twin Support Vector Machines , 2011 .

[50]  Glenn Fung,et al.  Multicategory Proximal Support Vector Machine Classifiers , 2005, Machine Learning.

[51]  Zhi-Hua Zhou,et al.  Exploratory Undersampling for Class-Imbalance Learning , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[52]  Shifei Ding,et al.  An improved multiple birth support vector machine for pattern classification , 2017, Neurocomputing.

[53]  Yuan-Hai Shao,et al.  Robust L1-norm non-parallel proximal support vector machine , 2016 .

[54]  Johan A. K. Suykens,et al.  Benchmarking Least Squares Support Vector Machine Classifiers , 2004, Machine Learning.

[55]  Yaping Lin,et al.  Synthetic minority oversampling technique for multiclass imbalance problems , 2017, Pattern Recognit..

[56]  Nathalie Japkowicz,et al.  Boosting Support Vector Machines for Imbalanced Data Sets , 2008, ISMIS.

[57]  J. Paul Brooks,et al.  Support Vector Machines with the Ramp Loss and the Hard Margin Loss , 2011, Oper. Res..

[58]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[59]  Zhi-Hua Zhou,et al.  Ieee Transactions on Knowledge and Data Engineering 1 Training Cost-sensitive Neural Networks with Methods Addressing the Class Imbalance Problem , 2022 .