Entropy based fuzzy least squares twin support vector machine for class imbalance learning

In classification problems, the data samples belonging to different classes have different number of samples. Sometimes, the imbalance in the number of samples of each class is very high and the interest is to classify the samples belonging to the minority class. Support vector machine (SVM) is one of the widely used techniques for classification problems which have been applied for solving this problem by using fuzzy based approach. In this paper, motivated by the work of Fan et al. (Knowledge-Based Systems 115: 87–99 2017), we have proposed two efficient variants of entropy based fuzzy SVM (EFSVM). By considering the fuzzy membership value for each sample, we have proposed an entropy based fuzzy least squares support vector machine (EFLSSVM-CIL) and entropy based fuzzy least squares twin support vector machine (EFLSTWSVM-CIL) for class imbalanced datasets where fuzzy membership values are assigned based on entropy values of samples. It solves a system of linear equations as compared to the quadratic programming problem (QPP) as in EFSVM. The least square versions of the entropy based SVM are faster than EFSVM and give higher generalization performance which shows its applicability and efficiency. Experiments are performed on various real world class imbalanced datasets and compared the results of proposed methods with new fuzzy twin support vector machine for pattern classification (NFTWSVM), entropy based fuzzy support vector machine (EFSVM), fuzzy twin support vector machine (FTWSVM) and twin support vector machine (TWSVM) which clearly illustrate the superiority of the proposed EFLSTWSVM-CIL.

[1]  Yanxi Liu,et al.  Cervical Cancer Detection Using SVM Based Feature Screening , 2004, MICCAI.

[2]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[3]  Kin Keung Lai,et al.  A new fuzzy support vector machine to evaluate credit risk , 2005, IEEE Transactions on Fuzzy Systems.

[4]  Minghu Ha,et al.  Fuzzy support vector machine based on non-equilibrium data , 2012, 2012 International Conference on Machine Learning and Cybernetics.

[5]  Hongyuan Zha,et al.  Entropy-based fuzzy support vector machine for imbalanced datasets , 2017, Knowl. Based Syst..

[6]  Eugene Borovikov.  An Evaluation of Support Vector Machines as a Pattern Recognition Tool , 2014, ArXiv.

[7]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Herbert Gish,et al.  Speaker identification via support vector classifiers , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[9]  Sonali Agarwal,et al.  Hybrid Feature Selection Based Weighted Least Squares Twin Support Vector Machine Approach for Diagnosing Breast Cancer, Hepatitis, and Diabetes , 2015, Adv. Artif. Neural Syst..

[10]  S. Balasundaram,et al.  A new approach for training Lagrangian twin support vector machine via unconstrained convex minimization , 2016, Applied Intelligence.

[11]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[12]  Yuan-Hai Shao,et al.  Weighted linear loss twin support vector machine for large-scale classification , 2015, Knowl. Based Syst..

[13]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[15]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[16]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[17]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[18]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[19]  James M. Keller,et al.  Incorporating Fuzzy Membership Functions into the Perceptron Algorithm , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Bhavani M. Thuraisingham,et al.  A new intrusion detection system using support vector machines and hierarchical clustering , 2007, The VLDB Journal.

[21]  Xiao-Jun Wu,et al.  A new fuzzy twin support vector machine for pattern classification , 2018, Int. J. Mach. Learn. Cybern..

[22]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[23]  Reshma Rastogi,et al.  Tree-based localized fuzzy twin support vector clustering with square loss function , 2017, Applied Intelligence.

[24]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[25]  Lei Guo,et al.  Forecasting stock composite index by fuzzy support vector machines regression , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[26]  Johan A. K. Suykens,et al.  Learning solutions to partial differential equations using LS-SVM , 2015, Neurocomputing.

[27]  S. Gunn.  Support Vector Machines for Classification and Regression , 1998 .

[28]  Shigeo Abe,et al.  Fuzzy least squares support vector machines , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[29]  Vo Thi Ngoc Chau,et al.  Fuzzy C-means for english sentiment classification in a distributed system , 2017, Applied Intelligence.

[30]  Divya Tomar,et al.  An Emotion Detection System Based on Multi Least Squares Twin Support Vector Machine , 2014, Adv. Artif. Intell..

[31]  Yuan-Hai Shao,et al.  An efficient weighted Lagrangian twin support vector machine for imbalanced data classification , 2014, Pattern Recognit..

[32]  P. Jonathon Phillips,et al.  Support Vector Machines Applied to Face Recognition , 1998, NIPS.

[33]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[34]  Arindam Chaudhuri,et al.  Fuzzy Support Vector Machine for bankruptcy prediction , 2011, Appl. Soft Comput..

[35]  Shunli Zhang,et al.  Single Object Tracking With Fuzzy Least Squares Support Vector Machine , 2015, IEEE Transactions on Image Processing.

[36]  R. Malhotra,et al.  Evaluating Consumer Loans Using Neural Networks , 2001 .

[37]  Nasser Ghadiri,et al.  Fuzzy Least Squares Twin Support Vector Machines , 2015, Eng. Appl. Artif. Intell..

[38]  Christopher J. C. Burges,et al.  Geometry and invariance in kernel based methods , 1999 .

[39]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[40]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[41]  Xufa Wang,et al.  A Gene-Prediction Algorithm Based on the Statistical Combination and the Classification in Terms of Gene Characteristics , 2005, 2005 International Conference on Neural Networks and Brain.

[42]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[43]  Muhammad Tanveer,et al.  Robust energy-based least squares twin support vector machines , 2015, Applied Intelligence.

[44]  Vasile Palade,et al.  FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning , 2010, IEEE Transactions on Fuzzy Systems.