Instance-based entropy fuzzy support vector machine for imbalanced data

Imbalanced classification has been a major challenge for machine learning because many standard classifiers mainly focus on balanced datasets and tend to have biased results toward the majority class. We modify entropy fuzzy support vector machine (EFSVM) and introduce instance-based entropy fuzzy support vector machine (IEFSVM). Both EFSVM and IEFSVM use the entropy information of k -nearest neighbors to determine the fuzzy membership value for each sample which prioritizes the importance of each sample. IEFSVM considers the diversity of entropy patterns for each sample when increasing the size of neighbors, k , while EFSVM uses single entropy information of the fixed size of neighbors for all samples. By varying k , we can reflect the component change of sample’s neighbors from near to far distance in the determination of fuzzy value membership. Numerical experiments on 35 public and 12 real-world imbalanced datasets are performed to validate IEFSVM, and area under the receiver operating characteristic curve (AUC) is used to compare its performance with other SVMs and machine learning methods. IEFSVM shows a much higher AUC value for datasets with high imbalance ratio, implying that IEFSVM is effective in dealing with the class imbalance problem.

[1]  Feiping Nie,et al.  Probabilistic Labeled Semi-supervised SVM , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[2]  Gretchen G. Moisen,et al.  Evaluating effectiveness of down-sampling for stratified designs and unbalanced prevalence in Random Forest models of tree species distributions in Nevada , 2012 .

[3]  Amit Prakash Singh,et al.  An empirical evaluation of translational and rotational invariance of descriptors and the classification of flower dataset , 2018, Pattern Analysis and Applications.

[4]  Feiping Nie,et al.  Multiclass Capped ℓp-Norm SVM for Robust Classifications , 2017, AAAI.

[5]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[6]  Vasile Palade,et al.  FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning , 2010, IEEE Transactions on Fuzzy Systems.

[7]  Mikel Galar,et al.  Analysing the classification of imbalanced data-sets with multiple classes: Binarization techniques and ad-hoc approaches , 2013, Knowl. Based Syst..

[8]  Xianli Pan,et al.  K-nearest neighbor based structural twin support vector machine , 2015, Knowl. Based Syst..

[9]  Jia Yu,et al.  KNN-based weighted rough ν-twin support vector machine , 2014, Knowl. Based Syst..

[10]  Euntai Kim,et al.  A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function , 2011, Expert Syst. Appl..

[11]  MengChu Zhou,et al.  A Noise-Filtered Under-Sampling Scheme for Imbalanced Classification , 2017, IEEE Transactions on Cybernetics.

[12]  Asifullah Khan,et al.  Intelligent churn prediction in telecom: employing mRMR feature selection and RotBoost based ensemble classification , 2013, Applied Intelligence.

[13]  Yongtao Hao,et al.  A feature weighted support vector machine and K-nearest neighbor algorithm for stock market indices prediction , 2017, Expert Syst. Appl..

[14]  Mostefa Mesbah,et al.  EEG rhythm/channel selection for fuzzy rule-based alertness state characterization , 2016, Neural Computing and Applications.

[15]  Tingting Zheng,et al.  Uncertainty measures of Neighborhood System-based rough sets , 2015, Knowl. Based Syst..

[16]  Jianping Yin,et al.  Boosting weighted ELM for imbalanced learning , 2014, Neurocomputing.

[17]  Hamido Fujita,et al.  Imbalanced enterprise credit evaluation with DTE-SBD: Decision tree ensemble based on SMOTE and bagging with differentiated sampling rates , 2018, Inf. Sci..

[18]  Ömer Faruk Ertugrul,et al.  A novel version of k nearest neighbor: Dependent nearest neighbor , 2017, Appl. Soft Comput..

[19]  Woojin Chang,et al.  Application of Instance-Based Entropy Fuzzy Support Vector Machine in Peer-To-Peer Lending Investment Decision , 2019, IEEE Access.

[20]  Zhe Wang,et al.  Gravitational fixed radius nearest neighbor for imbalanced problem , 2015, Knowl. Based Syst..

[21]  Yidi Wang,et al.  A new k-harmonic nearest neighbor classifier based on the multi-local means , 2017, Expert Syst. Appl..

[22]  Zhi-Hua Zhou,et al.  Exploratory Undersampling for Class-Imbalance Learning , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Jian Yang,et al.  Extended nearest neighbor chain induced instance-weights for SVMs , 2016, Pattern Recognit..

[24]  Valquiria Aparecida Rosa Duarte,et al.  A multiagent player system composed by expert agents in specific game stages operating in high performance environment , 2017, Applied Intelligence.

[25]  Xindong Wu,et al.  10 Challenging Problems in Data Mining Research , 2006, Int. J. Inf. Technol. Decis. Mak..

[26]  Robert B. Fisher,et al.  Classifying imbalanced data sets using similarity based hierarchical decomposition , 2015, Pattern Recognit..

[27]  Francisco Herrera,et al.  An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics , 2013, Inf. Sci..

[28]  Jun Zhang,et al.  Addressing the class imbalance problem in Twitter spam detection using ensemble learning , 2017, Comput. Secur..

[29]  Zhao Li,et al.  Learning from real imbalanced data of 14-3-3 proteins binding specificity , 2016, Neurocomputing.

[30]  Yu Xue,et al.  Measures of uncertainty for neighborhood rough sets , 2017, Knowl. Based Syst..

[31]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[32]  Francisco Herrera,et al.  Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics , 2012, Expert Syst. Appl..

[33]  Jian Yang,et al.  Neighbors' distribution property and sample reduction for support vector machines , 2014, Appl. Soft Comput..

[34]  Jing Zhao,et al.  ACOSampling: An ant colony optimization-based undersampling method for classifying imbalanced DNA microarray data , 2013, Neurocomputing.

[35]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[36]  Chi-Hyuck Jun,et al.  Instance categorization by support vector machines to adjust weights in AdaBoost for imbalanced data classification , 2017, Inf. Sci..

[37]  Cihan Kaleli.  An entropy-based neighbor selection approach for collaborative filtering , 2014, Knowl. Based Syst..

[38]  Mourad Oussalah,et al.  The joint use of sequence features combination and modified weighted SVM for improving daily activity recognition , 2016, Pattern Analysis and Applications.

[39]  Mehmet Fatih Amasyali,et al.  Locally adaptive k parameter selection for nearest neighbor classifier: one nearest cluster , 2017, Pattern Analysis and Applications.

[40]  Deepak Gupta,et al.  Entropy based fuzzy least squares twin support vector machine for class imbalance learning , 2018, Applied Intelligence.

[41]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[42]  Zhi Chen,et al.  Creating diversity in ensembles using synthetic neighborhoods of training samples , 2017, Applied Intelligence.

[43]  Shin Ando.  Classifying imbalanced data in distance-based feature space , 2015, Knowledge and Information Systems.

[44]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[45]  Bart Baesens,et al.  A new transferred feature selection algorithm for customer identification , 2017, Neural Computing and Applications.

[46]  Hong-Liang Dai,et al.  Class imbalance learning via a fuzzy total margin based support vector machine , 2015, Appl. Soft Comput..

[47]  Jacek Tabor,et al.  Extreme entropy machines: robust information theoretic classification , 2015, Pattern Analysis and Applications.

[48]  Charles X. Ling,et al.  Using AUC and accuracy in evaluating learning algorithms , 2005, IEEE Transactions on Knowledge and Data Engineering.

[49]  Francisco Herrera,et al.  An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes , 2011, Pattern Recognit..

[50]  Zhi Chen,et al.  A synthetic neighborhood generation based ensemble learning for the imbalanced data classification , 2017, Applied Intelligence.

[51]  Samia Boukir,et al.  Exploring issues of training data imbalance and mislabelling on random forest performance for large area land cover classification using the ensemble margin , 2015 .

[52]  Xinbo Gao,et al.  Random sampling for fast face sketch synthesis , 2017, Pattern Recognit..

[53]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[55]  Kemal Polat.  Similarity-based attribute weighting methods via clustering algorithms in the classification of imbalanced medical datasets , 2018, Neural Computing and Applications.

[56]  Zahir Tari,et al.  KRNN: k Rare-class Nearest Neighbour classification , 2017, Pattern Recognit..

[57]  Yuming Zhou,et al.  A novel ensemble method for classifying imbalanced data , 2015, Pattern Recognit..

[58]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[59]  Kihoon Yoon,et al.  A data reduction approach for resolving the imbalanced data issue in functional genomics , 2007, Neural Computing and Applications.

[60]  Hongyuan Zha,et al.  Entropy-based fuzzy support vector machine for imbalanced datasets , 2017, Knowl. Based Syst..

[61]  Bart Baesens,et al.  An empirical comparison of techniques for the class imbalance problem in churn prediction , 2017, Inf. Sci..

[62]  Jian Yang,et al.  A weighted one-class support vector machine , 2016, Neurocomputing.

[63]  Yongzhao Zhan,et al.  Improved pseudo nearest neighbor classification , 2014, Knowl. Based Syst..

[64]  Manuel Graña,et al.  Balanced training of a hybrid ensemble method for imbalanced datasets: a case of emergency department readmission prediction , 2017, Neural Computing and Applications.

[65]  Taghi M. Khoshgoftaar,et al.  RUSBoost: A Hybrid Approach to Alleviating Class Imbalance , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[66]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[67]  Ning Ye,et al.  Boundary detection and sample reduction for one-class Support Vector Machines , 2014, Neurocomputing.

[68]  Feiping Nie,et al.  New primal SVM solver with linear computational cost for big data classifications , 2014, ICML 2014.

[69]  Loris Nanni,et al.  Coupling different methods for overcoming the class imbalance problem , 2015, Neurocomputing.

[70]  mer Faruk Erturul,et al.  A novel version of k nearest neighbor , 2017 .

[71]  Deepak Gupta,et al.  A fuzzy twin support vector machine based on information entropy for class imbalance learning , 2019, Neural Computing and Applications.

[72]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[73]  Xinbo Gao,et al.  Data Augmentation-Based Joint Learning for Heterogeneous Face Recognition , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[74]  Hiroshi Inoue,et al.  Data Augmentation by Pairing Samples for Images Classification , 2018, ArXiv.

[75]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[76]  Yang Wang,et al.  Cost-sensitive boosting for classification of imbalanced data , 2007, Pattern Recognit..

[77]  Hong Gu,et al.  Imbalanced classification using support vector machine ensemble , 2011, Neural Computing and Applications.

[78]  Changming Zhu,et al.  Entropy-based matrix learning machine for imbalanced data sets , 2017, Pattern Recognit. Lett..

[79]  Sattar Hashemi,et al.  An entropy-based distance measure for analyzing and detecting metamorphic malware , 2017, Applied Intelligence.

[80]  Zhang Yi,et al.  Fuzzy SVM with a new fuzzy membership function , 2006, Neural Computing & Applications.

[81]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[82]  Ying Huang,et al.  An effective hybrid learning system for telecommunication churn prediction , 2013, Expert Syst. Appl..

[83]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[84]  Pei-Chann Chang,et al.  A novel ensemble decision tree based on under-sampling and clonal selection for web spam detection , 2017, Pattern Analysis and Applications.

[85]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[86]  Krung Sinapiromsaran,et al.  Decision tree induction based on minority entropy for the class imbalance problem , 2017, Pattern Analysis and Applications.

[87]  Xuhui Chen,et al.  An entropy-based uncertainty measurement approach in neighborhood systems , 2014, Inf. Sci..

[88]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[89]  Christophe Mues,et al.  An experimental comparison of classification algorithms for imbalanced credit scoring data sets , 2012, Expert Syst. Appl..