An asymmetric stagewise least square loss function for imbalanced classification

In this paper, we present an asymmetric stagewise least square (ASLS) loss function for imbalanced classification. While keeping all the advantages of the stagewise least square (SLS) loss function, such as, better robustness, computational efficiency and sparseness, the ASLS loss extends the SLS loss by adding another two parameters, namely, ramp coefficient and margin coefficient. Therefore, asymmetric ramps and margins can be formed which makes the ASLS loss be more flexible and appropriate for processing class imbalance problems. A reduced kernel classifier of the ASLS loss is also developed which only uses a small part of the dataset to generate an efficient nonlinear classifier. Experimental results confirm the effectiveness of the ASLS loss in imbalanced classification.

[1]  Nello Cristianini,et al.  Controlling the Sensitivity of Support Vector Machines , 1999 .

[2]  Stephen Kwek,et al.  Applying Support Vector Machines to Imbalanced Datasets , 2004, ECML.

[3]  Peng Li,et al.  Hybrid Kernel Machine Ensemble for Imbalanced Data Sets , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[4]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Johan A. K. Suykens,et al.  Benchmarking Least Squares Support Vector Machine Classifiers , 2004, Machine Learning.

[6]  Adam Kowalczyk,et al.  Extreme re-balancing for SVMs: a case study , 2004, SKDD.

[7]  José Carlos Príncipe,et al.  The C-loss function for pattern classification , 2014, Pattern Recognit..

[8]  Olvi L. Mangasarian,et al.  Generalized Support Vector Machines , 1998 .

[9]  Michael I. Jordan,et al.  Convexity, Classification, and Risk Bounds , 2006 .

[10]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[11]  Pedro M. Domingos.  MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[12]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[13]  Stan Matwin,et al.  Addressing the Curse of Imbalanced Training Sets: One-Sided Selection , 1997, ICML.

[14]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[15]  Joarder Kamruzzaman,et al.  z-SVM: An SVM for Improved Classification of Imbalanced Data , 2006, Australian Conference on Artificial Intelligence.

[16]  Nuno Vasconcelos,et al.  On the design of robust classifiers for computer vision , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  C. Scott.  Calibrated asymmetric surrogate losses , 2012 .

[18]  Ming-Syan Chen,et al.  On Generalizable Low False-Positive Learning Using Asymmetric Support Vector Machines , 2013, IEEE Transactions on Knowledge and Data Engineering.

[19]  Jianjun Wang,et al.  Margin calibration in SVM class-imbalanced learning , 2009, Neurocomputing.

[20]  Yi-Hung Liu,et al.  Face Recognition Using Total Margin-Based Adaptive Fuzzy Support Vector Machines , 2007, IEEE Transactions on Neural Networks.

[21]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[22]  C. Lee Giles,et al.  Learning on the border: active learning in imbalanced data classification , 2007, CIKM '07.

[23]  John Shawe-Taylor,et al.  Refining Kernels for Regression and Uneven Classification Problems , 2003, AISTATS.

[24]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[25]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[26]  Yanqing Zhang,et al.  SVMs Modeling for Highly Imbalanced Classification , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[28]  Glenn Fung,et al.  Multicategory Proximal Support Vector Machine Classifiers , 2005, Machine Learning.

[29]  Vasile Palade,et al.  FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning , 2010, IEEE Transactions on Fuzzy Systems.

[30]  Peter Williams,et al.  A Geometrical Method to Improve Performance of the Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[31]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[32]  Bao-Gang Hu,et al.  A New Strategy of Cost-Free Learning in the Class Imbalance Problem , 2014, IEEE Transactions on Knowledge and Data Engineering.

[33]  Edward Y. Chang,et al.  KBA: kernel boundary alignment considering imbalanced data distribution , 2005, IEEE Transactions on Knowledge and Data Engineering.

[34]  Shuang-Hong Yang,et al.  A Stagewise Least Square Loss Function for Classification , 2008, SDM.

[35]  Charles Elkan,et al.  The Foundations of Cost-Sensitive Learning , 2001, IJCAI.

[36]  Andy J. Keane,et al.  Some Greedy Learning Algorithms for Sparse Regression and Classification with Mercer Kernels , 2003, J. Mach. Learn. Res..

[37]  W. Wong,et al.  On ψ-Learning , 2003 .

[38]  Bao-Gang Hu,et al.  Cost-Free Learning for Support Vector Machines with a Reject Option , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[39]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[40]  Yunqian Ma,et al.  Foundations of Imbalanced Learning , 2013 .

[41]  Yunqian Ma,et al.  Imbalanced Learning: Foundations, Algorithms, and Applications , 2013 .

[42]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..