Least squares support vector machine for class imbalance learning and their applications to fault detection of aircraft engine

Abstract Imbalanced problems often occur when the size of majority class is bigger than that of the minority one. The Least squares support vector machine (LSSVM) is an effective method for solving classification problem on balanced datasets. However, LSSVM has bad performance on minority class facing with class imbalance learning for the classification boundary skewing toward the majority class. In order to overcome the drawback, LSSVM for class imbalance learning (LSSVM-CIL) is proposed. LSSVM-CIL utilizes two different regularization parameters C + and C − that evaluate different misclassification costs. Furthermore, a method of combining reduced technique and recursive strategy is proposed to reduce the size of support vectors and retain representative samples. In addition, decomposition of the matrices via Cholesky factorization is employed as a solution to enhance the computational stability. Furthermore, the effectiveness of the two algorithms presented in this paper is confirmed with experimental results on various real-world imbalanced datasets. Fault detection of aircraft engine can be regarded as a CIL problem and has the demand for the real time. Finally, experiments on aircraft engine indicate that the two algorithms can be selected as candidate techniques for fault detection of aircraft engine.

[1]  Feng Lu,et al.  Data-Driven Based Gas Path Fault Diagnosis for Turbo-Shaft Engine , 2012 .

[2]  Joarder Kamruzzaman,et al.  z-SVM: An SVM for Improved Classification of Imbalanced Data , 2006, Australian Conference on Artificial Intelligence.

[3]  Khashayar Khorasani,et al.  Fault detection and isolation of a dual spool gas turbine engine using dynamic neural networks and multiple model approach , 2014, Inf. Sci..

[4]  Q. Henry Wu,et al.  Association Rule Mining-Based Dissolved Gas Analysis for Fault Diagnosis of Power Transformers , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Donald L. Simon,et al.  Application of a Bank of Kalman Filters for Aircraft Engine Fault Diagnostics , 2003 .

[6]  Gary M. Weiss Mining with rarity: a unifying framework , 2004, SKDD.

[7]  Deepak Gupta,et al.  A fuzzy twin support vector machine based on information entropy for class imbalance learning , 2019, Neural Computing and Applications.

[8]  Deepak Gupta,et al.  Entropy based fuzzy least squares twin support vector machine for class imbalance learning , 2018, Applied Intelligence.

[9]  Khashayar Khorasani,et al.  Dynamic neural network-based fault diagnosis of gas turbine engines , 2014, Neurocomputing.

[10]  Stephen Kwek,et al.  Applying Support Vector Machines to Imbalanced Datasets , 2004, ECML.

[11]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[12]  S. Borguet,et al.  Comparison of adaptive filters for gas turbine performance monitoring , 2010, J. Comput. Appl. Math..

[13]  Jose Miguel Puerta,et al.  Improving the performance of Naive Bayes multinomial in e-mail foldering by introducing distribution-based balance of datasets , 2011, Expert Syst. Appl..

[14]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[15]  Allan J. Volponi,et al.  Development of an Information Fusion System for Engine Diagnostics and Health Management , 2013 .

[16]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[17]  S. Balasundaram,et al.  Lagrangian support vector regression via unconstrained convex minimization , 2014, Neural Networks.

[18]  Robert Sabourin,et al.  Iterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs , 2010, Pattern Recognit..

[19]  J.S.H. Tsai,et al.  A boundary method for outlier detection based on support vector domain description , 2009, Pattern Recognit..

[20]  Adam Kowalczyk,et al.  Extreme re-balancing for SVMs: a case study , 2004, SKDD.

[21]  Don R. Hush,et al.  Network constraints and multi-objective optimization for one-class classification , 1996, Neural Networks.

[22]  Nello Cristianini,et al.  Controlling the Sensitivity of Support Vector Machines , 1999 .

[23]  Wuyang Dai,et al.  Practical Conditions for Effectiveness of the Universum Learning , 2011, IEEE Transactions on Neural Networks.

[24]  Marius Kloft,et al.  Toward Supervised Anomaly Detection , 2014, J. Artif. Intell. Res..

[25]  Igor Loboda,et al.  Neural Networks for Gas Turbine Fault Identification: Multilayer Perceptron or Radial Basis Network? , 2011 .

[26]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[27]  Malik Yousef,et al.  One-Class SVMs for Document Classification , 2002, J. Mach. Learn. Res..

[28]  Young-Sik Choi,et al.  Least squares one-class support vector machine , 2009, Pattern Recognit. Lett..

[29]  Hongyuan Zha,et al.  Entropy-based fuzzy support vector machine for imbalanced datasets , 2017, Knowl. Based Syst..

[30]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[31]  Feng Xue,et al.  Jet engine gas path fault diagnosis using dynamic fusion of multiple classifiers , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[32]  Jianguo Sun,et al.  Recursive reduced least squares support vector regression , 2009, Pattern Recognit..

[33]  Vasile Palade,et al.  FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning , 2010, IEEE Transactions on Fuzzy Systems.

[34]  Yi-Hung Liu,et al.  Face Recognition Using Total Margin-Based Adaptive Fuzzy Support Vector Machines , 2007, IEEE Transactions on Neural Networks.

[35]  Hui Han,et al.  Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning , 2005, ICIC.

[36]  Yong Zhang,et al.  Fault classifier of rotating machinery based on weighted support vector data description , 2009, Expert Syst. Appl..

[37]  Haydemar Núñez,et al.  GSVM: An SVM for handling imbalanced accuracy between classes inbi-classification problems , 2014, Appl. Soft Comput..

[38]  Kenneth Lange Karush-Kuhn-Tucker Theory , 2013 .

[39]  Xiaowei Yang,et al.  Several SVM Ensemble Methods Integrated with Under-Sampling for Imbalanced Data Learning , 2009, ADMA.

[40]  Davide Anguita,et al.  The 'K' in K-fold Cross Validation , 2012, ESANN.

[41]  Paul Jen-Hwa Hu,et al.  A cost-sensitive technique for positive-example learning supporting content-based product recommendations in B-to-C e-commerce , 2012, Decis. Support Syst..

[42]  Xian-Da Zhang,et al.  Matrix Analysis and Applications , 2017 .

[43]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[44]  Licheng Jiao,et al.  Fast Sparse Approximation for Least Squares Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[45]  Ralescu Anca,et al.  ISSUES IN MINING IMBALANCED DATA SETS - A REVIEW PAPER , 2005 .

[46]  Zhi-Bo Zhu,et al.  Fault diagnosis based on imbalance modified kernel Fisher discriminant analysis , 2010 .

[47]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[48]  Johan A. K. Suykens,et al.  Sparse approximation using least squares support vector machines , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[49]  Donghua Zhou,et al.  Total projection to latent structures for process monitoring , 2009 .

[50]  Xin Yao,et al.  Ieee Transactions on Knowledge and Data Engineering 1 Relationships between Diversity of Classification Ensembles and Single-class Performance Measures , 2022 .

[51]  Hong-Liang Dai The fuzzy Laplacianclassifier , 2013, Neurocomputing.

[52]  K. Khorasani,et al.  Fault detection and isolation of gas turbine engines using a bank of neural networks , 2015 .

[53]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[54]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[55]  Nitesh V. Chawla,et al.  Editorial: special issue on learning from imbalanced data sets , 2004, SKDD.

[56]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..