Cost sensitive active learning using bidirectional gated recurrent neural networks for imbalanced fault diagnosis

Abstract Most existing fault diagnosis methods may fail in the following three scenarios: (1) serial correlations exist in the process data; (2) fault data are much less than normal data; and (3) it is impractical to obtain enough labeled data. In this paper, a novel form of the bidirectional gated recurrent unit (BGRU) is developed to underpin effective and efficient fault diagnosis using cost sensitive active learning. Specifically, BGRU is devised to consider the dynamic behavior of a complex process. In the training phase of BGRU, the idea of weighting each training example is proposed to reduce the effect of class imbalance. Besides, in order to explore the unlabeled data, cost sensitive active learning is utilized to select the candidate instances. The effectiveness of the proposed method is evaluated on the Tennessee Eastman (TE) dataset and a real plasma etching process dataset. The experiment results show that the proposed cost senstive active learning bidirectional gated recurrent unit (CSALBGRU) method achieves better performance in both binary fault diagnosis and multi-class fault diagnosis.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[3]  Ying Xu,et al.  Fault detection of multimode non-Gaussian dynamic process using dynamic Bayesian independent component analysis , 2016, Neurocomputing.

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Fouzi Harrou,et al.  An Improved Multivariate Chart Using Partial Least Squares With Continuous Ranked Probability Score , 2018, IEEE Sensors Journal.

[6]  Biao Huang,et al.  Mixtures of Probabilistic PCA With Common Structure Latent Bases for Process Monitoring , 2019, IEEE Transactions on Control Systems Technology.

[7]  Jicong Fan,et al.  Fault detection and diagnosis of non-linear non-Gaussian dynamic processes using kernel dynamic independent component analysis , 2014, Inf. Sci..

[8]  Hao Wu,et al.  Deep convolutional neural network model based chemical process fault diagnosis , 2018, Comput. Chem. Eng..

[9]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[10]  Hsuan-Tien Lin,et al.  Active Learning for Multiclass Cost-Sensitive Classification Using Probabilistic Models , 2013, 2013 Conference on Technologies and Applications of Artificial Intelligence.

[11]  G. Rong,et al.  Dynamic fault diagnosis using extended matrix and tensor locality preserving discriminant analysis , 2012 .

[12]  Haibo He,et al.  ADASYN: Adaptive synthetic sampling approach for imbalanced learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[13]  Lili Yin,et al.  Incorporate active learning to semi-supervised industrial fault classification , 2019, Journal of Process Control.

[14]  Jinrui Wang,et al.  A renewable fusion fault diagnosis network for the variable speed conditions under unbalanced samples , 2020, Neurocomputing.

[15]  Li Xiao,et al.  Applications of a Strong Track Filter and LDA for On-Line Identification of a Switched Reluctance Machine Stator Inter-Turn Shorted-Circuit Fault , 2019 .

[16]  Richard F. Reidy,et al.  Handbook for cleaning for semiconductor manufacturing : fundamentals and applications , 2011 .

[17]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[18]  Zhiqiang Ge,et al.  Semi-supervised fault classification based on dynamic Sparse Stacked auto-encoders model , 2017 .

[19]  E. F. Vogel,et al.  A plant-wide industrial process control problem , 1993 .

[20]  Seoung Bum Kim,et al.  Process monitoring using variational autoencoder for high-dimensional nonlinear processes , 2019, Eng. Appl. Artif. Intell..

[21]  Chenglin Wen,et al.  Deep learning fault diagnosis method based on global optimization GAN for unbalanced data , 2020, Knowl. Based Syst..

[22]  Fernando Nogueira,et al.  Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning , 2016, J. Mach. Learn. Res..

[23]  Arthur K. Kordon,et al.  Fault diagnosis based on Fisher discriminant analysis and support vector machines , 2004, Comput. Chem. Eng..

[24]  Taghi M. Khoshgoftaar,et al.  RUSBoost: A Hybrid Approach to Alleviating Class Imbalance , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[25]  Luis Puigjaner,et al.  Performance assessment of a novel fault diagnosis system based on support vector machines , 2009, Comput. Chem. Eng..

[26]  Zhiqiang Ge,et al.  Semi-supervised Fisher discriminant analysis model for fault classification in industrial processes , 2014 .

[27]  Peng Jiang,et al.  Fault Diagnosis Based on Chemical Sensor Data with an Active Deep Neural Network , 2016, Sensors.

[28]  Peng Jiang,et al.  An Imbalance Modified Deep Neural Network With Dynamical Incremental Learning for Chemical Fault Diagnosis , 2019, IEEE Transactions on Industrial Electronics.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Chudong Tong,et al.  Ensemble modified independent component analysis for enhanced non-Gaussian process monitoring , 2017 .

[31]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[32]  Jinsong Zhao,et al.  A novel process monitoring approach based on variational recurrent autoencoder , 2019, Comput. Chem. Eng..

[33]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[34]  Xin Yao,et al.  MWMOTE--Majority Weighted Minority Oversampling Technique for Imbalanced Data Set Learning , 2014 .

[35]  Tiago J. Rato,et al.  Fault detection in the Tennessee Eastman benchmark process using dynamic principal components analysis based on decorrelated residuals (DPCA-DR) , 2013 .

[36]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[37]  Gary King,et al.  Logistic Regression in Rare Events Data , 2001, Political Analysis.

[38]  Qiang Liu,et al.  A Modified Dynamic PLS for Quality Related Monitoring of Fractionation Processes , 2018 .

[39]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[40]  Jesse H. Krijthe,et al.  RSSL: Semi-supervised Learning in R , 2016, RRPR@ICPR.

[41]  Minping Jia,et al.  Deep Laplacian Auto-encoder and its application into imbalanced fault diagnosis of rotating machinery , 2020 .

[42]  Xiang Li,et al.  Diagnosing Rotating Machines With Weakly Supervised Data Using Deep Transfer Learning , 2020, IEEE Transactions on Industrial Informatics.

[43]  Sirish L. Shah,et al.  Fault detection and diagnosis in process data using one-class support vector machines , 2009 .

[44]  H. Karimi,et al.  Study on Support Vector Machine-Based Fault Detection in Tennessee Eastman Process , 2014 .

[45]  S. Joe Qin,et al.  A novel dynamic PCA algorithm for dynamic data modeling and process monitoring , 2017 .

[46]  Xu Li,et al.  Machinery fault diagnosis with imbalanced data using deep generative adversarial networks , 2020 .

[47]  Christos Georgakis,et al.  Plant-wide control of the Tennessee Eastman problem , 1995 .