A comparative study on long short-term memory and gated recurrent unit neural networks in fault diagnosis for chemical processes using visualization

Abstract Recurrent neural networks (RNNs), particularly those with gated units, such as long short-term memory (LSTM) and gated recurrent unit (GRU), have demonstrated clear superiority in sequence modeling. In this study, the fault diagnosis performance and classification mechanisms of basic LSTM and GRU were investigated to provide comparative information about suitable fault diagnosis models in chemical processes. Visualization techniques were used to interpret the behavior of LSTM and GRU when performing fault diagnosis in the Tennessee Eastman process (TEP). The analysis indicated that input and output gates were the two main gates of the LSTM model to filter information, while the forget gate of LSTM might be insignificant because it was usually opened to allow information to pass through. Therefore, the GRU model separated the faults better, especially Fault 15, and it provided more promising fault diagnosis performance compared to the LSTM model. The diagnosis accuracy for Fault 15 increased from 63%, while using the LSTM model, to 76% while using the GRU model. The simulation results of the TEP indicated that the GRU neural network in this study was superior to the LSTM neural network.

[1]  Jia-Lin Kang,et al.  Visualization analysis for fault diagnosis in chemical processes using recurrent neural networks , 2020 .

[2]  E. F. Vogel,et al.  A plant-wide industrial process control problem , 1993 .

[3]  Jinsong Zhao,et al.  A new unsupervised data mining method based on the stacked autoencoder for chemical process fault diagnosis , 2020, Comput. Chem. Eng..

[4]  Christos Georgakis,et al.  Disturbance detection and isolation by dynamic principal component analysis , 1995 .

[5]  Yangyang Wang,et al.  Long short-term memory neural network with weight amplification and its application into gear remaining useful life prediction , 2020, Eng. Appl. Artif. Intell..

[6]  Qian Liu,et al.  Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process , 2019, Reliab. Eng. Syst. Saf..

[7]  Danfeng Xie,et al.  A Hierarchical Deep Neural Network for Fault Diagnosis on Tennessee-Eastman Process , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[8]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[9]  Yongming Han,et al.  An optimized long short-term memory network based fault diagnosis model for chemical processes , 2020 .

[10]  Xinyu Shao,et al.  Data-driven remaining useful life prediction via multiple sensor signals and deep long short-term memory neural network. , 2020, ISA transactions.

[11]  Hao Wu,et al.  Deep convolutional neural network model based chemical process fault diagnosis , 2018, Comput. Chem. Eng..

[12]  Jing Yuan,et al.  An Intelligent Fault Diagnosis Method Using GRU Neural Network towards Sequential Data in Dynamic Processes , 2019, Processes.

[13]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[14]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[15]  Richard D. Braatz,et al.  Fault Detection and Identification using Bayesian Recurrent Neural Networks , 2019, Comput. Chem. Eng..

[16]  Chen Jing,et al.  SVM and PCA based fault classification approaches for complicated industrial process , 2015, Neurocomputing.

[17]  Wenjie Zhang,et al.  Data-driven reduced order model with temporal convolutional neural network , 2020 .

[18]  Moisès Graells,et al.  A semi-supervised approach to fault diagnosis for chemical processes , 2010, Comput. Chem. Eng..

[19]  Wei Liu,et al.  A Novel Fault Diagnosis Approach for Chillers Based on 1-D Convolutional Neural Network and Gated Recurrent Unit , 2020, Sensors.

[20]  Manabu Kano,et al.  A new multivariate statistical process monitoring method using principal component analysis , 2001 .

[21]  Yi Qin,et al.  Gated Dual Attention Unit Neural Networks for Remaining Useful Life Prediction of Rolling Bearings , 2021, IEEE Transactions on Industrial Informatics.

[22]  Steven X. Ding,et al.  A Review on Basic Data-Driven Approaches for Industrial Process Monitoring , 2014, IEEE Transactions on Industrial Electronics.

[23]  Maia B. Cook,et al.  Additional Tennessee Eastman Process Simulation Data for Anomaly Detection Evaluation , 2017 .

[24]  Arthur K. Kordon,et al.  Fault diagnosis based on Fisher discriminant analysis and support vector machines , 2004, Comput. Chem. Eng..

[25]  Lianhong Cai,et al.  Question detection from acoustic features using recurrent neural network with gated recurrent unit , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Xiaoqiang Lu,et al.  Auxiliary Loss Multimodal GRU Model in Audio-Visual Speech Recognition , 2018, IEEE Access.

[27]  Bo Jin,et al.  Sequential Fault Diagnosis Based on LSTM Neural Network , 2018, IEEE Access.

[28]  Richard D. Braatz,et al.  Fault Detection and Diagnosis in Industrial Systems , 2001 .

[29]  Leo H. Chiang,et al.  Fault diagnosis in chemical processes using Fisher discriminant analysis, discriminant partial least squares, and principal component analysis , 2000 .

[30]  Takehisa Yairi,et al.  A review on the application of deep learning in system health management , 2018, Mechanical Systems and Signal Processing.

[31]  Hadi Shahnazari,et al.  Fault diagnosis of nonlinear systems using recurrent neural networks , 2020 .

[32]  Yaguo Lei,et al.  Applications of machine learning to machine fault diagnosis: A review and roadmap , 2020 .

[33]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[34]  Jinsong Zhao,et al.  A novel process monitoring approach based on variational recurrent autoencoder , 2019, Comput. Chem. Eng..

[35]  Yoshua Bengio,et al.  Light Gated Recurrent Units for Speech Recognition , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[36]  Robert X. Gao,et al.  Deep heterogeneous GRU model for predictive analytics in smart manufacturing: Application to tool wear prediction , 2019, Comput. Ind..

[37]  Jorge Cadima,et al.  Principal component analysis: a review and recent developments , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[38]  Zhihua Xiong,et al.  Recurrent Neural Network Model with Self-Attention Mechanism for Fault Detection and Diagnosis , 2019, 2019 Chinese Automation Congress (CAC).

[39]  Ping Zhang,et al.  A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process , 2012 .