Sparsity and manifold regularized convolutional auto-encoders-based feature learning for fault detection of multivariate processes

Abstract Deep neural networks (DNNs) are popular in process monitoring for its remarkable feature extraction from data. However, the increased dimension and correlation of the process variables degrade performance of these DNNs in feature extraction of data. This paper proposes a sparsity and manifold regularized convolutional auto-encoders (SMRCAE) for fault detection of complex multivariate processes. SMRCAE can learn high-level features from the data in an unsupervised way. A sparsity-and-manifold-regularization term is integrated into the learning procedure of SMRCAE, which allows SMRCAE to perform feature selection and capture intrinsic data information. Moreover, a depthwise separable convolution (dsConv) block is used to reduce the computational cost. Two typical fault detection statistics, namely Hotelling’s T-squared ( T 2 ) and the squared prediction error (SPE), are developed on the feature space and residual space of SMRCAE, respectively. The performance of SMRCAE is evaluated on an industrial benchmark, i.e., Tennessee Eastman process (TEP) and a real process of industrial conveyor belts. The experimental results show the feasibility of SMRCAE in extracting representative features for process fault detection. The average fault detection rate of SMRCAE is 92.03% and 100% on the two cases, respectively.

[1]  Mingyang Wu,et al.  Depthwise separable convolution architectures for plant disease classification , 2019, Comput. Electron. Agric..

[2]  Syed Imtiaz,et al.  Fault detection and diagnosis in process system using artificial intelligence-based cognitive technique , 2020, Comput. Chem. Eng..

[3]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Chang Ouk Kim,et al.  A Convolutional Neural Network for Fault Classification and Diagnosis in Semiconductor Manufacturing Processes , 2017, IEEE Transactions on Semiconductor Manufacturing.

[5]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[6]  Peng Jiang,et al.  Fault Diagnosis Based on Chemical Sensor Data with an Active Deep Neural Network , 2016, Sensors.

[7]  Chudong Tong,et al.  Ensemble modified independent component analysis for enhanced non-Gaussian process monitoring , 2017 .

[8]  Bo Jin,et al.  Sequential Fault Diagnosis Based on LSTM Neural Network , 2018, IEEE Access.

[9]  Ping Zhang,et al.  A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process , 2012 .

[10]  Licheng Jiao,et al.  Dense connection and depthwise separable convolution based CNN for polarimetric SAR image classification , 2020, Knowl. Based Syst..

[11]  Chudong Tong,et al.  Statistical process monitoring based on orthogonal multi-manifold projections and a novel variable contribution analysis. , 2016, ISA transactions.

[12]  Xingsheng Gu,et al.  Multi-block statistics local kernel principal component analysis algorithm and its application in nonlinear process fault detection , 2020, Neurocomputing.

[13]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[14]  Jianbo Yu,et al.  Process monitoring through manifold regularization-based GMM with global/local information , 2016 .

[15]  Bo Zhou,et al.  Process monitoring of iron-making process in a blast furnace with PCA-based methods , 2016 .

[16]  E. F. Vogel,et al.  A plant-wide industrial process control problem , 1993 .

[17]  Xiaoping Shen,et al.  Kernel Density Estimation for An Anomaly Based Intrusion Detection System , 2006, MLMTA.

[18]  Shijin Wang,et al.  One-dimensional convolutional auto-encoder-based feature learning for fault diagnosis of multivariate processes , 2020 .

[19]  Wen-An Yang.  Monitoring and diagnosing of mean shifts in multivariate manufacturing processes using two-level selective ensemble of learning vector quantization neural networks , 2015, J. Intell. Manuf..

[20]  Youqing Wang,et al.  Key-Performance-Indicator-Related Process Monitoring Based on Improved Kernel Partial Least Squares , 2021, IEEE Transactions on Industrial Electronics.

[21]  William J. Dally,et al.  SCNN: An accelerator for compressed-sparse convolutional neural networks , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[22]  Xiaofeng Yuan,et al.  Deep Learning With Spatiotemporal Attention-Based LSTM for Industrial Soft Sensor Model Development , 2020, IEEE Transactions on Industrial Electronics.

[23]  Jian Hou,et al.  Recent advances on SVM based fault diagnosis and process monitoring in complicated industrial processes , 2016, Neurocomputing.

[24]  Ruomu Tan,et al.  Nonstationary Discrete Convolution Kernel for Multimodal Process Monitoring , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Chenglin Wen,et al.  Dynamic reconstruction based representation learning for multivariable process monitoring , 2019, Journal of Process Control.

[26]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Lin Li,et al.  Nonlinear Dynamic Soft Sensor Modeling With Supervised Long Short-Term Memory Network , 2020, IEEE Transactions on Industrial Informatics.

[28]  Weihua Gui,et al.  Distributed dictionary learning for high-dimensional process monitoring , 2020 .

[29]  Jianbo Yu,et al.  One-Dimensional Residual Convolutional Autoencoder Based Feature Learning for Gearbox Fault Diagnosis , 2020, IEEE Transactions on Industrial Informatics.

[30]  Hao Wu,et al.  Deep convolutional neural network model based chemical process fault diagnosis , 2018, Comput. Chem. Eng..

[31]  Ruochen Liu,et al.  Deep Depthwise Separable Convolutional Network for Change Detection in Optical Aerial Images , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[32]  Xuefeng Yan,et al.  Design teacher and supervised dual stacked auto-encoders for quality-relevant fault detection in industrial process , 2019, Appl. Soft Comput..

[33]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[34]  Wei Xiong,et al.  Stacked Convolutional Denoising Auto-Encoders for Feature Representation , 2017, IEEE Transactions on Cybernetics.

[35]  B. Bakshi.  Multiscale PCA with application to multivariate statistical process monitoring , 1998 .

[36]  Ibrahim Masood,et al.  Bivariate quality control using two-stage intelligent monitoring scheme , 2014, Expert Syst. Appl..

[37]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[38]  Furong Gao,et al.  Data-Driven Two-Dimensional Deep Correlated Representation Learning for Nonlinear Batch Process Monitoring , 2020, IEEE Transactions on Industrial Informatics.

[39]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[40]  Xuefeng Yan,et al.  Active features extracted by deep belief network for process monitoring. , 2019, ISA transactions.

[41]  Li Lin,et al.  Intelligent remote monitoring and diagnosis of manufacturing processes using an integrated approach of neural networks and rough sets , 2003, J. Intell. Manuf..

[42]  Jianbo Yu,et al.  Local and global principal component analysis for process monitoring , 2012 .

[43]  Xuefeng Yan,et al.  Whole Process Monitoring Based on Unstable Neuron Output Information in Hidden Layers of Deep Belief Network , 2019, IEEE Transactions on Cybernetics.

[44]  A. J. Morris,et al.  Non-parametric confidence bounds for process performance monitoring charts☆ , 1996 .

[45]  Hui Cheng,et al.  Local–Global Modeling and Distributed Computing Framework for Nonlinear Plant-Wide Process Monitoring With Industrial Big Data , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.

[47]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[48]  Boualem Boashash,et al.  1-D CNNs for structural damage detection: Verification on a structural health monitoring benchmark data , 2018, Neurocomputing.

[49]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[50]  Xuefeng Yan,et al.  Data-driven individual–joint learning framework for nonlinear process monitoring , 2020 .

[51]  Jianbo Yu,et al.  Stacked denoising autoencoder‐based feature learning for out‐of‐control source recognition in multivariate manufacturing process , 2018, Qual. Reliab. Eng. Int..

[52]  Dong Ni,et al.  A batch-wise LSTM-encoder decoder network for batch process monitoring , 2020 .

[53]  Di He,et al.  Nonlinear fault detection for batch processes via improved chordal kernel tensor locality preserving projections , 2020 .