Deep Principal Component Analysis Based on Layerwise Feature Extraction and Its Application to Nonlinear Process Monitoring

In order to deeply exploit intrinsic data feature information hidden among the process data, an improved kernel principal component analysis (KPCA) method is proposed, which is referred to as deep principal component analysis (DePCA). Specifically, motivated by the deep learning strategy, we design a hierarchical statistical model structure to extract multilayer data features, including both the linear and nonlinear principal components. To reduce the computation complexity in nonlinear feature extraction, the feature-samples’ selection technique is applied to build the sparse kernel model for DePCA. To integrate the monitoring statistics at each feature layer, Bayesian inference is used to transform the monitoring statistics into fault probabilities, and then, two probability-based DePCA monitoring statistics are constructed by weighting the fault probabilities at all the feature layers. Two case studies involving a simulated nonlinear system and the benchmark Tennessee Eastman process demonstrate the superior fault detection performance of the proposed DePCA method over the traditional KPCA-based methods.

[1]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[2]  Christos Georgakis,et al.  Disturbance detection and isolation by dynamic principal component analysis , 1995 .

[3]  Sheng Chen,et al.  Nonlinear Identification Using Orthogonal Forward Regression With Nested Optimal Regularization , 2015, IEEE Transactions on Cybernetics.

[4]  Huijun Gao,et al.  Data-Based Techniques Focused on Modern Industry: An Overview , 2015, IEEE Transactions on Industrial Electronics.

[5]  Ludmila I. Kuncheva,et al.  PCA Feature Extraction for Change Detection in Multidimensional Unlabeled Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Si-Zhao Joe Qin,et al.  Survey on data-driven industrial process monitoring and diagnosis , 2012, Annu. Rev. Control..

[7]  Zhiming Li,et al.  Modeling and Monitoring of Dynamic Processes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Biao Huang,et al.  GMM and optimal principal components-based Bayesian method for multimode fault diagnosis , 2016, Comput. Chem. Eng..

[9]  Ping Zhang,et al.  A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process , 2012 .

[10]  Zhiqiang Ge,et al.  Data Mining and Analytics in the Process Industry: The Role of Machine Learning , 2017, IEEE Access.

[11]  Yi Hu,et al.  A novel local neighborhood standardization strategy and its application in fault detection of multimode processes , 2012 .

[12]  Hairong Qi,et al.  Hybrid Dimensionality Reduction Method Based on Support Vector Machine and Independent Component Analysis , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Sheng Chen,et al.  Monitoring Nonlinear and Non-Gaussian Processes Using Gaussian Mixture Model-Based Weighted Kernel Independent Component Analysis , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Ying Xu,et al.  Fault detection of multimode non-Gaussian dynamic process using dynamic Bayesian independent component analysis , 2016, Neurocomputing.

[15]  Guangming Shi,et al.  Bottom–Up Visual Saliency Estimation With Deep Autoencoder-Based Sparse Reconstruction , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Zhiqiang Ge,et al.  Review on data-driven modeling and monitoring for plant-wide industrial processes , 2017 .

[17]  Mohd Azlan Hussain,et al.  Fault diagnosis of Tennessee Eastman process with multi- scale PCA and ANFIS , 2013 .

[18]  Zhiqiang Ge,et al.  Nonlinear process monitoring based on linear subspace and Bayesian inference , 2010 .

[19]  Zhiqiang Ge,et al.  Performance-driven ensemble learning ICA model for improved non-Gaussian process monitoring , 2013 .

[20]  Sheng Chen,et al.  Probability Density Estimation With Tunable Kernels Using Orthogonal Forward Regression , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Kaixiang Peng,et al.  A Quality-Based Nonlinear Fault Diagnosis Framework Focusing on Industrial Multimode Batch Processes , 2016, IEEE Transactions on Industrial Electronics.

[22]  Sheng Chen,et al.  Nonlinear Process Fault Diagnosis Based on Serial Principal Component Analysis , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Xuefeng Yan,et al.  Nonlinear plant-wide process monitoring using MI-spectral clustering and Bayesian inference-based multiblock KPCA , 2015 .

[24]  Sheng Chen,et al.  Sparse modeling using orthogonal forward regression with PRESS statistic and regularization , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Bing Lam Luk,et al.  Construction of Tunable Radial Basis Function Networks Using Orthogonal Forward Selection , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Soowoong Kim,et al.  Deep belief network based statistical feature learning for fingerprint liveness detection , 2016, Pattern Recognit. Lett..

[27]  G. Baudat,et al.  Feature vector selection and projection using kernels , 2003, Neurocomputing.

[28]  Chun-Chin Hsu,et al.  Adaptive Kernel Principal Component Analysis (KPCA) for Monitoring Small Disturbances of Nonlinear Processes , 2010 .

[29]  Richard D. Braatz,et al.  A combined canonical variate analysis and Fisher discriminant analysis (CVA-FDA) approach for fault diagnosis , 2015, Comput. Chem. Eng..

[30]  Jian Wang,et al.  Fault Diagnosis Method of Joint Fisher Discriminant Analysis Based on the Local and Global Manifold Learning and Its Kernel Version , 2016, IEEE Transactions on Automation Science and Engineering.

[31]  Marco E. Sanjuan,et al.  An improved weighted recursive PCA algorithm for adaptive fault detection , 2016 .

[32]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[34]  Xiaogang Deng,et al.  Nonlinear process fault pattern recognition using statistics kernel PCA similarity factor , 2013, Neurocomputing.

[35]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[36]  Chi Ma,et al.  Fault diagnosis of nonlinear processes using multiscale KPCA and multiscale KPLS , 2011 .

[37]  Qiang Liu,et al.  Multiblock Concurrent PLS for Decentralized Monitoring of Continuous Annealing Processes , 2014, IEEE Transactions on Industrial Electronics.

[38]  Donghua Zhou,et al.  A New Method of Dynamic Latent-Variable Modeling for Process Monitoring , 2014, IEEE Transactions on Industrial Electronics.

[39]  Xuemin Tian,et al.  Sparse Kernel Locality Preserving Projection and Its Application in Nonlinear Process Fault Detection , 2013 .

[40]  Jin Hyun Park,et al.  Fault detection and identification of nonlinear processes based on kernel PCA , 2005 .

[41]  Sheng Chen,et al.  Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Xiaogang Deng,et al.  Modified kernel principal component analysis based on local structure analysis and its application to nonlinear process fault diagnosis , 2013 .

[43]  Chunhui Zhao,et al.  Statistical modeling and online fault detection for multiphase batch processes with analysis of between-phase relative changes , 2014 .

[44]  Osonde Osoba,et al.  Noise-enhanced convolutional neural networks , 2016, Neural Networks.

[45]  Xiaoling Zhang,et al.  Multiway kernel independent component analysis based on feature samples for batch process monitoring , 2009, Neurocomputing.

[46]  Zhiqiang Ge,et al.  Improved kernel PCA-based monitoring approach for nonlinear processes , 2009 .

[47]  Cristobal Ruiz-Carcel,et al.  Canonical Variate Analysis for Performance Degradation under Faulty Conditions , 2016 .

[48]  C. Yoo,et al.  Nonlinear process monitoring using kernel principal component analysis , 2004 .

[49]  Theodora Kourti,et al.  Process analysis, monitoring and diagnosis, using multivariate projection methods , 1995 .

[50]  E. F. Vogel,et al.  A plant-wide industrial process control problem , 1993 .

[51]  Xuemin Tian,et al.  Multimode Process Fault Detection Using Local Neighborhood Similarity Analysis , 2014 .

[52]  Haibo He,et al.  Optimized Relative Transformation Matrix Using Bacterial Foraging Algorithm for Process Fault Detection , 2016, IEEE Transactions on Industrial Electronics.

[53]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[54]  Sheng Chen,et al.  Noise-resistant joint diagonalization independent component analysis based process fault detection , 2015, Neurocomputing.

[55]  Zhiqiang Ge,et al.  HMM-Driven Robust Probabilistic Principal Component Analyzer for Dynamic Process Fault Classification , 2015, IEEE Transactions on Industrial Electronics.

[56]  Qiang Liu,et al.  Decentralized Fault Diagnosis of Continuous Annealing Processes Based on Multilevel PCA , 2013, IEEE Transactions on Automation Science and Engineering.