Monitoring multimode processes: a modified PCA algorithm with continual learning ability

For multimode processes, one has to establish local monitoring models corresponding to local modes. However, the significant features of previous modes may be catastrophically forgotten when a monitoring model for the current mode is built. It would result in an abrupt performance decrease. Is it possible to make local monitoring model remember the features of previous modes? Choosing the principal component analysis (PCA) as a basic monitoring model, we try to resolve this problem. A modified PCA algorithm is built with continual learning ability for monitoring multimode processes, which adopts elastic weight consolidation (EWC) to overcome catastrophic forgetting of PCA for successive modes. It is called PCA-EWC, where the significant features of previous modes are preserved when a PCA model is established for the current mode. The computational complexity and key parameters are discussed to further understand the relationship between PCA and the proposed algorithm. Numerical case study and a practical industrial system in China are employed to illustrate the effectiveness of the proposed algorithm.

[1]  Wei Jiang,et al.  A self-starting control chart for high-dimensional short-run processes , 2014 .

[2]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[3]  Antoine Soubeyran,et al.  Global convergence of a proximal linearized algorithm for difference of convex functions , 2015, Optimization Letters.

[4]  Ramin Moghaddass,et al.  A Double Hybrid State-Space Model for Real-Time Sensor-Driven Monitoring of Deteriorating Systems , 2020, IEEE Transactions on Automation Science and Engineering.

[5]  Chunhui Zhao,et al.  Enhanced canonical variate analysis with slow feature for dynamic process status analytics , 2020 .

[6]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[7]  Chunhui Zhao,et al.  Condition-Driven Data Analytics and Monitoring for Wide-Range Nonstationary and Transient Continuous Processes , 2021, IEEE Transactions on Automation Science and Engineering.

[8]  Yi Hu,et al.  A novel local neighborhood standardization strategy and its application in fault detection of multimode processes , 2012 .

[9]  Giovanni Sansavini,et al.  Combined Fault Location and Classification for Power Transmission Lines Fault Diagnosis With Integrated Feature Extraction , 2018, IEEE Transactions on Industrial Electronics.

[10]  Pierantonio Facco,et al.  A hybrid framework for process monitoring: Enhancing data-driven methodologies with state and parameter estimation , 2020 .

[11]  Songhang Chen,et al.  An Improved Mixture of Probabilistic PCA for Nonlinear Data-Driven Process Monitoring , 2019, IEEE Transactions on Cybernetics.

[12]  S. Qin,et al.  Multimode process monitoring with Bayesian inference‐based finite Gaussian mixture models , 2008 .

[13]  James R. Ottewill,et al.  An on-line framework for monitoring nonlinear processes with multiple operating modes , 2020 .

[14]  Yang Tang,et al.  Multimode Process Monitoring and Fault Detection: A Sparse Modeling and Dictionary Learning Method , 2017, IEEE Transactions on Industrial Electronics.

[15]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[16]  Zhonghua Li,et al.  Self-starting control chart for simultaneously monitoring process mean and variance , 2010 .

[17]  P. Alam.  ‘S’ , 2021, Composites Engineering: An A–Z Guide.

[18]  Lei Wang,et al.  Nonlinear Multimode Industrial Process Fault Detection Using Modified Kernel Principal Component Analysis , 2017, IEEE Access.

[19]  Weihua Li,et al.  Recursive PCA for adaptive process monitoring , 1999 .

[20]  Giuseppe Toscani,et al.  WASSERSTEIN METRIC AND LARGE-TIME ASYMPTOTICS OF NONLINEAR DIFFUSION EQUATIONS , 2005 .

[21]  Donghua Zhou,et al.  Distributed self-triggered formation control for multi-agent systems , 2020, Science China Information Sciences.

[22]  Lei Xie,et al.  Multimode process monitoring with PCA mixture model , 2014, Comput. Electr. Eng..

[23]  Donghua Zhou,et al.  Isolating incipient sensor fault based on recursive transformed component statistical analysis , 2018 .

[24]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[25]  Michael N. Jones,et al.  Catastrophic Interference in Neural Embedding Models , 2018, CogSci.

[26]  Ping Zhang,et al.  A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process , 2012 .

[27]  Zhenyu James Kong,et al.  A Spectral Graph Theoretic Approach for Monitoring Multivariate Time Series Data From Complex Dynamical Processes , 2018, IEEE Transactions on Automation Science and Engineering.

[28]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[29]  Nicolas Y. Masse,et al.  Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization , 2018, Proceedings of the National Academy of Sciences.

[30]  Yi Cao,et al.  Canonical Variate Dissimilarity Analysis for Process Incipient Fault Detection , 2018, IEEE Transactions on Industrial Informatics.

[31]  Stefan Wermter,et al.  Continual Lifelong Learning with Neural Networks: A Review , 2019, Neural Networks.

[32]  Chunhua Yang,et al.  Structure Dictionary Learning-Based Multimode Process Monitoring and its Application to Aluminum Electrolysis Process , 2020, IEEE Transactions on Automation Science and Engineering.

[33]  Feiping Nie,et al.  Robust Manifold Nonnegative Matrix Factorization , 2014, ACM Trans. Knowl. Discov. Data.

[34]  Hongbo Shi,et al.  Key principal components with recursive local outlier factor for multimode chemical process monitoring , 2016 .

[35]  Ferenc Huszár,et al.  Note on the quadratic penalties in elastic weight consolidation , 2017, Proceedings of the National Academy of Sciences.

[36]  Feng Qian,et al.  Monitoring for Nonlinear Multiple Modes Process Based on LL-SVDD-MRDA , 2014, IEEE Transactions on Automation Science and Engineering.

[37]  Daniel Zurita,et al.  Multimodal Forecasting Methodology Applied to Industrial Process Monitoring , 2018, IEEE Transactions on Industrial Informatics.

[38]  Satish T. S. Bukkapatnam,et al.  Dirichlet Process Gaussian Mixture Models for Real-Time Monitoring and Their Application to Chemical Mechanical Planarization , 2017, IEEE Transactions on Automation Science and Engineering.

[39]  Giovanna Capizzi,et al.  Self-Starting CUSCORE Control Charts for Individual Multivariate Observations , 2010 .

[40]  Xuefeng Yan,et al.  A Novel Decentralized Process Monitoring Scheme Using a Modified Multiblock PCA Algorithm , 2017, IEEE Transactions on Automation Science and Engineering.

[41]  Yuan Yao,et al.  Statistical analysis and online monitoring for multimode processes with between-mode transitions , 2010 .

[42]  Satish T. S. Bukkapatnam,et al.  Heterogeneous Sensor Data Fusion Approach for Real-time Monitoring in Ultraprecision Machining (UPM) Process Using Non-Parametric Bayesian Clustering and Evidence Theory , 2016, IEEE Transactions on Automation Science and Engineering.

[43]  Chunhui Zhao,et al.  Comprehensive Subspace Decomposition with Analysis of Between-Mode Relative Changes for Multimode Process Monitoring , 2015 .

[44]  Neil D. Lawrence,et al.  Data-Driven Mode Identification and Unsupervised Fault Detection for Nonlinear Multimode Processes , 2020, IEEE Transactions on Industrial Informatics.

[45]  Kaixiang Peng,et al.  Quality-related prediction and monitoring of multi-mode processes using multiple PLS with application to an industrial hot strip mill , 2015, Neurocomputing.

[46]  Douglas M. Hawkins,et al.  Self-Starting Multivariate Exponentially Weighted Moving Average Control Charting , 2007, Technometrics.

[47]  Rama Chellappa,et al.  Learning Without Memorizing , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Le Yao,et al.  Bayesian Nonlinear Gaussian Mixture Regression and its Application to Virtual Sensing for Multimode Industrial Processes , 2020, IEEE Transactions on Automation Science and Engineering.

[49]  Orestes Llanes-Santiago,et al.  Data-driven monitoring of multimode continuous processes: A review , 2019, Chemometrics and Intelligent Laboratory Systems.

[50]  Steven X. Ding,et al.  A Review on Basic Data-Driven Approaches for Industrial Process Monitoring , 2014, IEEE Transactions on Industrial Electronics.

[51]  Stefan Wermter,et al.  Continual Lifelong Learning with Neural Networks: A Review , 2018, Neural Networks.

[52]  Qiang Liu,et al.  Comprehensive monitoring of nonlinear processes based on concurrent kernel projection to latent structures , 2016, IEEE Transactions on Automation Science and Engineering.

[53]  Rahaf Aljundi,et al.  Continual Learning in Neural Networks , 2019, ArXiv.

[54]  Bernard De Baets,et al.  An approach to supervised distance metric learning based on difference of convex functions programming , 2018, Pattern Recognit..

[55]  Jacques Pinaton,et al.  Translation-Invariant Multiscale Energy-Based PCA for Monitoring Batch Processes in Semiconductor Manufacturing , 2017, IEEE Transactions on Automation Science and Engineering.

[56]  Richard E. Turner,et al.  Variational Continual Learning , 2017, ICLR.