Variational Bayesian Approach to Nonstationary and Oscillatory Slow Feature Analysis With Applications in Soft Sensing and Process Monitoring

Extraction of underlying patterns from measured variables is central to various data-driven control applications, such as soft-sensor modeling, statistical process monitoring, and fault detection and diagnosis. More often than not, the observed variables display nonstationary characteristics and oscillations due to the changes in operating conditions and problems in controller tuning. Such variations pose a great challenge to conventional feature extraction methods. Hence, we present a probabilistic drift-type nonstationary oscillating slow feature model that can separate oscillating patterns and nonstationary variations from measured data. Furthermore, the measurement noise of each variable is independently modeled to account for the fact that not all the observed variables have the same level of uncertainty. Finally, the feature extractor parameters are estimated under a variational Bayesian framework to incorporate the prior information and obtain corresponding posterior distributions. The proposed methodology is applied to solve a fouling monitoring problem for an industrial oil production process.

[1]  Steven X. Ding,et al.  Data-Driven Fault Diagnosis for Traction Systems in High-Speed Trains: A Survey, Challenges, and Perspectives , 2022, IEEE Transactions on Intelligent Transportation Systems.

[2]  Dexian Huang,et al.  A Holistic Probabilistic Framework for Monitoring Nonstationary Dynamic Industrial Processes , 2021, IEEE Transactions on Control Systems Technology.

[3]  Biao Huang,et al.  Complex probabilistic slow feature extraction with applications in process data analytics , 2021, Comput. Chem. Eng..

[4]  Biao Huang,et al.  Multimodal process monitoring based on variational Bayesian PCA and Kullback-Leibler divergence between mixture models , 2021 .

[5]  Biao Huang,et al.  Transfer Learning for Dynamic Feature Extraction Using Variational Bayesian Inference , 2021, IEEE Transactions on Knowledge and Data Engineering.

[6]  Witold Pedrycz,et al.  A Novel Semi-Supervised Sparse Bayesian Regression Based on Variational Inference for Industrial Datasets With Incomplete Outputs , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[7]  Zhiqiang Ge,et al.  Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor Application , 2020, IEEE Transactions on Cybernetics.

[8]  Xuefeng Yan,et al.  Deep Discriminative Representation Learning for Nonlinear Process Fault Detection , 2020, IEEE Transactions on Automation Science and Engineering.

[9]  Youqing Wang,et al.  Multistep Dynamic Slow Feature Analysis for Industrial Process Monitoring , 2020, IEEE Transactions on Instrumentation and Measurement.

[10]  Yining Dong,et al.  Efficient Dynamic Latent Variable Analysis for High-Dimensional Time Series Data , 2020, IEEE Transactions on Industrial Informatics.

[11]  Yanjun Ma,et al.  Extracting nonstationary features for process data analytics and application in fouling detection , 2020, Comput. Chem. Eng..

[12]  Weiming Shao,et al.  Semisupervised Bayesian Gaussian Mixture Models for Non-Gaussian Soft Sensor , 2019, IEEE Transactions on Cybernetics.

[13]  Arun K. Tangirala,et al.  A Novel Causality Method for Reconstruction of Process Topology in Multivariable LTI Dynamical Systems , 2019, 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE).

[14]  Arun K. Tangirala,et al.  Inferring Direct Causality from Noisy Data using Convergent Cross Mapping , 2019, 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE).

[15]  Xin Peng,et al.  Real-Time Semisupervised Predictive Modeling Strategy for Industrial Continuous Catalytic Reforming Process with Incomplete Data Using Slow Feature Analysis , 2019, Industrial & Engineering Chemistry Research.

[16]  Biao Huang,et al.  Output relevant slow feature extraction using partial least squares , 2019, Chemometrics and Intelligent Laboratory Systems.

[17]  Biao Huang,et al.  Variational Bayesian Approach for Causality and Contemporaneous Correlation Features Inference in Industrial Process Data , 2019, IEEE Transactions on Cybernetics.

[18]  Fang Deng,et al.  A Fast Distributed Variational Bayesian Filtering for Multisensor LTV System With Non-Gaussian Noise , 2019, IEEE Transactions on Cybernetics.

[19]  Chunhui Zhao,et al.  A new soft-sensor algorithm with concurrent consideration of slowness and quality interpretation for dynamic chemical process , 2019, Chemical Engineering Science.

[20]  Chunhui Zhao,et al.  Slow-Feature-Analysis-Based Batch Process Monitoring With Comprehensive Interpretation of Operation Condition Deviation and Dynamic Anomaly , 2019, IEEE Transactions on Industrial Electronics.

[21]  Arun K. Tangirala,et al.  Efficient Reconstruction of Granger-Causal Networks in Linear Multivariable Dynamical Processes , 2019, Industrial & Engineering Chemistry Research.

[22]  Biao Huang,et al.  Semi‐supervised dynamic latent variable modeling: I/O probabilistic slow feature analysis approach , 2018, AIChE Journal.

[23]  Fábio Porto,et al.  Nonstationary time series transformation methods: An experimental review , 2019, Knowl. Based Syst..

[24]  Yang Shi,et al.  Dynamic Coverage Control in a Time-Varying Environment Using Bayesian Prediction , 2019, IEEE Transactions on Cybernetics.

[25]  Biao Huang,et al.  Process monitoring using a generalized probabilistic linear latent variable model , 2018, Autom..

[26]  Yanjun Ma,et al.  A Variational Bayesian Approach to Modelling with Random Time-varying Time Delays , 2018, 2018 Annual American Control Conference (ACC).

[27]  Yanjun Ma,et al.  Extracting dynamic features with switching models for process data analytics and application in soft sensing , 2018 .

[28]  Chunhui Zhao,et al.  A full‐condition monitoring method for nonstationary dynamic chemical processes with cointegration and slow feature analysis , 2018 .

[29]  Chunhui Zhao,et al.  Incipient Fault Detection for Complex Industrial Processes with Stationary and Nonstationary Hybrid Characteristics , 2018 .

[30]  Biao Huang,et al.  Identification of robust probabilistic slow feature regression model for process data contaminated with outliers , 2018 .

[31]  Alireza Fatehi,et al.  Robust Estimation of ARX Models With Time Varying Time Delays Using Variational Bayesian Approach , 2018, IEEE Transactions on Cybernetics.

[32]  Si-Zhao Joe Qin,et al.  Dynamic latent variable analytics for process operations and control , 2017, Comput. Chem. Eng..

[33]  S. Joe Qin,et al.  A novel dynamic PCA algorithm for dynamic data modeling and process monitoring , 2017 .

[34]  Iqbal M. Mujtaba,et al.  Dynamic modelling of Heat Exchanger fouling in multistage flash (MSF) desalination , 2017 .

[35]  Yanjun Ma,et al.  Bayesian Learning for Dynamic Feature Extraction With Application in Soft Sensing , 2017, IEEE Transactions on Industrial Electronics.

[36]  Biao Huang,et al.  Wavelet Transform Based Methodology for Detection and Characterization of Multiple Oscillations in Nonstationary Variables , 2017 .

[37]  Yaojie Lu,et al.  A Variational Bayesian Approach to Robust Identification of Switched ARX Models , 2016, IEEE Transactions on Cybernetics.

[38]  Dexian Huang,et al.  Monitoring of operating point and process dynamics via probabilistic slow feature analysis , 2016 .

[39]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[40]  Dexian Huang,et al.  Probabilistic slow feature analysis‐based representation learning from massive process data for soft sensor modeling , 2015 .

[41]  Dexian Huang,et al.  Detecting and isolating plant-wide oscillations via slow feature analysis , 2015, 2015 American Control Conference (ACC).

[42]  Zhiqiang Ge,et al.  Nonlinear semisupervised principal component regression for soft sensor modeling and its mixture form , 2014 .

[43]  Dirk Ostwald,et al.  A tutorial on variational Bayes for latent linear stochastic time-series models , 2014 .

[44]  Zhiqiang Ge,et al.  Mixture semisupervised principal component regression model and soft sensor application , 2014 .

[45]  Junbin Gao,et al.  Supervised Latent Linear Gaussian Process Latent Variable Model for Dimensionality Reduction , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[46]  George Sugihara,et al.  Detecting Causality in Complex Ecosystems , 2012, Science.

[47]  Stephen J. Roberts,et al.  A tutorial on variational Bayesian inference , 2012, Artificial Intelligence Review.

[48]  Michael I. Jordan,et al.  Variational Bayesian Inference with Stochastic Search , 2012, ICML.

[49]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Xuelong Li,et al.  Supervised Gaussian Process Latent Variable Model for Dimensionality Reduction , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[51]  S. Selvanathan,et al.  Diagnosis of Oscillations Due to Multiple Sources in Model-Based Control Loops Using Wavelet Transforms , 2009 .

[52]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[53]  Richard E. Turner,et al.  A Maximum-Likelihood Interpretation for Slow Feature Analysis , 2007, Neural Computation.

[54]  Sirish L. Shah,et al.  Non-Negative Matrix Factorization for Detection and Diagnosis of Plantwide Oscillations , 2007 .

[55]  David Barber,et al.  Unified Inference for Variational Bayesian Linear Gaussian State-Space Models , 2006, NIPS.

[56]  N. Nasios,et al.  Variational learning for Gaussian mixture models , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[57]  Nina F. Thornhill,et al.  PSCMAP: A new tool for plant-wide oscillation detection , 2005 .

[58]  D. Seborg,et al.  Estimating product composition profiles in batch distillation via partial least squares regression , 2004 .

[59]  Stephen M. Smith,et al.  Probabilistic independent component analysis for functional magnetic resonance imaging , 2004, IEEE Transactions on Medical Imaging.

[60]  William D. Penny,et al.  Variational Bayes for generalized autoregressive models , 2002, IEEE Trans. Signal Process..

[61]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[62]  James V. Stone.  Independent component analysis: an introduction , 2002, Trends in Cognitive Sciences.

[63]  Mats G. Gustafsson,et al.  A Probabilistic Derivation of the Partial Least-Squares Algorithm , 2001, J. Chem. Inf. Comput. Sci..

[64]  Lei Fan.  Robust Latent Variable Modeling Using Probabilistic Slow Feature Analysis , 2020 .

[65]  Le Yao,et al.  Streaming parallel variational Bayesian supervised factor analysis for adaptive soft sensor modeling with big process data , 2020 .

[66]  M. Kreider,et al.  Thermal performance degradation and heat-transfer fouling , 2017 .

[67]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[68]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[69]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[70]  Bhupinder S. Dayal,et al.  Recursive exponentially weighted PLS and its applications to adaptive control and prediction , 1997 .

[71]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .