Probabilistic Sequential Network for Deep Learning of Complex Process Data and Soft Sensor Application

Soft sensing of quality/key variables is critical to the control and optimization of industrial processes. One of the main drawbacks of data-driven soft sensors is to deal with the dynamic and nonlinear characteristics of process data. This paper proposes a deep learning structure and corresponding training algorithm for the purpose of soft sensor, which is called probabilistic sequential network. The proposed model merges unsupervised feature extraction and supervised dynamic modeling approaches to improve the prediction performance. It is mainly based on the Gaussian-Bernoulli restricted Boltzmann machine and the recurrent neural network structure. To avoid the overfitting problem in the training procedure of deep learning algorithms, the L2 regularization and dropout technique are adopted. The new method can not only deeply extract the nonlinear feature but also widely capture dynamic characteristic of process data. Effectiveness and superiority of the new method are validated through an actual CO2 absorption column, compared to traditional methods.

[1]  Rui Araújo,et al.  Online Mixture of Univariate Linear Regression Models for Adaptive Soft Sensors , 2014, IEEE Transactions on Industrial Informatics.

[2]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[3]  Biao Huang,et al.  Deep Learning-Based Feature Representation and Its Application for Soft Sensor Modeling With Variable-Wise Weighted SAE , 2018, IEEE Transactions on Industrial Informatics.

[4]  Zhiqiang Ge,et al.  Process Data Analytics via Probabilistic Latent Variable Models: A Tutorial Review , 2018, Industrial & Engineering Chemistry Research.

[5]  C. Yoo,et al.  Nonlinear process monitoring using kernel principal component analysis , 2004 .

[6]  Kailash Singh,et al.  Recurrent Neural Network based Soft Sensor for Monitoring and Controlling a Reactive Distillation Column , 2018 .

[7]  Sergey A. Shevchik,et al.  Prediction of Failure in Lubricated Surfaces Using Acoustic Time–Frequency Features and Random Forest Algorithm , 2017, IEEE Transactions on Industrial Informatics.

[8]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[9]  Zhiqiang Ge,et al.  Deep Learning of Semisupervised Process Data With Hierarchical Extreme Learning Machine and Soft Sensor Application , 2018, IEEE Transactions on Industrial Electronics.

[10]  Dae-Ki Kang,et al.  Biased Dropout and Crossmap Dropout: Learning towards effective Dropout regularization in convolutional neural network , 2018, Neural Networks.

[11]  Zhiqiang Ge,et al.  Semisupervised Kernel Learning for FDA Model and its Application for Fault Classification in Industrial Processes , 2016, IEEE Transactions on Industrial Informatics.

[12]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[13]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[14]  Zhiqiang Ge,et al.  Data Mining and Analytics in the Process Industry: The Role of Machine Learning , 2017, IEEE Access.

[15]  Chao Yang,et al.  Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes , 2018 .

[16]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Tianyou Chai,et al.  Soft sensing based on artificial neural network , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[18]  Di Tang,et al.  A Data-Driven Soft Sensor Modeling Method Based on Deep Learning and its Application , 2017, IEEE Transactions on Industrial Electronics.

[19]  Udo Weimar,et al.  On-line novelty detection by recursive dynamic principal component analysis and gas sensor arrays under drift conditions , 2006 .

[20]  Yongheng Jiang,et al.  Soft sensor development and applications based on LSTM in deep neural networks , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[21]  Zhiqiang Ge,et al.  Dynamic Probabilistic Latent Variable Model for Process Data Modeling and Regression Application , 2019, IEEE Transactions on Control Systems Technology.

[22]  Bhaskar D. Kulkarni,et al.  Development of a soft sensor for a batch distillation column using support vector regression techniques , 2007 .

[23]  Zhiqiang Ge,et al.  Distributed predictive modeling framework for prediction and diagnosis of key performance index in plant-wide processes , 2017 .

[24]  Zhiqiang Ge,et al.  Locally Weighted Prediction Methods for Latent Factor Analysis With Supervised and Semisupervised Process Data , 2017, IEEE Transactions on Automation Science and Engineering.

[25]  Ahmed F. Mashaly,et al.  MLP and MLR models for instantaneous thermal efficiency prediction of solar still under hyper-arid environment , 2016, Comput. Electron. Agric..

[26]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[27]  Herbert Jaeger,et al.  Echo state network , 2007, Scholarpedia.

[28]  Zhiqiang Ge,et al.  Big data quality prediction in the process industry: A distributed parallel modeling framework , 2018, Journal of Process Control.

[29]  Zhiqiang Ge,et al.  Scalable Semisupervised GMM for Big Data Quality Prediction in Multimode Processes , 2019, IEEE Transactions on Industrial Electronics.

[30]  Vladimir Pavlovic,et al.  A Dynamic Bayesian Network Approach to Tracking Using Learned Switching Dynamic Models , 2000, HSCC.

[31]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[32]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[33]  Zhiqiang Ge,et al.  Review on data-driven modeling and monitoring for plant-wide industrial processes , 2017 .

[34]  Zhihua Xiong,et al.  Dynamic Soft-Sensing Model by Combining Diagonal Recurrent Neural Network with Levinson Predictor , 2006, ISNN.