Physically Consistent Soft-Sensor Development Using Sequence-to-Sequence Neural Networks

Soft sensors attempt to predict the key quality variables that are infrequently available using the sensor and manipulated variables that are readily available. Since only limited amount of labeled data are available, there is always the concern whether the underlying physics were captured so that the model can be reasonably extrapolated. A sequence-to-sequence model in the form of a nonlinear state-observer/encoder and predictor/decoder was proposed. The observer can be trained using a large amount of unlabeled data, but in a supervised manner in which the process dynamics is tracked. The encoder output and manipulated variables are used to train the quality predictor. The model is applied to the product impurity predictions of an industrial column. Results show that good predictions and excellent consistency in the sign of estimated gains can be achieved even with limited amount of data. These findings indicated that the proposed sequence-to-sequence data-driven approach is able to capture the underlying physics of the process.

[1]  Jun Wang,et al.  Model Predictive Control of Unknown Nonlinear Dynamical Systems Based on Recurrent Neural Networks , 2012, IEEE Transactions on Industrial Electronics.

[2]  Jie Zhang,et al.  Recurrent neuro-fuzzy networks for nonlinear process modeling , 1999, IEEE Trans. Neural Networks.

[3]  Bogdan Gabrys,et al.  Data-driven Soft Sensors in the process industry , 2009, Comput. Chem. Eng..

[4]  Manabu Kano,et al.  Data-based process monitoring, process control, and quality improvement: Recent developments and applications in steel industry , 2008, Comput. Chem. Eng..

[5]  Francesco Corona,et al.  Data-derived soft-sensors for biological wastewater treatment plants: An overview , 2013, Environ. Model. Softw..

[6]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[7]  Dexian Huang,et al.  Data-driven soft sensor development based on deep learning technique , 2014 .

[8]  Kaamran Raahemifar,et al.  Artificial neural network (ANN) based model predictive control (MPC) and optimization of HVAC systems: A state of the art review and case study of a residential HVAC system , 2017 .

[9]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[10]  Quoc V. Le,et al.  Recurrent Neural Networks for Noise Reduction in Robust ASR , 2012, INTERSPEECH.

[11]  Sten Bay Jørgensen,et al.  A systematic approach for soft sensor development , 2007, Comput. Chem. Eng..

[12]  Zhiqiang Ge,et al.  Semisupervised Bayesian method for soft sensor modeling with unlabeled data samples , 2011 .

[13]  Luiz Augusto da Cruz Meleiro,et al.  ANN-based soft-sensor for real-time process monitoring and control of an industrial polymerization process , 2009, Comput. Chem. Eng..

[14]  Moustafa Elshafei,et al.  Soft sensor for NOx and O2 using dynamic neural networks , 2009, Comput. Electr. Eng..

[15]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[16]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[17]  David Shan-Hill Wong,et al.  Comparison of Embedded System Design for Industrial Applications , 2011, IEEE Transactions on Industrial Informatics.

[18]  Zhiqiang Ge,et al.  Deep Learning of Semisupervised Process Data With Hierarchical Extreme Learning Machine and Soft Sensor Application , 2018, IEEE Transactions on Industrial Electronics.

[19]  Bogdan Gabrys,et al.  Review of adaptation mechanisms for data-driven soft sensors , 2011, Comput. Chem. Eng..

[20]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[21]  Zhiqiang Ge,et al.  Weighted Linear Dynamic System for Feature Representation and Soft Sensor Application in Nonlinear Dynamic Industrial Processes , 2018, IEEE Transactions on Industrial Electronics.

[22]  Biao Huang,et al.  Deep Learning-Based Feature Representation and Its Application for Soft Sensor Modeling With Variable-Wise Weighted SAE , 2018, IEEE Transactions on Industrial Informatics.

[23]  Rui Araújo,et al.  Review of soft sensor methods for regression applications , 2016 .

[24]  Yuan Kang,et al.  Virtual Metrology Technique for Semiconductor Manufacturing , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[25]  Jialin Liu,et al.  Development of Self-Validating Soft Sensors Using Fast Moving Window Partial Least Squares , 2010 .

[26]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[27]  Zhiqiang Ge,et al.  Parallel Computing and SGD-Based DPMM For Soft Sensor Development With Large-Scale Semisupervised Data , 2019, IEEE Transactions on Industrial Electronics.

[28]  Liwei Wang,et al.  The Expressive Power of Neural Networks: A View from the Width , 2017, NIPS.

[29]  Chao Yang,et al.  Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes , 2018 .

[30]  B. Bequette,et al.  Product property and production rate control of styrene polymerization , 2002 .

[31]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[32]  Peter Young,et al.  Smart Reply: Automated Response Suggestion for Email , 2016, KDD.

[33]  Vijander Singh,et al.  Development of soft sensor for neural network based control of distillation column. , 2013, ISA transactions.

[34]  Ching-Chih Tsai,et al.  Generalized predictive control using recurrent fuzzy neural networks for industrial processes , 2007 .

[35]  Lin Li,et al.  Nonlinear Dynamic Soft Sensor Modeling With Supervised Long Short-Term Memory Network , 2020, IEEE Transactions on Industrial Informatics.

[36]  S. Joe Qin,et al.  Process data analytics in the era of big data , 2014 .

[37]  Zhiqiang Ge,et al.  Probabilistic Sequential Network for Deep Learning of Complex Process Data and Soft Sensor Application , 2019, IEEE Transactions on Industrial Informatics.

[38]  Morimasa Ogawa,et al.  The state of the art in chemical process control in Japan: Good practice and questionnaire survey , 2010 .

[39]  Karol Antczak,et al.  Deep Recurrent Neural Networks for ECG Signal Denoising , 2018, ArXiv.