A Deep Supervised Learning Framework for Data-Driven Soft Sensor Modeling of Industrial Processes

Deep learning has been recently introduced for soft sensors in industrial processes. However, most of the existing deep networks, such as stacked autoencoder, are pretrained in a layerwise unsupervised way to learn feature representations for the raw input data itself. For soft sensors, it is necessary to extract quality-relevant features for quality prediction. Thus, a deep layerwise supervised pretraining framework is proposed for quality-relevant feature extraction and soft sensor modeling in this article, which is based on stacked supervised encoder–decoder (SSED). In SSED, hierarchical quality-relevant features are successively learned by a number of supervised encoder–decoder (SED) models. For each SED, the features from the previous hidden layer are served as new inputs to generate the high-level features that are learned with the constraint of predicting the quality data as good as possible at the output layer of this SED. With this new structure, the SED can learn quality-relevant features that can largely improve the prediction performance. By stacking multiple SEDs, hierarchical quality-relevant features can be progressively learned, and irrelevant information is gradually reduced by deep SSED network. The effectiveness of the proposed model is demonstrated on a numerical example and an industrial process of the debutanizer column.

[1]  Biao Huang,et al.  Dealing with Irregular Data in Soft Sensors: Bayesian Method and Comparative Study , 2008 .

[2]  Weihua Gui,et al.  Probabilistic density-based regression model for soft sensing of nonlinear industrial processes , 2017 .

[3]  Zhiqiang Ge,et al.  Semisupervised JITL Framework for Nonlinear Industrial Soft Sensing Based on Locally Semisupervised Weighted PCR , 2017, IEEE Transactions on Industrial Informatics.

[4]  Roman Rosipal,et al.  Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space , 2002, J. Mach. Learn. Res..

[5]  Steven X. Ding,et al.  Fault Detection for Non-Gaussian Processes Using Generalized Canonical Correlation Analysis and Randomized Algorithms , 2018, IEEE Transactions on Industrial Electronics.

[6]  Junghui Chen,et al.  Flame Images for Oxygen Content Prediction of Combustion Systems Using DBN , 2017 .

[7]  Zhiqiang Ge,et al.  Deep Learning of Semisupervised Process Data With Hierarchical Extreme Learning Machine and Soft Sensor Application , 2018, IEEE Transactions on Industrial Electronics.

[8]  Salvatore Graziani,et al.  Low-order Nonlinear Finite-Impulse Response Soft Sensors for Ionic Electroactive Actuators Based on Deep Learning , 2019, IEEE Transactions on Instrumentation and Measurement.

[9]  Chao Yang,et al.  Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes , 2018 .

[10]  Weiming Shao,et al.  Adaptive soft sensor for quality prediction of chemical processes based on selective ensemble of local partial least squares models , 2015 .

[11]  Di Tang,et al.  A Data-Driven Soft Sensor Modeling Method Based on Deep Learning and its Application , 2017, IEEE Transactions on Industrial Electronics.

[12]  Weihua Gui,et al.  Deep quality-related feature extraction for soft sensing modeling: A deep learning approach with hybrid VW-SAE , 2020, Neurocomputing.

[13]  Chunhua Yang,et al.  Nonlinear VW-SAE Based Deep Learning for Quality-Related Feature Learning and Soft Sensor Modeling , 2018, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society.

[14]  Zhiqiang Ge,et al.  A Probabilistic Just-in-Time Learning Framework for Soft Sensor Development With Missing Data , 2017, IEEE Transactions on Control Systems Technology.

[15]  Xiaofeng Yuan,et al.  Multi‐similarity measurement driven ensemble just‐in‐time learning for soft sensing of industrial processes , 2018 .

[16]  Luigi Fortuna,et al.  Soft sensors for product quality monitoring in debutanizer distillation columns , 2005 .

[17]  Witold Pedrycz,et al.  Hybrid Neural Prediction and Optimized Adjustment for Coke Oven Gas System in Steel Industry , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Jen-Tzung Chien,et al.  Introduction to the Special Section on Deep Learning for Speech and Language Processing , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Biao Huang,et al.  Dynamic Modeling, Predictive Control and Performance Monitoring: A Data-driven Subspace Approach , 2008 .

[20]  Weihua Gui,et al.  Deep learning for quality prediction of nonlinear dynamic processes with variable attention‐based long short‐term memory network , 2020 .

[21]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[22]  Tetsuya Ogata,et al.  Audio-visual speech recognition using deep learning , 2014, Applied Intelligence.

[23]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[24]  Zhiqiang Ge,et al.  Soft Sensor Modeling of Nonlinear Industrial Processes Based on Weighted Probabilistic Projection Regression , 2017, IEEE Transactions on Instrumentation and Measurement.

[25]  Ning Chen,et al.  Temperature Prediction Model for Roller Kiln by ALD-Based Double Locally Weighted Kernel Principal Component Regression , 2018, IEEE Transactions on Instrumentation and Measurement.

[26]  Sarthak Tiwari,et al.  A deep learning based data driven soft sensor for bioprocesses , 2018, Biochemical Engineering Journal.

[27]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[28]  Zhiqiang Ge,et al.  Multimode Process Monitoring Based on Switching Autoregressive Dynamic Latent Variable Model , 2018, IEEE Transactions on Industrial Electronics.

[29]  Zhiqiang Ge,et al.  Probabilistic Sequential Network for Deep Learning of Complex Process Data and Soft Sensor Application , 2019, IEEE Transactions on Industrial Informatics.

[30]  Weihua Gui,et al.  A Layer-Wise Data Augmentation Strategy for Deep Learning Networks and Its Soft Sensor Application in an Industrial Hydrocracking Process , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Lin Li,et al.  Nonlinear Dynamic Soft Sensor Modeling With Supervised Long Short-Term Memory Network , 2020, IEEE Transactions on Industrial Informatics.

[32]  Zhiqiang Ge,et al.  Weighted Linear Dynamic System for Feature Representation and Soft Sensor Application in Nonlinear Dynamic Industrial Processes , 2018, IEEE Transactions on Industrial Electronics.

[33]  Zhi-huan Song,et al.  Locally Weighted Kernel Principal Component Regression Model for Soft Sensing of Nonlinear Time-Variant Processes , 2014 .

[34]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Dexian Huang,et al.  Data-driven soft sensor development based on deep learning technique , 2014 .

[36]  Manabu Kano,et al.  Data-based process monitoring, process control, and quality improvement: Recent developments and applications in steel industry , 2008, Comput. Chem. Eng..

[37]  Weihua Gui,et al.  A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network. , 2019, ISA transactions.

[38]  Biao Huang,et al.  Hierarchical Quality-Relevant Feature Representation for Soft Sensor Modeling: A Novel Deep Learning Strategy , 2020, IEEE Transactions on Industrial Informatics.

[39]  Luigi Fortuna,et al.  Soft Sensors for Monitoring and Control of Industrial Processes (Advances in Industrial Control) , 2006 .

[40]  Junghui Chen,et al.  Deep Learning of Complex Batch Process Data and Its Application on Quality Prediction , 2020, IEEE Transactions on Industrial Informatics.

[41]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[42]  Biao Huang,et al.  Deep Learning-Based Feature Representation and Its Application for Soft Sensor Modeling With Variable-Wise Weighted SAE , 2018, IEEE Transactions on Industrial Informatics.

[43]  Chang Ouk Kim,et al.  A Convolutional Neural Network for Fault Classification and Diagnosis in Semiconductor Manufacturing Processes , 2017, IEEE Transactions on Semiconductor Manufacturing.

[44]  Zhiqiang Ge,et al.  Quality variable prediction for chemical processes based on semisupervised Dirichlet process mixture of Gaussians , 2019, Chemical Engineering Science.

[45]  Weihua Gui,et al.  Temperature prediction for roller kiln based on hybrid first-principle model and data-driven MW-DLWKPCR model. , 2020, ISA transactions.

[46]  Xiaofeng Yuan,et al.  A two‐layer ensemble learning framework for data‐driven soft sensor of the diesel attributes in an industrial hydrocracking process , 2019, Journal of Chemometrics.

[47]  Steven X. Ding,et al.  Improved canonical correlation analysis-based fault detection methods for industrial processes , 2016 .