Automatic Deep Extraction of Robust Dynamic Features for Industrial Big Data Modeling and Soft Sensor Application

Dynamic is one of the main bottlenecks in the industrial soft sensor application, due to the difficulties in representing and extracting dynamic data features. Meanwhile, an end-to-end deep network owns the ability to characterize sequence data information, but its fitting ability requires improvements in practical applications. In this article, an ensemble tree model with transferable and robust dynamic features extracted by a newly developed automatic dynamic feature extractor is proposed. First, the dynamic feature extractor with an encoding–decoding structure can provide effective dynamic features, which is equivalent to crossing and nonlinear mapping of sequences under the supervision of a decoder. Meanwhile, a new “regularization” method by smoothing dynamic features based on attention weights is proposed to denoise and alleviate the overfitting of the regressor after adding new features. Then, the extracted dynamic features can be transferred to the regressor with strong generalization ability, which takes into account the feature extraction of the deep network and the generalization of strong models. Finally, application results on a debutanizer distillation process show that the incorporation of robust dynamic features can significantly improve the soft sensing performance, compared to traditional methods. Moreover, the proposed model is further implemented through a cloud computing platform for industrial big data analytics.

[1]  Hiram Calvo,et al.  A novel recurrent neural network soft sensor via a differential evolution training algorithm for the tire contact patch , 2017, Neurocomputing.

[2]  Chonghun Han,et al.  A nonlinear soft sensor based on multivariate smoothing procedure for quality estimation in distillation columns , 2000 .

[3]  William Zhu,et al.  Feature selection for multi-label classification using neighborhood preservation , 2018, IEEE/CAA Journal of Automatica Sinica.

[4]  Donghua Zhou,et al.  Quality Relevant Data-Driven Modeling and Monitoring of Multivariate Dynamic Processes: The Dynamic T-PLS Approach , 2011, IEEE Transactions on Neural Networks.

[5]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[6]  Soon Keat Tan,et al.  Localized, Adaptive Recursive Partial Least Squares Regression for Dynamic System Modeling , 2012 .

[7]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[8]  Zhiqiang Ge,et al.  Local Parameter Optimization of LSSVM for Industrial Soft Sensing With Big Data and Cloud Implementation , 2020, IEEE Transactions on Industrial Informatics.

[9]  L. T. Fan,et al.  Monitoring the process of curing of epoxy/graphite fiber composites with a recurrent neural network as a soft sensor , 1998 .

[10]  Zhiqiang Ge,et al.  Distributed Parallel PCA for Modeling and Monitoring of Large-Scale Plant-Wide Processes With Big Data , 2017, IEEE Transactions on Industrial Informatics.

[11]  Di Tang,et al.  A Data-Driven Soft Sensor Modeling Method Based on Deep Learning and its Application , 2017, IEEE Transactions on Industrial Electronics.

[12]  Davide Fissore,et al.  Design and validation of an innovative soft-sensor for pharmaceuticals freeze-drying monitoring , 2011 .

[13]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[14]  Zhiqiang Ge,et al.  Big data quality prediction in the process industry: A distributed parallel modeling framework , 2018, Journal of Process Control.

[15]  Zhiqiang Ge,et al.  Process Data Analytics via Probabilistic Latent Variable Models: A Tutorial Review , 2018, Industrial & Engineering Chemistry Research.

[16]  Luigi Fortuna,et al.  Soft Sensors for Monitoring and Control of Industrial Processes (Advances in Industrial Control) , 2006 .

[17]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[18]  Zhiqiang Ge,et al.  Data Mining and Analytics in the Process Industry: The Role of Machine Learning , 2017, IEEE Access.

[19]  Li Wang,et al.  Functional-type single-input-rule-modules connected neural fuzzy system for wind speed prediction , 2017, IEEE/CAA Journal of Automatica Sinica.

[20]  Biao Huang,et al.  Deep Learning-Based Feature Representation and Its Application for Soft Sensor Modeling With Variable-Wise Weighted SAE , 2018, IEEE Transactions on Industrial Informatics.

[21]  Zhiqiang Ge,et al.  Dynamic Probabilistic Latent Variable Model for Process Data Modeling and Regression Application , 2019, IEEE Transactions on Control Systems Technology.

[22]  MengChu Zhou,et al.  An online fault detection model and strategies based on SVM-grid in clouds , 2018, IEEE/CAA Journal of Automatica Sinica.

[23]  Dexian Huang,et al.  Probabilistic slow feature analysis‐based representation learning from massive process data for soft sensor modeling , 2015 .

[24]  Biao Huang,et al.  Design of inferential sensors in the process industry: A review of Bayesian methods , 2013 .

[25]  Michael I. Jordan,et al.  Bayesian Nonparametric Inference of Switching Dynamic Linear Models , 2010, IEEE Transactions on Signal Processing.

[26]  Zhiqiang Ge,et al.  Scalable Semisupervised GMM for Big Data Quality Prediction in Multimode Processes , 2019, IEEE Transactions on Industrial Electronics.

[27]  MengChu Zhou,et al.  Bilevel Feature Extraction-Based Text Mining for Fault Diagnosis of Railway Systems , 2017, IEEE Transactions on Intelligent Transportation Systems.

[28]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.