Nonlinear industrial soft sensor development based on semi-supervised probabilistic mixture of extreme learning machines

Abstract Soft sensors play an important role in process industries for monitoring and control of key quality variables, and calibration of analyzers. Owing to the merits of fast learning speed and good generalization performance, extreme learning machines (ELMs) have been widely accepted to develop soft sensor models for nonlinear industrial processes. However, there still exist some challenges in developing high-accuracy ELM-based soft sensors. Specifically, ELMs with shallow networks seem to have inadequate representation capabilities for complex nonlinearities, while ELMs with deep networks have difficulties in determining the number of hidden layers and hidden nodes for each layer which readily results in overfitting. In addition, in soft sensor applications, labeled samples are usually limited due to technical or economical reasons, which adds obstacles to model training. To deal with these issues, we propose a semi-supervised probabilistic mixture of ELMs (referred to as the ‘S 2 PMELMs’). In the S 2 PMELMs, localized ELMs are trained and combined, which are completed in a unified probabilistic way such that process nonlinearities and uncertainties can be accommodated. Moreover, based on the variational Bayes expectation–maximization algorithm, we develop a training algorithm for the S 2 PMELMs, where unlabeled samples are able to be exploited and the regularization parameter for each ELM can be adaptively determined. The performance of the S 2 PMELMs is evaluated through two real-world industrial processes, and the results demonstrate the advantages of the proposed method in contrast with several state-of-the-art relevant soft sensing approaches.

[1]  Jianzhong Wang,et al.  Adaptive multiple graph regularized semi-supervised extreme learning machine , 2018, Soft Comput..

[2]  Zhiqiang Ge,et al.  Soft-Sensor Development for Processes With Multiple Operating Modes Based on Semisupervised Gaussian Mixture Regression , 2019, IEEE Transactions on Control Systems Technology.

[3]  Zhiqiang Ge,et al.  Data Mining and Analytics in the Process Industry: The Role of Machine Learning , 2017, IEEE Access.

[4]  Chuanhou Gao,et al.  A comparative analysis of support vector machines and extreme learning machines , 2012, Neural Networks.

[5]  Zhiqiang Ge,et al.  Deep Learning of Semisupervised Process Data With Hierarchical Extreme Learning Machine and Soft Sensor Application , 2018, IEEE Transactions on Industrial Electronics.

[6]  Chunxia Zhang,et al.  Generalized extreme learning machine autoencoder and a new deep neural network , 2017, Neurocomputing.

[7]  Jie Chen,et al.  A new Self-Organizing Extreme Learning Machine soft sensor model and its applications in complicated chemical processes , 2017, Eng. Appl. Artif. Intell..

[8]  Sarthak Tiwari,et al.  A deep learning based data driven soft sensor for bioprocesses , 2018, Biochemical Engineering Journal.

[9]  José David Martín-Guerrero,et al.  Regularized extreme learning machine for regression problems , 2011, Neurocomputing.

[10]  Chao Yang,et al.  Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes , 2018 .

[11]  Danwei Wang,et al.  Sparse Extreme Learning Machine for Classification , 2014, IEEE Transactions on Cybernetics.

[12]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[13]  Zhiqiang Ge,et al.  Semi-supervised mixture of latent factor analysis models with application to online key variable estimation , 2019 .

[14]  Alessandro Chiuso,et al.  The harmonic analysis of kernel functions , 2017, Autom..

[15]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[16]  Luigi Fortuna,et al.  Soft Sensors for Monitoring and Control of Industrial Processes (Advances in Industrial Control) , 2006 .

[17]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[18]  Shahaboddin Shamshirband,et al.  Using SVM-RSM and ELM-RSM Approaches for Optimizing the Production Process of Methyl and Ethyl Esters , 2018, Energies.

[19]  Miao Zhang,et al.  A soft sensor for industrial melt index prediction based on evolutionary extreme learning machine , 2016 .

[20]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Biao Huang,et al.  Deep Learning-Based Feature Representation and Its Application for Soft Sensor Modeling With Variable-Wise Weighted SAE , 2018, IEEE Transactions on Industrial Informatics.

[22]  Dipankar Das,et al.  Enhanced SenticNet with Affective Labels for Concept-Based Opinion Mining , 2013, IEEE Intelligent Systems.

[23]  Shahaboddin Shamshirband,et al.  Sugarcane growth prediction based on meteorological parameters using extreme learning machine and artificial neural network , 2018 .

[24]  Zhiqiang Ge,et al.  Process Data Analytics via Probabilistic Latent Variable Models: A Tutorial Review , 2018, Industrial & Engineering Chemistry Research.

[25]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[26]  Zhiqiang Ge,et al.  Parallel Computing and SGD-Based DPMM For Soft Sensor Development With Large-Scale Semisupervised Data , 2019, IEEE Transactions on Industrial Electronics.

[27]  S. Graziani,et al.  A deep learning based soft sensor for a sour water stripping plant , 2017, 2017 IEEE International Instrumentation and Measurement Technology Conference (I2MTC).

[28]  Rui Araújo,et al.  Mixture of partial least squares experts and application in prediction settings with multiple operating modes , 2014 .

[29]  Minxia Luo,et al.  Outlier-robust extreme learning machine for regression problems , 2015, Neurocomputing.

[30]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[31]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[32]  Yu Tian,et al.  A Framework and Modeling Method of Data-Driven Soft Sensors Based on Semisupervised Gaussian Regression , 2016 .

[33]  Chao Wang,et al.  Variational Bayesian extreme learning machine , 2014, Neural Computing and Applications.

[34]  Biao Huang,et al.  Multiple model based soft sensor development with irregular/missing process output measurement , 2011, 2011 International Symposium on Advanced Control of Industrial Processes (ADCONIP).

[35]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[36]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[37]  Yan-Lin He,et al.  Soft sensor development for the key variables of complex chemical processes using a novel robust bagging nonlinear model integrating improved extreme learning machine with partial least square , 2016 .

[38]  Li Wang,et al.  Dual learning-based online ensemble regression approach for adaptive soft sensor modeling of nonlinear time-varying processes , 2016 .

[39]  Florian Steinke,et al.  Semi-supervised Regression using Hessian energy with an application to semi-supervised dimensionality reduction , 2009, NIPS.

[40]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[41]  Di Tang,et al.  A Data-Driven Soft Sensor Modeling Method Based on Deep Learning and its Application , 2017, IEEE Transactions on Industrial Electronics.

[42]  Yan-Lin He,et al.  Data driven soft sensor development for complex chemical processes using extreme learning machine , 2015 .

[43]  Chu Zhang,et al.  Multi-step ahead wind speed forecasting using a hybrid model based on two-stage decomposition technique and AdaBoost-extreme learning machine , 2017 .

[44]  Antonio J. Serrano,et al.  BELM: Bayesian Extreme Learning Machine , 2011, IEEE Transactions on Neural Networks.

[45]  Manuel Mucientes,et al.  STAC: A web platform for the comparison of algorithms using statistical tests , 2015, 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[46]  Dexian Huang,et al.  Data-driven soft sensor development based on deep learning technique , 2014 .

[47]  Yan-Lin He,et al.  Soft-sensing model development using PLSR-based dynamic extreme learning machine with an enhanced hidden layer , 2016 .