Dual learning-based online ensemble regression approach for adaptive soft sensor modeling of nonlinear time-varying processes

Abstract Soft sensors have been widely used to estimate difficult-to-measure variables in the process industry. However, the nonlinear nature and time-varying behavior of many processes pose significant challenges for accurate quality prediction. Thus a novel adaptive soft sensor, referred to as dual learning-based online ensemble regression (DLOER), is proposed for nonlinear time-varying processes. To deal with process nonlinearity, just-in-time (JIT) learning is used to build local domains and local models simultaneously while statistical hypothesis testing is employed to remove redundant local models. As a result, multiple diverse local models are constructed for characterizing various process states. Then the posterior probabilities of each test sample with respect to different local models are estimated through Bayesian inference and further set as adaptive weights to combine local predictions into a final output. Moreover, DLOER is equipped with incremental local learning and JIT learning for model adaptation, which enables recursive adaptation and online inclusion of local models, respectively. Therefore, process nonlinearity can be well handled under the local learning framework while both gradual and abrupt changes of processes can be efficiently addressed using the dual learning-based adaptation mechanism. The effectiveness of the DLOER approach is demonstrated through a fed-batch penicillin fermentation process.

[1]  Lei Wu,et al.  Adaptive soft sensor modeling framework based on just-in-time learning and kernel partial least squares regression for nonlinear multiphase batch processes , 2014, Comput. Chem. Eng..

[2]  Li Wang,et al.  Adaptive Soft Sensor Development Based on Online Ensemble Gaussian Process Regression for Nonlinear Time-Varying Batch Processes , 2015 .

[3]  Luigi Fortuna,et al.  Soft Sensors for Monitoring and Control of Industrial Processes (Advances in Industrial Control) , 2006 .

[4]  Rui Araújo,et al.  Design and application of Soft Sensor using Ensemble Methods , 2011, ETFA2011.

[5]  Morimasa Ogawa,et al.  The state of the art in chemical process control in Japan: Good practice and questionnaire survey , 2010 .

[6]  Gülnur Birol,et al.  A modular simulation package for fed-batch fermentation: penicillin production , 2002 .

[7]  Sanjeev S. Tambe,et al.  Soft-sensor development for fed-batch bioreactors using support vector regression , 2006 .

[8]  Dražen Slišković,et al.  Adaptive soft sensor for online prediction and process monitoring based on a mixture of Gaussian process models , 2013, Comput. Chem. Eng..

[9]  Lei Xie,et al.  Novel Just-In-Time Learning-Based Soft Sensor Utilizing Non-Gaussian Information , 2014, IEEE Transactions on Control Systems Technology.

[10]  Luiz Augusto da Cruz Meleiro,et al.  ANN-based soft-sensor for real-time process monitoring and control of an industrial polymerization process , 2009, Comput. Chem. Eng..

[11]  Zhiqiang Ge,et al.  Soft sensor model development in multiphase/multimode processes based on Gaussian mixture regression , 2014 .

[12]  S. Qin,et al.  Multiway Gaussian Mixture Model Based Multiphase Batch Process Monitoring , 2009 .

[13]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[15]  Manabu Kano,et al.  Data-based process monitoring, process control, and quality improvement: Recent developments and applications in steel industry , 2008, Comput. Chem. Eng..

[16]  Daniel E. Rivera,et al.  A 'Model-on-Demand' identification methodology for non-linear process systems , 2001 .

[17]  Biao Huang,et al.  Design of inferential sensors in the process industry: A review of Bayesian methods , 2013 .

[18]  Yan-Lin He,et al.  Data driven soft sensor development for complex chemical processes using extreme learning machine , 2015 .

[19]  Lei Wu,et al.  Hybrid intelligent control of substrate feeding for industrial fed-batch chlortetracycline fermentation process. , 2014, ISA transactions.

[20]  George Cybenko,et al.  Just-in-Time Learning and Estimation , 1996 .

[21]  Zhiqiang Ge,et al.  Probabilistic combination of local independent component regression model for multimode quality prediction in chemical processes , 2014 .

[22]  Rui Araújo,et al.  An on-line weighted ensemble of regressor models to handle concept drifts , 2015, Eng. Appl. Artif. Intell..

[23]  A. Saptoro State of the Art in the Development of Adaptive Soft Sensors based on Just-in-Time Models , 2014 .

[24]  Jie Yu,et al.  Soft Sensor Model Maintenance: A Case Study in Industrial Processes∗ , 2015 .

[25]  Manabu Kano,et al.  Soft‐sensor development using correlation‐based just‐in‐time modeling , 2009 .

[26]  David W. Aha,et al.  Lazy Learning , 1997, Springer Netherlands.

[27]  Weiming Shao,et al.  Adaptive soft sensor for quality prediction of chemical processes based on selective ensemble of local partial least squares models , 2015 .

[28]  Jialin Liu,et al.  On-line soft sensor for polyethylene process with multiple production grades , 2007 .

[29]  Yi Liu,et al.  Industrial melt index prediction with the ensemble anti-outlier just-in-time Gaussian process regression modeling method , 2015 .

[30]  Željko Hocenski,et al.  Methods for Plant Data-Based Process Modeling in Soft-Sensor Development , 2011 .

[31]  Cha Zhang,et al.  Ensemble Machine Learning , 2012 .

[32]  Bogdan Gabrys,et al.  Local learning‐based adaptive soft sensor for catalyst activation prediction , 2011 .

[33]  Sirish L. Shah,et al.  Inferential sensors for estimation of polymer quality parameters: Industrial application of a PLS-based soft sensor for a LDPE plant , 2006 .

[34]  Junghui Chen,et al.  Auto-Switch Gaussian Process Regression-Based Probabilistic Soft Sensors for Industrial Multigrade Processes with Transitions , 2015 .

[35]  Hiromasa Kaneko,et al.  Adaptive soft sensor model using online support vector regression with time variable and discussion of appropriate hyperparameter settings and window size , 2013, Comput. Chem. Eng..

[36]  Tao Chen,et al.  Bagging for Gaussian process regression , 2009, Neurocomputing.

[37]  Zhiqiang Ge,et al.  Mixture semisupervised principal component regression model and soft sensor application , 2014 .

[38]  Giorgio Picci,et al.  Identification, adaptation, learning : the science of learning models from data , 1996 .

[39]  Student,et al.  THE PROBABLE ERROR OF A MEAN , 1908 .

[40]  Fuli Wang,et al.  Improved calibration investigation using phase-wise local and cumulative quality interpretation and prediction , 2009 .

[41]  Bogdan Gabrys,et al.  Data-driven Soft Sensors in the process industry , 2009, Comput. Chem. Eng..

[42]  H. Shi,et al.  Dynamic Multimode Process Modeling and Monitoring Using Adaptive Gaussian Mixture Models , 2012 .

[43]  Stefan Schaal,et al.  Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning , 2002, Applied Intelligence.

[44]  S. Qin Recursive PLS algorithms for adaptive data modeling , 1998 .

[45]  S. Wold,et al.  PLS-regression: a basic tool of chemometrics , 2001 .

[46]  Zhiqiang Ge,et al.  External analysis‐based regression model for robust soft sensing of multimode chemical processes , 2014 .

[47]  Zhiqiang Ge,et al.  Ensemble independent component regression models and soft sensing application , 2014 .

[48]  S. Qin,et al.  Multimode process monitoring with Bayesian inference‐based finite Gaussian mixture models , 2008 .

[49]  Francesco Corona,et al.  Data-derived soft-sensors for biological wastewater treatment plants: An overview , 2013, Environ. Model. Softw..

[50]  Jie Yu,et al.  A Bayesian inference based two-stage support vector regression framework for soft sensor development in batch bioprocesses , 2012, Comput. Chem. Eng..

[51]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[52]  Jingqi Yuan,et al.  Data-driven prediction of the product formation in industrial 2-keto-l-gulonic acid fermentation , 2012, Comput. Chem. Eng..

[53]  Rui Araújo,et al.  A dynamic and on-line ensemble regression for changing environments , 2015, Expert Syst. Appl..

[54]  Soon Keat Tan,et al.  Moving-Window GPR for Nonlinear Dynamic System Modeling with Dual Updating and Dual Preprocessing , 2012 .

[55]  Yi Liu,et al.  Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes , 2013 .

[56]  Koichi Fujiwara,et al.  Virtual sensing technology in process industries: Trends and challenges revealed by recent industria , 2013 .

[57]  Bogdan Gabrys,et al.  Review of adaptation mechanisms for data-driven soft sensors , 2011, Comput. Chem. Eng..

[58]  M. Chiu,et al.  A new data-based methodology for nonlinear process modeling , 2004 .

[59]  Ping Wu,et al.  Online dual updating with recursive PLS model and its application in predicting crystal size of purified terephthalic acid (PTA) process , 2006 .

[60]  Steven D. Brown,et al.  A localized adaptive soft sensor for dynamic system modeling , 2014 .

[61]  Bhupinder S. Dayal,et al.  Recursive exponentially weighted PLS and its applications to adaptive control and prediction , 1997 .

[62]  Gülnur Birol,et al.  Batch Fermentation: Modeling: Monitoring, and Control , 2003 .

[63]  Jie Yu Multiway Gaussian Mixture Model Based Adaptive Kernel Partial Least Squares Regression Method for Soft Sensor Estimation and Reliable Quality Prediction of Nonlinear Multiphase Batch Processes , 2012 .

[64]  Hiromasa Kaneko,et al.  Adaptive soft sensor based on online support vector regression and Bayesian ensemble learning for various states in chemical plants , 2014 .

[65]  Zhiqiang Ge,et al.  Spatio‐temporal adaptive soft sensor for nonlinear time‐varying and variable drifting processes based on moving window LWPLS and time difference model , 2016 .

[66]  Xiangguang Chen,et al.  Online local learning based adaptive soft sensor and its application to an industrial fed-batch chlortetracycline fermentation process , 2015 .

[67]  Hiromasa Kaneko,et al.  Classification of the Degradation of Soft Sensor Models and Discussion on Adaptive Models , 2013 .

[68]  Li Wang,et al.  Multi-model adaptive soft sensor modeling method using local learning and online support vector regression for nonlinear time-variant batch processes , 2015 .

[69]  Manabu Kano,et al.  Long-Term Industrial Applications of Inferential Control Based on Just-In-Time Soft-Sensors: Economical Impact and Challenges , 2013 .

[70]  Yi Liu,et al.  Real-time property prediction for an industrial rubber-mixing process with probabilistic ensemble Gaussian process regression models , 2015 .

[71]  Fuli Wang,et al.  Neural network ensemble modeling for nosiheptide fermentation process based on partial least squares regression , 2011 .