Parallel Computing and SGD-Based DPMM For Soft Sensor Development With Large-Scale Semisupervised Data

Soft sensors based on Gaussian mixture models (GMM) have been widely used in industrial process systems for modeling the nonlinearity, non-Gaussianity, and uncertainties. However, there are still some challenging issues in developing high-accuracy GMM-based soft sensors. First, labeled samples are usually scarce due to technical or economical limitations, causing traditional supervised GMM-based soft sensing methods fail to provide satisfactory performance. Second, tremendous amounts of unlabeled samples are gathered, nevertheless, how to fully exploit those unlabeled samples in terms of improving both the predictive accuracy and computational efficiency remains unresolved. In this paper, in order to deal with these issues, two computationally efficient soft sensing methods, namely the parallel computing-based semisupervised Dirichlet process mixture models (P–S<inline-formula><tex-math notation="LaTeX">$^2$</tex-math></inline-formula>DPMM) and stochastic gradient descent-based S<inline-formula><tex-math notation="LaTeX">$^2$</tex-math></inline-formula>DPMM (SGD–S<inline-formula><tex-math notation="LaTeX">$^2$</tex-math></inline-formula>DPMM), are proposed. The S<inline-formula><tex-math notation="LaTeX">$^2$</tex-math></inline-formula>DPMM is first developed to mine information contained in both labeled and unlabeled samples for predictive accuracy enhancement, and subsequently is extended to the P–S<inline-formula><tex-math notation="LaTeX">$^2$</tex-math></inline-formula>DPMM and SGD–S<inline-formula><tex-math notation="LaTeX">$^2$</tex-math></inline-formula>DPMM to handle large-scale process data with sufficient and limited computing resources, respectively. Two case studies are carried out on real-world industrial processes, and the results obtained demonstrate the effectiveness of the proposed methods.

[1]  Zhiqiang Ge,et al.  Distributed Parallel PCA for Modeling and Monitoring of Large-Scale Plant-Wide Processes With Big Data , 2017, IEEE Transactions on Industrial Informatics.

[2]  Zhiqiang Ge,et al.  Process Data Analytics via Probabilistic Latent Variable Models: A Tutorial Review , 2018, Industrial & Engineering Chemistry Research.

[3]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[4]  Sheng Chen,et al.  Monitoring Nonlinear and Non-Gaussian Processes Using Gaussian Mixture Model-Based Weighted Kernel Independent Component Analysis , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Wei Zhang,et al.  JITL based MWGPR soft sensor for multi-mode process with dual-updating strategy , 2016, Comput. Chem. Eng..

[6]  Lei Xie,et al.  Novel Just-In-Time Learning-Based Soft Sensor Utilizing Non-Gaussian Information , 2014, IEEE Transactions on Control Systems Technology.

[7]  Arnaud Doucet,et al.  Semi-supervised learning scheme using Dirichlet process EM-algorithm (パターン認識・メディア理解) , 2009 .

[8]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[9]  Zhiqiang Ge,et al.  Semisupervised Bayesian method for soft sensor modeling with unlabeled data samples , 2011 .

[10]  Zhiqiang Ge,et al.  Mixture semisupervised principal component regression model and soft sensor application , 2014 .

[11]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[12]  Zhiqiang Ge,et al.  A Probabilistic Just-in-Time Learning Framework for Soft Sensor Development With Missing Data , 2017, IEEE Transactions on Control Systems Technology.

[13]  Hamid Reza Karimi,et al.  A Novel Memory Filtering Design for Semi-Markovian Jump Time-Delay Systems , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[14]  Zhiqiang Ge,et al.  Deep Learning of Semisupervised Process Data With Hierarchical Extreme Learning Machine and Soft Sensor Application , 2018, IEEE Transactions on Industrial Electronics.

[15]  Sidan Du,et al.  A multi-manifold semi-supervised Gaussian mixture model for pattern classification , 2013, Pattern Recognit. Lett..

[16]  Zhi-Hua Zhou,et al.  Semi-Supervised Regression with Co-Training , 2005, IJCAI.

[17]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[18]  Zhiqiang Ge,et al.  Data Mining and Analytics in the Process Industry: The Role of Machine Learning , 2017, IEEE Access.

[19]  Zhiqiang Ge,et al.  Variational Bayesian Gaussian Mixture Regression for Soft Sensing Key Variables in Non-Gaussian Industrial Processes , 2017, IEEE Transactions on Control Systems Technology.

[20]  Zhiqiang Ge,et al.  Co-training partial least squares model for semi-supervised soft sensor development , 2015 .

[21]  Fan Yang,et al.  Application of SsVGMM to medical data - classification with novelty detection , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[22]  Nilanjan Senroy,et al.  Event-Triggered Communication Based Distributed Control Scheme for DC Microgrid , 2018, IEEE Transactions on Power Systems.

[23]  Peng Ren,et al.  An adaptive bilateral filter based framework for image denoising , 2014, Neurocomputing.

[24]  Zhiqiang Ge,et al.  Big data quality prediction in the process industry: A distributed parallel modeling framework , 2018, Journal of Process Control.

[25]  Jie Yu,et al.  Online quality prediction of nonlinear and non-Gaussian chemical processes with shifting dynamics using finite mixture model based Gaussian process regression approach , 2012 .

[26]  Jie Yu.  Multiway Gaussian Mixture Model Based Adaptive Kernel Partial Least Squares Regression Method for Soft Sensor Estimation and Reliable Quality Prediction of Nonlinear Multiphase Batch Processes , 2012 .

[27]  Okyay Kaynak,et al.  Big Data for Modern Industry: Challenges and Trends [Point of View] , 2015, Proc. IEEE.

[28]  Chee Khiang Pang,et al.  Gaussian Mixture Model Using Semisupervised Learning for Probabilistic Fault Diagnosis Under New Data Categories , 2017, IEEE Transactions on Instrumentation and Measurement.

[29]  Zhiqiang Ge,et al.  Moving window adaptive soft sensor for state shifting process based on weighted supervised latent factor analysis , 2017 .

[30]  Zhiqiang Ge,et al.  Weighted Linear Dynamic System for Feature Representation and Soft Sensor Application in Nonlinear Dynamic Industrial Processes , 2018, IEEE Transactions on Industrial Electronics.

[31]  Jianbin Qiu,et al.  Sliding mode control for semi-Markovian jump systems via output feedback , 2017, Autom..

[32]  Vipin Kumar,et al.  Trends in big data analytics , 2014, J. Parallel Distributed Comput..

[33]  Eyal Dassau,et al.  Model predictive control with event-triggered communication for an embedded artificial pancreas , 2017, 2017 IEEE Conference on Control Technology and Applications (CCTA).

[34]  Yu Tian,et al.  A Framework and Modeling Method of Data-Driven Soft Sensors Based on Semisupervised Gaussian Regression , 2016 .

[35]  Zhiqiang Ge,et al.  Soft sensor model development in multiphase/multimode processes based on Gaussian mixture regression , 2014 .