Data-driven Soft Sensors in the process industry

In the last two decades Soft Sensors established themselves as a valuable alternative to the traditional means for the acquisition of critical process variables, process monitoring and other tasks which are related to process control. This paper discusses characteristics of the process industry data which are critical for the development of data-driven Soft Sensors. These characteristics are common to a large number of process industry fields, like the chemical industry, bioprocess industry, steel industry, etc. The focus of this work is put on the data-driven Soft Sensors because of their growing popularity, already demonstrated usefulness and huge, though yet not completely realised, potential. A comprehensive selection of case studies covering the three most important Soft Sensor application fields, a general introduction to the most popular Soft Sensor modelling techniques as well as a discussion of some open issues in the Soft Sensor development and maintenance and their possible solutions are the main contributions of this work.

[1]  Sven Serneels,et al.  Principal component analysis for data containing outliers and missing elements , 2008, Comput. Stat. Data Anal..

[2]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[3]  R. Bro.  Multiway calibration. Multilinear PLS , 1996 .

[4]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Shengwei Wang,et al.  Sensor-fault detection, diagnosis and estimation for centrifugal chiller systems using principal-component analysis method , 2005 .

[6]  S. Qin,et al.  Self-validating inferential sensors with application to air emission monitoring , 1997 .

[7]  John F. MacGregor,et al.  Multivariate SPC charts for monitoring batch processes , 1995 .

[8]  J. Friedman,et al.  [A Statistical View of Some Chemometrics Regression Tools]: Response , 1993 .

[9]  S. Joe Qin,et al.  Subspace approach to multidimensional fault identification and reconstruction , 1998 .

[10]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[11]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[12]  K. Esbensen,et al.  Regression on multivariate images: Principal component regression for modeling, prediction and visual diagnostic tools , 1991 .

[13]  Ronald K. Pearson,et al.  Outliers in process modeling and identification , 2002, IEEE Trans. Control. Syst. Technol..

[14]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[15]  Bogdan Gabrys,et al.  Adaptive Local Learning Soft Sensor for Inferential Control Support , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[16]  Arthur K. Kordon,et al.  Robust Inferential Sensors Based on Ensemble of Predictors Generated by Genetic Programming , 2004, PPSN.

[17]  S. Joe Qin,et al.  Joint diagnosis of process and sensor faults using principal component analysis , 1998 .

[18]  Theodora Kourti,et al.  Analysis, monitoring and fault diagnosis of batch processes using multiblock and multiway PLS , 1995 .

[19]  C. E. Schlags,et al.  Multivariate statistical analysis of an emulsion batch process , 1998 .

[20]  FengDING,et al.  Modeling and Identification of Multirate Systems , 2005 .

[21]  Junghui Chen,et al.  On-line batch process monitoring using dynamic PCA and dynamic PLS models , 2002 .

[22]  B. Bequette,et al.  Product property and production rate control of styrene polymerization , 2002 .

[23]  Feng Qian,et al.  Soft sensing modeling via artificial neural network based on PSO-Alopex , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[24]  R. K. Pearson,et al.  Exploring process data , 2001 .

[25]  Bor-Sen Chen,et al.  System parameter estimation with input/output noisy data and missing measurements , 2000, IEEE Trans. Signal Process..

[26]  D. Ruta,et al.  An Overview of Classifier Fusion Methods , 2000 .

[27]  Tianyou Chai,et al.  Adaptive moving window MPCA for online batch monitoring , 2004, 2004 5th Asian Control Conference (IEEE Cat. No.04EX904).

[28]  Thomas F. Edgar,et al.  Sensor Fault Identification and Reconstruction Using Principal Component Analysis , 1996 .

[29]  A Chéruy,et al.  Software sensors in bioprocess engineering , 1997 .

[30]  G.D. Gonzalez,et al.  Soft sensors for processing plants , 1999, Proceedings of the Second International Conference on Intelligent Processing and Manufacturing of Materials. IPMM'99 (Cat. No.99EX296).

[31]  Bernd Schmidt,et al.  Combining process and spectroscopic data to improve batch modeling , 2006 .

[32]  Marcel Rijckaert,et al.  Application of feedforward neural networks for soft sensors in the sugar industry , 2002, VII Brazilian Symposium on Neural Networks, 2002. SBRN 2002. Proceedings..

[33]  Guido Smits,et al.  Hybrid model development methodology for industrial soft sensors , 2003, Proceedings of the 2003 American Control Conference, 2003..

[34]  A Delgado,et al.  Functional nodes in dynamic neural networks for bioprocess modelling , 2003, Bioprocess and biosystems engineering.

[35]  Nikola Kasabov,et al.  Evolving computational intelligence systems , 2005 .

[36]  Greg Welch,et al.  An Introduction to Kalman Filter , 1995, SIGGRAPH 2001.

[37]  In-Beum Lee,et al.  Adaptive multivariate statistical process control for monitoring time-varying processes , 2006 .

[38]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[39]  M Daszykowski,et al.  Dealing with missing values and outliers in principal component analysis. , 2007, Talanta.

[40]  Thomas J. McAvoy,et al.  Nonlinear PLS Modeling Using Neural Networks , 1992 .

[41]  John F. MacGregor,et al.  Multi-way partial least squares in monitoring batch processes , 1995 .

[42]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[43]  Graham C. Goodwin,et al.  Predicting the performance of soft sensors as a route to low cost automation , 2000 .

[44]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[45]  Rubens Maciel Filho,et al.  Soft sensors development for on-line bioreactor state estimation , 2000 .

[46]  M. Dudzic,et al.  Industrial use of multivariate statistical analysis for process monitoring and control , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[47]  G. Irwin,et al.  Process monitoring approach using fast moving window PCA , 2005 .

[48]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[49]  Marcos E. Orchard,et al.  Local models for soft-sensors in a rougher flotation bank , 2003 .

[50]  Eduardo Gómez-Sánchez,et al.  Automatization of a penicillin production process with soft sensors and an adaptive controller based on neuro fuzzy systems , 2004 .

[51]  Shuang-Hua Yang,et al.  Neural network based fault diagnosis using unmeasurable inputs , 2000 .

[52]  Theodora Kourti,et al.  Process analysis and abnormal situation detection: from theory to practice , 2002 .

[53]  Mouloud Amazouz,et al.  Use of Multivariate Data Analysis for Lumber Drying Process Monitoring and Fault Detection , 2006, DMIN.

[54]  H. Hotelling.  The Generalization of Student’s Ratio , 1931 .

[55]  Andrew W. Dorsey,et al.  Monitoring of batch processes through state‐space models , 2004 .

[56]  D. Massart,et al.  Dealing with missing data: Part II , 2001 .

[57]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[58]  Andrzej Bargiela,et al.  Neural Networks Based Decision Support in Presence of Uncertainties , 1999 .

[59]  Bogdan Gabrys,et al.  Genetic algorithms in classifier fusion , 2006, Appl. Soft Comput..

[60]  Arthur K. Kordon,et al.  Competitive advantages of evolutionary computation for industrial applications , 2005, 2005 IEEE Congress on Evolutionary Computation.

[61]  Furong Gao,et al.  A soft-sensor development for melt-flow-length measurement during injection mold filling , 2004 .

[62]  Wei Shen,et al.  A soft sensor modeling approach using support vector machines , 2003, Proceedings of the 2003 American Control Conference, 2003..

[63]  S. Wold,et al.  Multi‐way principal components‐and PLS‐analysis , 1987 .

[64]  M. Rao,et al.  Soft sensors for quality prediction in batch chemical pulping processes , 1993, Proceedings of 8th IEEE International Symposium on Intelligent Control.

[65]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part I: Quantitative model-based methods , 2003, Comput. Chem. Eng..

[66]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[67]  Ilya V. Kolmanovsky,et al.  Predictive energy management of a power-split hybrid electric vehicle , 2009, 2009 American Control Conference.

[68]  Arthur K. Kordon.  Hybrid intelligent systems for industrial data analysis , 2004, Int. J. Intell. Syst..

[69]  Dale E. Seborg,et al.  DEVELOPMENT OF A SOFT SENSOR FOR A BATCH DISTILLATION COLUMN USING LINEAR AND NONLINEAR PLS REGRESSION TECHNIQUES , 2002 .

[70]  Weihua Li,et al.  Recursive PCA for adaptive process monitoring , 1999 .

[71]  E. Harth,et al.  The Alopex process: Visual receptive fields by response feedback , 1979, Biological Cybernetics.

[72]  Bogdan Gabrys,et al.  Classifier selection for majority voting , 2005, Inf. Fusion.

[73]  Paramasivan Saratchandran,et al.  Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction , 2006, Fuzzy Sets Syst..

[74]  Chonghun Han,et al.  A nonlinear soft sensor based on multivariate smoothing procedure for quality estimation in distillation columns , 2000 .

[75]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[76]  Mohamad H. Hassoun.  Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Intelligent Systems [Book review] , 1996, IEEE Transactions on Neural Networks.

[77]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part II: Qualitative models and search strategies , 2003, Comput. Chem. Eng..

[78]  L. T. Fan,et al.  Monitoring the process of curing of epoxy/graphite fiber composites with a recurrent neural network as a soft sensor , 1998 .

[79]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[80]  L. C. Zullo,et al.  Model predictive control of a slurry polymerisation reactor , 1996 .

[81]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[82]  Jose C. Principe,et al.  Neural and adaptive systems , 2000 .

[83]  D. Seborg,et al.  Estimating product composition profiles in batch distillation via partial least squares regression , 2004 .

[84]  Barry Lennox,et al.  Real-time monitoring of an industrial batch process , 2006, Comput. Chem. Eng..

[85]  H. Abdi.  Partial Least Squares (PLS) Regression. , 2003 .

[86]  Ronald Eugene Shaffer,et al.  Multi‐ and Megavariate Data Analysis. Principles and Applications, I. Eriksson, E. Johansson, N. Kettaneh‐Wold and S. Wold, Umetrics Academy, Umeå, 2001, ISBN 91‐973730‐1‐X, 533pp. , 2002 .

[87]  Scott C. James,et al.  Comparative study of black-box and hybrid estimation methods in fed-batch fermentation , 2002 .

[88]  B. K. Panigrahi,et al.  ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE , 2010 .

[89]  Dong Dong,et al.  Emission monitoring using multivariate soft sensors , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[90]  Chonghun Han,et al.  Intelligent integrated plant operation system for Six Sigma , 2002, Annu. Rev. Control..

[91]  D. Lewin,et al.  Ethylene compressor monitoring using model-based PCA , 2000 .

[92]  Howard Smith,et al.  Business Process Management: The Third Wave , 2003 .

[93]  Age K. Smilde,et al.  Statistical batch process monitoring using gray models , 2005 .

[94]  Ming-guang Zhang,et al.  Study on least squares support vector machines algorithm and its application , 2005, 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'05).

[95]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[96]  Bogdan Gabrys,et al.  Learnt Topology Gating Artificial Neural Networks , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[97]  An adaptive neuro-fuzzy inference system as a soft sensor for viscosity in rubber mixing process , 2000 .

[98]  Yin Wang,et al.  A self-organizing neural-network-based fuzzy system , 1999, Fuzzy Sets Syst..

[99]  ChangKyoo Yoo,et al.  On-line batch process monitoring using a consecutively updated multiway principal component analysis model , 2003, Comput. Chem. Eng..

[100]  Arthur K. Kordon,et al.  Robust soft sensors based on integration of genetic programming, analytical neural networks, and support vector machines , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[101]  S.Joe Qin,et al.  Neural Networks for Intelligent Sensors and Control — Practical Issues and Some Solutions , 1997 .

[102]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[103]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[104]  S. Qin.  Recursive PLS algorithms for adaptive data modeling , 1998 .

[105]  Dong Dong,et al.  Nonlinear principal component analysis-based on principal curves and neural networks , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[106]  Giorgio Valentini,et al.  Ensembles of Learning Machines , 2002, WIRN.

[107]  Francis J. Doyle,et al.  Nonlinear inferential control for process applications , 1997 .

[108]  Peter A Vanrolleghem,et al.  Monitoring of a sequencing batch reactor using adaptive multiblock principal component analysis. , 2003, Biotechnology and bioengineering.

[109]  S. Wold,et al.  PLS-regression: a basic tool of chemometrics , 2001 .

[110]  J. E. Jackson,et al.  Control Procedures for Residuals Associated With Principal Component Analysis , 1979 .

[111]  Svante Wold,et al.  Modelling and diagnostics of batch processes and analogous kinetic experiments , 1998 .

[112]  Sten Bay Jørgensen,et al.  A systematic approach for soft sensor development , 2007, Comput. Chem. Eng..

[113]  Girijesh Prasad,et al.  Statistical and computational intelligence techniques for inferential model development: a comparative evaluation and a novel proposition for fusion , 2004, Eng. Appl. Artif. Intell..

[114]  Sanjeev S. Tambe,et al.  Soft-sensor development for fed-batch bioreactors using support vector regression , 2006 .

[115]  J. Macgregor,et al.  Monitoring batch processes using multiway principal component analysis , 1994 .

[116]  Girijesh Prasad,et al.  Development of a hybrid PCA-ANFIS measurement system for monitoring product quality in the coating industry , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[117]  Lei Wang,et al.  Radial Basis Function Neural Networks-Based Modeling of the Membrane Separation Process: Hydrogen Recovery from Refinery Gases , 2006 .

[118]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[119]  Seppo J. Ovaska,et al.  Industrial applications of soft computing: a review , 2001, Proc. IEEE.

[120]  Plamen Angelov,et al.  Hybrid modelling of biotechnological processes using neural networks , 1999 .

[121]  Olli Simula,et al.  Process Monitoring and Modeling Using the Self-Organizing Map , 1999, Integr. Comput. Aided Eng..

[122]  Luigi Fortuna,et al.  Soft Sensors for Monitoring and Control of Industrial Processes (Advances in Industrial Control) , 2006 .

[123]  Detlef Nauck,et al.  Foundations Of Neuro-Fuzzy Systems , 1997 .

[124]  Age K. Smilde,et al.  Critical evaluation of approaches for on-line batch process monitoring , 2002 .

[125]  L.A.C Meleiro,et al.  A self-tuning adaptive control applied to an industrial large scale ethanol production , 2000 .

[126]  Sten Bay Jørgensen,et al.  Data-Driven Modeling of Batch Processes , 2003 .

[127]  Luigi Fortuna,et al.  Soft sensors for product quality monitoring in debutanizer distillation columns , 2005 .

[128]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[129]  Xiao Fan Wang,et al.  Soft sensing modeling based on support vector machine and Bayesian model selection , 2004, Comput. Chem. Eng..

[130]  Bogdan Gabrys,et al.  Learning hybrid neuro-fuzzy classifier models from data: to combine or not to combine? , 2004, Fuzzy Sets Syst..

[131]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[132]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[133]  Sing Kiong Nguang,et al.  Soft sensors for on-line biomass measurements , 2004, Bioprocess and biosystems engineering.

[134]  A.K. Kordon,et al.  Application issues of industrial soft computing systems , 2005, NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society.

[135]  Tiina M. Komulainen,et al.  Fault detection and isolation of an on-line analyzer for an ethylene cracking process , 2008 .

[136]  D. Dochain,et al.  On-Line Estimation and Adaptive Control of Bioreactors , 2013 .

[137]  Desire L. Massart,et al.  ROBUST PRINCIPAL COMPONENTS REGRESSION AS A DETECTION TOOL FOR OUTLIERS , 1995 .

[138]  Student,et al.  THE PROBABLE ERROR OF A MEAN , 1908 .

[139]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[140]  Tianyou Chai,et al.  Soft sensing based on artificial neural network , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[141]  Heinz Unbehauen,et al.  Fault Detection and Diagnosis with the Help of Fuzzy-Logic and with Application to a Laboratory Turbogenerator , 1996 .

[142]  Plamen P. Angelov,et al.  Flexible models with evolving structure , 2004, Int. J. Intell. Syst..

[143]  Q. Peter He,et al.  A New Fault Diagnosis Method Using Fault Directions in Fisher Discriminant Analysis , 2005 .

[144]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[145]  Barry Lennox,et al.  Integrated condition monitoring and control of fed-batch fermentation processes , 2004 .

[146]  Plamen P. Angelov,et al.  Identification of evolving fuzzy rule-based models , 2002, IEEE Trans. Fuzzy Syst..

[147]  Ana Casali,et al.  Particle size distribution soft-sensor for a grinding circuit , 1998 .

[148]  J. P. Park.  The Identification Of Multiple Outliers , 2000 .

[149]  Fu Xiao,et al.  AHU sensor fault diagnosis using principal component analysis method , 2004 .

[150]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[151]  Gerhard Widmer,et al.  Learning in the Presence of Concept Drift and Hidden Contexts , 1996, Machine Learning.

[152]  Xiaorong He,et al.  Application of steady-state detection method based on wavelet transform , 2003, Comput. Chem. Eng..

[153]  Bogdan Gabrys,et al.  Neuro-fuzzy approach to processing inputs with missing values in pattern recognition problems , 2002, Int. J. Approx. Reason..

[154]  C. S. George Lee,et al.  Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems , 1996 .

[155]  P. Angelov,et al.  A Method for Predicting Quality of the Crude Oil Distillation , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[156]  Hui Shao,et al.  Developing soft sensors using hybrid soft computing methodology: a neurofuzzy system based on rough set theory and genetic algorithms , 2006, Soft Comput..

[157]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part III: Process history based methods , 2003, Comput. Chem. Eng..

[158]  John F. MacGregor,et al.  Adaptive batch monitoring using hierarchical PCA , 1998 .

[159]  Bhupinder S. Dayal,et al.  Recursive exponentially weighted PLS and its applications to adaptive control and prediction , 1997 .

[160]  In-Beum Lee,et al.  Sensor fault identification based on time-lagged PCA in dynamic processes , 2004 .

[161]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[162]  Alan Bundy,et al.  Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence - IJCAI-95 , 1995 .

[163]  Tongwen Chen,et al.  Modeling and Identification of Multirate Systems , 2005 .

[164]  Abdul Rahman Mohamed,et al.  Neural networks for the identification and control of blast furnace hot metal quality , 2000 .

[165]  D. Massart,et al.  Dealing with missing data , 2001 .