Process monitoring using variational autoencoder for high-dimensional nonlinear processes

Abstract In many industries, statistical process monitoring techniques play a key role in improving processes through variation reduction and defect prevention. Modern large-scale industrial processes require appropriate monitoring techniques that can efficiently address high-dimensional nonlinear processes. Such processes have been successfully monitored with several latent variable-based methods. However, because these monitoring methods use Hotelling’s T 2 statistics in the reduced space, a normality assumption underlies the construction of these tools. This assumption has limited the use of latent variable-based monitoring charts in both nonlinear and nonnormal situations. In this study, we propose a variational autoencoder (VAE) as a monitoring method that can address both nonlinear and nonnormal situations in high-dimensional processes. VAE is appropriate for T 2 charts because it causes the reduced space to follow a multivariate normal distribution. The effectiveness and applicability of the proposed VAE-based chart were demonstrated through experiments on simulated data and real data from a thin-film-transistor liquid-crystal display process.

[1]  Barry M. Wise,et al.  The process chemometrics approach to process monitoring and fault detection , 1995 .

[2]  Douglas C. Montgomery,et al.  Introduction to Statistical Quality Control , 1986 .

[3]  Zhiqiang Ge,et al.  Improved kernel PCA-based monitoring approach for nonlinear processes , 2009 .

[4]  Nan Li,et al.  Ensemble Kernel Principal Component Analysis for Improved Nonlinear Process Monitoring , 2015 .

[5]  Theodora Kourti,et al.  Application of latent variable methods to process control and multivariate statistical process control in industry , 2005 .

[6]  William H. Woodall,et al.  Controversies and Contradictions in Statistical Process Control , 2000 .

[7]  C. Yoo,et al.  Nonlinear process monitoring using kernel principal component analysis , 2004 .

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  J. Edward Jackson,et al.  Quality Control Methods for Several Related Variables , 1959 .

[10]  Joseph Aguilar-Martin,et al.  Automaton based on fuzzy clustering methods for monitoring industrial processes , 2013, Eng. Appl. Artif. Intell..

[11]  T. McAvoy,et al.  Nonlinear principal component analysis—Based on principal curves and neural networks , 1996 .

[12]  Thong Ngee Goh,et al.  CONTROL CHART FOR MULTIVARIATE ATTRIBUTE PROCESSES , 1998 .

[13]  Seoung Bum Kim,et al.  Principal component analysis-based control charts for multivariate nonnormal distributions , 2013, Expert Syst. Appl..

[14]  Xu-Cheng Yin,et al.  A Denoising Based Autoassociative Model for Robust Sensor Monitoring in Nuclear Power Plants , 2016 .

[15]  I. Jolliffe.  Principal Component Analysis and Factor Analysis , 1986 .

[16]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[17]  Seoung Bum Kim,et al.  One-class classification-based control charts for multivariate process monitoring , 2009 .

[18]  J. E. Jackson,et al.  Control Procedures for Residuals Associated With Principal Component Analysis , 1979 .

[19]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[20]  Jonathon Shlens,et al.  A Tutorial on Principal Component Analysis , 2014, ArXiv.

[21]  Hoang Pham.  Recent Advances in Reliability and Quality Engineering , 2001, Series on Quality, Reliability and Engineering Statistics.

[22]  H. Hotelling,et al.  Multivariate Quality Control , 1947 .

[23]  Min-Sen Chiu,et al.  Nonlinear process monitoring using JITL-PCA , 2005 .

[24]  Måns Thulin,et al.  A high-dimensional two-sample test for the mean using random subspaces , 2013, Comput. Stat. Data Anal..

[25]  Theodora Kourti,et al.  Process analysis, monitoring and diagnosis, using multivariate projection methods , 1995 .

[26]  Jin Hyun Park,et al.  Fault detection and identification of nonlinear processes based on kernel PCA , 2005 .

[27]  Zhiqiang Ge,et al.  Process Data Analytics via Probabilistic Latent Variable Models: A Tutorial Review , 2018, Industrial & Engineering Chemistry Research.

[28]  Xu Yongmao,et al.  Multivariate Statistical Process Monitoring Using Robust Nonlinear Principal Component Analysis , 2005 .

[29]  Seungjin Choi,et al.  Echo-state conditional variational autoencoder for anomaly detection , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[30]  Carl Doersch,et al.  Tutorial on Variational Autoencoders , 2016, ArXiv.

[31]  Joaquím Meléndez,et al.  N-dimensional extension of unfold-PCA for granular systems monitoring , 2018, Eng. Appl. Artif. Intell..

[32]  Furong Gao,et al.  Review of Recent Research on Data-Based Process Monitoring , 2013 .

[33]  Douglas C. Montgomery,et al.  Statistical process monitoring with principal components , 1996 .

[34]  Zhiqiang Ge,et al.  Review on data-driven modeling and monitoring for plant-wide industrial processes , 2017 .

[35]  D L Massart,et al.  A journey into low-dimensional spaces with autoassociative neural networks. , 2003, Talanta.

[36]  Age K. Smilde,et al.  Critical evaluation of approaches for on-line batch process monitoring , 2002 .

[37]  John C. Young,et al.  Multivariate statistical process control with industrial applications , 2002, ASA-SIAM series on statistics and applied probability.

[38]  Yang Hu,et al.  Fault detection based on signal reconstruction with Auto-Associative Extreme Learning Machines , 2017, Eng. Appl. Artif. Intell..

[39]  Zhiqiang Ge,et al.  Deep Learning of Semisupervised Process Data With Hierarchical Extreme Learning Machine and Soft Sensor Application , 2018, IEEE Transactions on Industrial Electronics.

[40]  Zhiqiang Ge,et al.  Probabilistic Sequential Network for Deep Learning of Complex Process Data and Soft Sensor Application , 2019, IEEE Transactions on Industrial Informatics.

[41]  Christos Georgakis,et al.  Disturbance detection and isolation by dynamic principal component analysis , 1995 .

[42]  Daniel Aguado,et al.  Multivariate statistical monitoring of continuous wastewater treatment plants , 2008, Eng. Appl. Artif. Intell..

[43]  Ping Zhang,et al.  A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process , 2012 .

[44]  Jin Wang,et al.  Statistical process monitoring as a big data analytics tool for smart manufacturing , 2017, Journal of Process Control.

[45]  Chenglin Wen,et al.  Fault diagnosis based on deep learning , 2016, 2016 American Control Conference (ACC).

[46]  Douglas C. Montgomery,et al.  A review of multivariate control charts , 1995 .

[47]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[48]  Thomas F. Edgar,et al.  Process Dynamics and Control , 1989 .

[49]  J. Royston.  An Extension of Shapiro and Wilk's W Test for Normality to Large Samples , 1982 .

[50]  J. Romagnoli,et al.  A multi-scale orthogonal nonlinear strategy for multi-variate statistical process monitoring , 2006 .

[51]  G. Box.  Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification , 1954 .