Detecting abnormal situations using the Kullback-Leibler divergence

This article develops statistics based on the Kullback-Leibler (KL) divergence to monitor large-scale technical systems. These statistics detect anomalous system behavior by comparing estimated density functions for the current process behavior with reference density functions. For Gaussian distributed process variables, the paper proves that the difference in density functions, measured by the KL divergence, is a more sensitive measure than existing work involving multivariate statistics. To cater for a wide range of potential application areas, the paper develops monitoring concepts for linear static systems, that can produce Gaussian as well as non-Gaussian distributed process variables. Using recorded data from a glass melter, the article demonstrates the increased sensitivity of the KL-based statistics by comparing them to competitive ones.

[1]  Xun Wang,et al.  Nonlinear PCA With the Local Approach for Diesel Engine Fault Detection and Diagnosis , 2008, IEEE Transactions on Control Systems Technology.

[2]  Douglas C. Montgomery,et al.  Applied Statistics and Probability for Engineers, Third edition , 1994 .

[3]  Michael S. Dudzic,et al.  An industrial perspective on implementing on-line applications of multivariate statistics , 2004 .

[4]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part III: Process history based methods , 2003, Comput. Chem. Eng..

[5]  C. Chu,et al.  Semiparametric density estimation under a two-sample density ratio model , 2004 .

[6]  Sirish L. Shah,et al.  Model Identification and Error Covariance Matrix Estimation from Noisy Data Using PCA , 2004 .

[7]  Steffen Bickel,et al.  Dirichlet-Enhanced Spam Filtering based on Biased Samples , 2006, NIPS.

[8]  Manabu Kano,et al.  Evolution of multivariate statistical process control: application of independent component analysis and external analysis , 2004, Comput. Chem. Eng..

[9]  Josep Lladós,et al.  A similarity measure between vector sequences with application to handwritten word image retrieval , 2009, CVPR.

[10]  Takafumi Kanamori,et al.  A Least-squares Approach to Direct Importance Estimation , 2009, J. Mach. Learn. Res..

[11]  Barry Lennox,et al.  Monitoring a complex refining process using multivariate statistics , 2008 .

[12]  Richard E. Blahut,et al.  Principles and practice of information theory , 1987 .

[13]  Uwe Kruger,et al.  Improved principal component monitoring using the local approach , 2007, Autom..

[14]  Jordi Inglada,et al.  Change detection on SAR images by using a parametric estimation of the Kullback-Leibler divergence , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[15]  Thomas F. Edgar,et al.  Identification of faulty sensors using principal component analysis , 1996 .

[16]  Zhiqiang Ge,et al.  Local ICA for multivariate statistical fault diagnosis in systems with unknown signal and error distributions , 2012 .

[17]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[18]  Lei Xie,et al.  Developments and Applications of Nonlinear Principal Component Analysis – a Review , 2008 .

[19]  L. L. Cam,et al.  Asymptotic methods in statistical theory , 1986 .

[20]  G. Irwin,et al.  Process monitoring approach using fast moving window PCA , 2005 .

[21]  John R. Hershey,et al.  Approximating the Kullback Leibler Divergence Between Gaussian Mixture Models , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[22]  Uwe Kruger,et al.  Regularised kernel density estimation for clustered process data , 2004 .

[23]  E. Lehmann.  Elements of large-sample theory , 1998 .

[24]  Dmitry I. Belov,et al.  Distributions of the Kullback-Leibler divergence with applications. , 2011, The British journal of mathematical and statistical psychology.

[25]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[26]  E. Lima,et al.  A unified statistical framework for monitoring multivariate systems with unknown source and error signals , 2010 .

[27]  Lei Xie,et al.  Statistical‐based monitoring of multivariate non‐Gaussian systems , 2008 .

[28]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[29]  In-Beum Lee,et al.  Fault detection and diagnosis based on modified independent component analysis , 2006 .

[30]  Theodora Kourti,et al.  Application of latent variable methods to process control and multivariate statistical process control in industry , 2005 .