Kernel methods for changes detection in covariance matrices

ABSTRACT Several methods have been proposed to solve the one-class classification problem for vectors. Three methods are mainly used: density estimation, boundary methods, and reconstruction methods. The focus here is on boundary methods. These methods include the k-center method, the nearest neighbor method, one-class support vector machine (OCSVM), and the support vector data description (SVDD). In industrial applications, like statistical process control (SPC), practitioners successfully used SVDD to detect anomalies or outliers in the process. However, when a multivariate process shifts, it occurs in either location or scale. This far, most of the research effort, like the OCSVM or SVDD, has focused on location (vectors or mean vectors). Several methods have been proposed recently to monitor the scale, i.e., the covariance matrix. Most of these methods deal with a full rank covariance matrix, i.e., a situation where the number of rational subgroups is larger than the number of variables. When the number of variables is nearly as large as, or larger than, the number of observations, most Shewhart-type charts are unable to solve this problem as the estimated covariance matrix is not full rank. This work will extend the one-class classification method using kernels to detect changes in the covariance matrix when the number of observations available is less than the number of variables.

[1]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[2]  Chien-Wei Wu,et al.  A multivariate EWMA control chart for monitoring process variability with individual observations , 2005 .

[3]  Arthur B. Yeh,et al.  Monitoring the covariance matrix via penalized likelihood estimation , 2013 .

[4]  Douglas M. Hawkins,et al.  Self-Starting Multivariate Exponentially Weighted Moving Average Control Charting , 2007, Technometrics.

[5]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[6]  Chien-Wei Wu,et al.  Monitoring Multivariate Process Variability for Individual Observations , 2007 .

[7]  Wei-Cheng Chang A Revisit to Support Vector Data Description ( SVDD ) , 2013 .

[8]  Bo Li,et al.  Simultaneous monitoring of process mean vector and covariance matrix via penalized likelihood estimation , 2014, Comput. Stat. Data Anal..

[9]  Edgard M. Maboudou-Tchao,et al.  A LASSO Chart for Monitoring the Covariance Matrix , 2013 .

[10]  Zhisheng Zhang,et al.  SVC-Based Multivariate Control Charts for Automatic Anomaly Detection in Computer Networks , 2007, Third International Conference on Autonomic and Autonomous Systems (ICAS'07).

[11]  Fugee Tsung,et al.  A kernel-distance-based multivariate control chart using support vector methods , 2003 .

[12]  Arthur B. Yeh,et al.  Monitoring multivariate process variability with individual observations via penalised likelihood estimation , 2012 .

[13]  Joseph J. Pignatiello,et al.  Monitoring Process Dispersion without Subgrouping , 2000 .

[14]  Hongdong Li,et al.  Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Edgard M. Maboudou-Tchao,et al.  Monitoring the covariance matrix with fewer observations than variables , 2013, Comput. Stat. Data Anal..

[16]  Douglas M. Hawkins,et al.  Self-Starting Multivariate Control Charts for Location and Scale , 2011 .

[17]  H. Hotelling,et al.  Multivariate Quality Control , 1947 .

[18]  Charles W. Champ,et al.  The Performance of Control Charts for Monitoring Process Variation , 1995 .

[19]  Gyo-Young Cho,et al.  Multivariate Control Charts for Monitoring the Mean Vector and Covariance Matrix , 2006 .

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  Manoj Kumar Tiwari,et al.  Kernel distance-based robust support vector methods and its application in developing a robust K-chart , 2006 .

[22]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[23]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[24]  Douglas M. Hawkins,et al.  Multivariate Exponentially Weighted Moving Covariance Matrix , 2008, Technometrics.