A Minimum Volume Covering Approach with a Set of Ellipsoids

A technique for adjusting a minimum volume set of covering ellipsoids technique is elaborated. Solutions to this problem have potential application in one-class classification and clustering problems. Its main original features are: 1) It avoids the direct evaluation of determinants by using diagonalization properties of the involved matrices, 2) it identifies and removes outliers from the estimation process, 3) it avoids binary variables resulting from the combinatorial character of the assignment problem that are replaced by continuous variables in the range [0, 1], 4) the problem can be solved by a bilevel algorithm that in its first level determines the ellipsoids and in its second level reassigns the data points to ellipsoids and identifies outliers based on an algorithm that forces the Karush-Kuhn-Tucker conditions to be satisfied. Two theorems provide rigorous bases for the proposed methods. Finally, a set of examples of application in different fields is given to illustrate the power of the method and its practical performance.

[1]  Leonid Khachiyan,et al.  On the complexity of approximating the maximal inscribed ellipsoid for a polytope , 1993, Math. Program..

[2]  E. Barnes.  An algorithm for separating patterns by ellipsoids , 1982 .

[3]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[4]  Nathalie Japkowicz,et al.  Supervised Versus Unsupervised Binary-Learning by Feedforward Neural Networks , 2004, Machine Learning.

[5]  P. D. McFadden,et al.  Model for the vibration produced by a single point defect in a rolling element bearing , 1984 .

[6]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[7]  Michael N. Vrahatis,et al.  The New k-Windows Algorithm for Improving the k-Means Clustering Algorithm , 2002, J. Complex..

[8]  Enrique F. Castillo,et al.  A General Method for Local Sensitivity Analysis With Application to Regression Models and Other Optimization Problems , 2004, Technometrics.

[9]  Enrique Castillo,et al.  Building and Solving Mathematical Programming Models in Engineering and Science , 2001 .

[10]  Leonid Khachiyan,et al.  Rounding of Polytopes in the Real Number Model of Computation , 1996, Math. Oper. Res..

[11]  Bernhard Schölkopf,et al.  Support Vector Novelty Detection Applied to Jet Engine Vibration Spectra , 2000, NIPS.

[12]  Antonio J. Conejo,et al.  Decomposition Techniques in Mathematical Programming: Engineering and Science Applications , 2006 .

[13]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[14]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[15]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[16]  A. Conejo,et al.  Perturbation Approach to Sensitivity Analysis in Mathematical Programming , 2006 .

[17]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[18]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[19]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[20]  Antonio J. Conejo,et al.  A closed formula for local sensitivity analysis in mathematical programming , 2006 .

[21]  Amparo Alonso-Betanzos,et al.  Power wind mill fault detection via one-class ν-SVM vibration signal analysis , 2011, The 2011 International Joint Conference on Neural Networks.

[22]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[23]  Kim-Chuan Toh,et al.  Solving Log-Determinant Optimization Problems by a Newton-CG Primal Proximal Point Algorithm , 2010, SIAM J. Optim..

[24]  Pannagadatta K. Shivaswamy.  Ellipsoidal Kernel Machines , 2007 .

[25]  James I. Taylor,et al.  The Vibration Analysis Handbook , 1994 .

[26]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[27]  Piyush Kumar,et al.  Minimum-Volume Enclosing Ellipsoids and Core Sets , 2005 .

[28]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[29]  Antonio J. Conejo,et al.  Closed formulas in local sensitivity analysis for some classes of linear and non-linear problems , 2007 .

[30]  Nathalie Japkowicz,et al.  Concept learning in the absence of counterexamples: an autoassociation-based approach to classification , 1999 .

[31]  Alexander Hinneburg,et al.  DENCLUE 2.0: Fast Clustering Based on Kernel Density Estimation , 2007, IDA.

[32]  D. Harville.  Matrix Algebra From a Statistician's Perspective , 1998 .

[33]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[34]  F. John.  Extremum Problems with Inequalities as Subsidiary Conditions , 2014 .

[35]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[36]  Heiko Hoffmann,et al.  Kernel PCA for novelty detection , 2007, Pattern Recognit..

[37]  W. R. Buckland,et al.  Outliers in Statistical Data , 1979 .

[38]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[39]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[40]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[41]  John Shawe-Taylor,et al.  The Minimum Volume Covering Ellipsoid Estimation in Kernel-Defined Feature Spaces , 2006, ECML.

[42]  Michael J. Todd,et al.  On Khachiyan's algorithm for the computation of minimum-volume enclosing ellipsoids , 2007, Discret. Appl. Math..

[43]  Panu Somervuo,et al.  Self-Organizing Maps and Learning Vector Quantization for Feature Sequences , 1999, Neural Processing Letters.

[44]  Christopher Leckie,et al.  An efficient hyperellipsoidal clustering algorithm for resource-constrained environments , 2011, Pattern Recognit..

[45]  Peng Sun,et al.  Computation of Minimum Volume Covering Ellipsoids , 2002, Oper. Res..

[46]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[47]  Robert P. W. Duin,et al.  Robust machine fault detection with independent component analysis and support vector data description , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[48]  Pasi Fränti,et al.  Iterative shrinking method for clustering problems , 2006, Pattern Recognit..