Maximum Volume Outlier Detection and its Applications in Credit Risk Analysis

Because of the scarcity and diversity of outliers, it is very difficult to design a robust outlier detector. In this paper, we first propose to use the maximum margin criterion to sift unknown outliers, which demonstrates superior performance. However, the resultant learning task is formulated as a Mixed Integer Programming (MIP) problem, which is computationally hard. Therefore, we alter the recently developed label generating technique, which efficiently solves a convex relaxation of the MIP problem of outlier detection. Specifically, we propose an effective procedure to find a largely violated labeling vector for identifying rare outliers from abundant normal patterns, and its convergence is also presented. Then, a set of largely violated labeling vectors are combined by multiple kernel learning methods to robustly detect outliers. Besides these, in order to further enhance the efficacy of our outlier detector, we also explore the use of maximum volume criterion to measure the quality of separation between outliers and normal patterns. This criterion can be easily incorporated into our proposed framework by introducing an additional regularization. Comprehensive experiments on toy and real-world data sets verify that the outlier detectors using the two proposed criteria outperform existing outlier detection methods. Furthermore, our models are employed to detect corporate credit risk and demonstrate excellent performance.

[1]  Ivor W. Tsang,et al.  Text-based image retrieval using progressive multi-instance learning , 2011, 2011 International Conference on Computer Vision.

[2]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[3]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[4]  Zengyou He,et al.  Discovering cluster-based local outliers , 2003, Pattern Recognit. Lett..

[5]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[6]  Taghi M. Khoshgoftaar,et al.  Evaluating indirect and direct classification techniques for network intrusion detection , 2005, Intell. Data Anal..

[7]  Manik Varma,et al.  More generality in efficient multiple kernel learning , 2009, ICML '09.

[8]  Jieping Ye,et al.  A Small Sphere and Large Margin Approach for Novelty Detection Using Training Data with Outliers , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  R. Horst,et al.  Global Optimization: Deterministic Approaches , 1992 .

[10]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[11]  Xuxian Jiang,et al.  vEye: behavioral footprinting for self-propagating worm detection and profiling , 2008, Knowledge and Information Systems.

[12]  Sebastian Nowozin,et al.  Infinite Kernel Learning , 2008, NIPS 2008.

[13]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[14]  Gang Niu,et al.  Maximum Volume Clustering , 2011, AISTATS.

[15]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[16]  Ivor W. Tsang,et al.  Maximum Margin Clustering Made Practical , 2009, IEEE Trans. Neural Networks.

[17]  Jing Gao,et al.  Semi-supervised outlier detection , 2006, SAC '06.

[18]  Ivor W. Tsang,et al.  Relevance vector machine based infinite decision agent ensemble learning for credit risk analysis , 2012, Expert Syst. Appl..

[19]  Ashok N. Srivastava,et al.  Multiple kernel learning for heterogeneous anomaly detection: algorithm and aviation safety case study , 2010, KDD.

[20]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[21]  Fei Tony Liu,et al.  Isolation-Based Anomaly Detection , 2012, TKDD.

[22]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD 2000.

[23]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[24]  Matthias Hein,et al.  Measure Based Regularization , 2003, NIPS.

[25]  Ivor W. Tsang,et al.  Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets , 2010, ICML.

[26]  L. Hogben Handbook of Linear Algebra , 2006 .

[27]  Sanjay Chawla,et al.  Finding Local Anomalies in Very High Dimensional Space , 2010, 2010 IEEE International Conference on Data Mining.

[28]  Ran El-Yaniv,et al.  Large margin vs. large volume in transductive learning , 2008, Machine Learning.

[29]  Filip Radlinski,et al.  A support vector method for optimizing average precision , 2007, SIGIR.

[30]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[31]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[32]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[33]  Ivor W. Tsang,et al.  Tighter and Convex Maximum Margin Clustering , 2009, AISTATS.

[34]  Takehisa Yairi,et al.  An approach to spacecraft anomaly detection problem using kernel feature space , 2005, KDD '05.

[35]  Jian-Bo Yang,et al.  Hierarchical Maximum Margin Learning for Multi-Class Classification , 2011, UAI.

[36]  Ivor W. Tsang,et al.  A Convex Method for Locating Regions of Interest with Multi-instance Learning , 2009, ECML/PKDD.

[37]  Xindong Wu,et al.  Vague One-Class Learning for Data Streams , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[38]  Damminda Alahakoon,et al.  Minority report in fraud detection: classification of skewed data , 2004, SKDD.

[39]  Ivor W. Tsang,et al.  Learning the Kernel in Mahalanobis One-Class Support Vector Machines , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[40]  Le Song,et al.  Relative Novelty Detection , 2009, AISTATS.

[41]  Philip S. Yu,et al.  Exploiting Local Data Uncertainty to Boost Global Outlier Detection , 2010, 2010 IEEE International Conference on Data Mining.

[42]  Gilles Blanchard,et al.  Semi-Supervised Novelty Detection , 2010, J. Mach. Learn. Res..

[43]  Zeeshan Syed,et al.  Unsupervised Risk Stratification in Clinical Datasets: Identifying Patients at Risk of Rare Outcomes , 2010, ICML.

[44]  Bianca Zadrozny,et al.  Outlier detection by active learning , 2006, KDD '06.