Low-Rank Outlier Detection

In this chapter, we present a novel low-rank outlier detection approach, which incorporates a low-rank constraint into the support vector data description (SVDD) model. Different from the traditional SVDD, our approach learns multiple hyper-spheres to fit the normal data. The low-rank constraint helps us group the complicated dataset into several clusters dynamically. We present both primal and dual solutions to solve this problem, and provide the detailed strategy of outlier detection. Moreover, the kernel-trick used in SVDD becomes unnecessary in our approach, which implies that the training time and memory space could be substantially reduced. The performance of our approach, along with other related methods, was evaluated using three image databases. Results show our approach outperforms other methods in most scenarios.

[1]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[2]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Kun Li,et al.  Efficient Clustering-Based Outlier Detection Algorithm for Dynamic Data Stream , 2008, 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery.

[4]  Trung Le,et al.  Multi-Sphere Support Vector Clustering , 2011, ICONIP.

[5]  Ping Chen,et al.  Using the fractal dimension to cluster datasets , 2000, KDD '00.

[6]  Xindong Wu,et al.  Multi-sphere Support Vector Data Description for Outliers Detection on Multi-distribution Data , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[7]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[8]  Markus Breitenbach,et al.  Clustering through ranking on manifolds , 2005, ICML '05.

[9]  Trung Le,et al.  A Theoretical Framework for Multi-sphere Support Vector Data Description , 2010, ICONIP.

[10]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[11]  Shashi Shekhar,et al.  Detecting graph-based spatial outliers: algorithms and applications (a summary of results) , 2001, KDD '01.

[12]  Ivor W. Tsang,et al.  Maximum Margin/Volume Outlier Detection , 2011, 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence.

[13]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[14]  Trung Le,et al.  Proximity multi-sphere support vector clustering , 2013, Neural Computing and Applications.

[15]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[16]  Salvatore J. Stolfo,et al.  One Class Support Vector Machines for Detecting Anomalous Windows Registry Accesses , 2003 .

[17]  Vic Barnett,et al.  Outliers in Statistical Data , 1980 .

[18]  Hongxing He,et al.  Outlier Detection Using Replicator Neural Networks , 2002, DaWaK.

[19]  Michael Georgiopoulos,et al.  A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes , 2010, Data Mining and Knowledge Discovery.

[20]  Ming Shao,et al.  Locality linear fitting one-class SVM with low-rank constraints for outlier detection , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[21]  Larry S. Davis,et al.  Learning Structured Low-Rank Representations for Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Yun Fu,et al.  Robust Subspace Discovery through Supervised Low-Rank Constraints , 2014, SDM.

[23]  Hongxing He,et al.  A comparative study of RNN for outlier detection in data mining , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[24]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[25]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[26]  Anthony K. H. Tung,et al.  Mining top-n local outliers in large databases , 2001, KDD '01.

[27]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[28]  Stephen D. Bay,et al.  Mining distance-based outliers in near linear time with randomization and a simple pruning rule , 2003, KDD '03.

[29]  Eric J. Pauwels,et al.  One Class Classification for Anomaly Detection: Support Vector Data Description Revisited , 2011, ICDM.

[30]  Yun Fu,et al.  Low-Rank Coding with b-Matching Constraint for Semi-Supervised Classification , 2013, IJCAI.

[31]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[32]  Xiaowei Zhou,et al.  Automatic mitral leaflet tracking in echocardiography by outlier detection in the low-rank representation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  LinLin Shen,et al.  Directional binary code with application to PolyU near-infrared face database , 2010, Pattern Recognit. Lett..

[34]  Jun-Geol Baek,et al.  Density weighted support vector data description , 2014, Expert Syst. Appl..

[35]  Francis R. Bach,et al.  Consistency of trace norm minimization , 2007, J. Mach. Learn. Res..

[36]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[37]  Ning Ye,et al.  Boundary detection and sample reduction for one-class Support Vector Machines , 2014, Neurocomputing.

[38]  Amit Banerjee,et al.  Fast Hyperspectral Anomaly Detection via SVDD , 2007, 2007 IEEE International Conference on Image Processing.

[39]  Yun Fu,et al.  Discriminative dictionary learning with low-rank regularization for face recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[40]  Stefanie Nowak,et al.  Using one-class SVM outliers detection for verification of collaboratively tagged image training sets , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[41]  Pei-Yi Hao,et al.  A New Multi-class Support Vector Machine with Multi-sphere in the Feature Space , 2007, IEA/AIE.

[42]  Longbing Cao,et al.  SVDD-based outlier detection on uncertain data , 2012, Knowledge and Information Systems.