Efficient Data Point Pruning for One-Class SVM

One-class SVM is a popular method for one-class classification but it needs high computation cost. This paper proposes Quix as an efficient training algorithm for one-class SVM. It prunes unnecessary data points before applying the SVM solver by computing upper and lower bounds of a parameter that determines the hyper-plane. Since we can efficiently check optimality of the hyper-plane by using the bounds, it guarantees the identical classification results to the original approach. Experiments show that it is up to 6800 times faster than existing approaches without degrading optimality.

[1]  Cameron Musco,et al.  Recursive Sampling for the Nystrom Method , 2016, NIPS.

[2]  Koh Takeuchi,et al.  SVD-Based Screening for the Graphical Lasso , 2017, IJCAI.

[3]  Tomoharu Iwata,et al.  Inferring Latent Triggers of Purchases with Consideration of Social Effects and Media Advertisements , 2016, WSDM '16.

[4]  Machiko Toyoda,et al.  Adaptive Message Update for Fast Affinity Propagation , 2015, KDD.

[5]  Jie Chen,et al.  Revisiting Random Binning Features: Fast Convergence and Strong Parallelizability , 2016, KDD.

[6]  Koh Takeuchi,et al.  Scaling Locally Linear Embedding , 2017, SIGMOD Conference.

[7]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[8]  Yasuhiro Fujiwara,et al.  Linked taxonomies to capture users' subjective assessments of items to facilitate accurate collaborative filtering , 2014, Artif. Intell..

[9]  Valeria V. Krzhizhanovskaya,et al.  Crack Detection in Earth Dam and Levee Passive Seismic Data Using Support Vector Machines , 2016, ICCS.

[10]  D. J. H. Garling,et al.  The Cauchy-Schwarz Master Class: An Introduction to the Art of Mathematical Inequalities by J. Michael Steele , 2005, Am. Math. Mon..

[11]  James A. Hendler,et al.  Semantic Data Representation for Improving Tensor Factorization , 2014, AAAI.

[12]  Rong Jin,et al.  Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison , 2012, NIPS.

[13]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[14]  Shambhu J. Upadhyaya,et al.  User authentication with keystroke dynamics in long-text data , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[15]  Petros Drineas,et al.  On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning , 2005, J. Mach. Learn. Res..

[16]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[17]  Klaus-Robert Müller,et al.  Incremental Support Vector Learning: Analysis, Implementation and Applications , 2006, J. Mach. Learn. Res..

[18]  Paul Honeine,et al.  Online one-class machines based on the coherence criterion , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[19]  Miguel Lázaro-Gredilla,et al.  Adaptive One-Class Support Vector Machine , 2011, IEEE Transactions on Signal Processing.

[20]  Yasuhiro Fujiwara,et al.  Madeus: Database Live Migration Middleware under Heavy Workloads for Cloud Environment , 2015, SIGMOD Conference.

[21]  Yasuhiro Fujiwara,et al.  Fast and Exact Top-k Algorithm for PageRank , 2013, AAAI.

[22]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[23]  Rong Jin,et al.  Fast and Accurate Refined Nyström-Based Kernel SVM , 2016, AAAI.

[24]  Katelyn Gao.  Online One-Class SVMs with Active-Set Optimization for Data Streams , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).