Support vector machines resilient against training data integrity attacks

Abstract Support Vector Machines (SVMs) are vulnerable to integrity attacks, where malicious attackers distort the training data in order to compromise the decision boundary of the learned model. With increasing real-world applications of SVMs, malicious data that is classified as innocuous may have harmful consequences. This paper presents a novel framework that utilizes adversarial learning, nonlinear data projections, and game theory to improve the resilience of SVMs against such training-data-integrity attacks. The proposed approach introduces a layer of uncertainty through the use of random projections on top of the learners, making it challenging for the adversary to guess the specific configurations of the learners. To find appropriate projection directions, we introduce novel indices that ensure the contraction of the data and maximize the detection accuracy. Experiments with benchmark data sets show increases in detection rates up to 13.5% for OCSVMs and up to 14.1% for binary SVMs under different attack algorithms when compared with the respective base algorithms.

[1]  Fabio Roli,et al.  Security Evaluation of Support Vector Machines in Adversarial Environments , 2014, ArXiv.

[2]  Guodong Guo,et al.  Support Vector Machines Applications , 2014 .

[3]  James C. Bezdek,et al.  Some new indexes of cluster validity , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[4]  J. Nash.  NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[5]  James Bailey,et al.  Training robust models using Random Projection , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[6]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[7]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[8]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[9]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[10]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[11]  Bhavani M. Thuraisingham,et al.  Adversarial support vector machine learning , 2012, KDD.

[12]  András Varga,et al.  An overview of the OMNeT++ simulation environment , 2008, SimuTools.

[13]  Christopher Leckie,et al.  R1SVM: A Randomised Nonlinear Approach to Large-Scale Anomaly Detection , 2015, AAAI.

[14]  Claudia Eckert,et al.  Support vector machines under adversarial label contamination , 2015, Neurocomputing.

[15]  Pedro M. Domingos,et al.  Adversarial classification , 2004, KDD.

[16]  J. Zico Kolter,et al.  Scaling provable adversarial defenses , 2018, NeurIPS.

[17]  Marius Kloft,et al.  Security analysis of online centroid anomaly detection , 2010, J. Mach. Learn. Res..

[18]  Blaine Nelson,et al.  Poisoning Attacks against Support Vector Machines , 2012, ICML.

[19]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[20]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[21]  Christopher Leckie,et al.  Unsupervised Adversarial Anomaly Detection using One-Class Support Vector Machines , 2018 .

[22]  Percy Liang,et al.  Certified Defenses for Data Poisoning Attacks , 2017, NIPS.

[23]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[24]  Claudia Eckert,et al.  Adversarial Label Flips Attack on Support Vector Machines , 2012, ECAI.

[25]  Tansu Alpcan,et al.  Network Security , 2010 .

[26]  Christopher Leckie,et al.  Large-scale strategic games and adversarial machine learning , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[27]  Mario Michael Krell,et al.  New one-class classifiers based on the origin separation approach , 2015, Pattern Recognit. Lett..

[28]  Christos Boutsidis,et al.  Random Projections for Linear Support Vector Machines , 2012, TKDD.

[29]  Christopher Leckie,et al.  Detection of Anomalous Communications with SDRs and Unsupervised Adversarial Learning , 2018, 2018 IEEE 43rd Conference on Local Computer Networks (LCN).

[30]  Marimuthu Palaniswami,et al.  Pattern based anomalous user detection in cognitive radio networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Ricky Laishram,et al.  Curie: A method for protecting SVM Classifier from Poisoning Attack , 2016, ArXiv.

[32]  Blaine Nelson,et al.  Support Vector Machines Under Adversarial Label Noise , 2011, ACML.

[33]  J. Doug Tygar,et al.  Adversarial machine learning , 2019, AISec '11.

[34]  Christopher Leckie,et al.  High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning , 2016, Pattern Recognit..

[35]  Fabio Roli,et al.  Evasion Attacks against Machine Learning at Test Time , 2013, ECML/PKDD.

[36]  Yevgeniy Vorobeychik,et al.  A General Retraining Framework for Scalable Adversarial Classification , 2016, ArXiv.

[37]  Shie Mannor,et al.  Robustness and Regularization of Support Vector Machines , 2008, J. Mach. Learn. Res..