Evasion Attacks against Machine Learning at Test Time

In security-sensitive applications, the success of machine learning depends on a thorough vetting of their resistance to adversarial data. In one pertinent, well-motivated attack scenario, an adversary may attempt to evade a deployed system at test time by carefully manipulating attack samples. In this work, we present a simple but effective gradient-based approach that can be exploited to systematically assess the security of several, widely-used classification algorithms against evasion attacks. Following a recently proposed framework for security evaluation, we simulate attack scenarios that exhibit different risk levels for the classifier by increasing the attacker's knowledge of the system and her ability to manipulate attack samples. This gives the classifier designer a better picture of the classifier performance under evasion attacks, and allows him to perform a more informed model selection (or parameter setting). We evaluate our approach on the relevant security task of malware detection in PDF files, and show that such systems can be easily evaded. We also sketch some countermeasures suggested by our analysis.

[1]  Harris Drucker,et al.  Comparison of learning algorithms for handwritten digit recognition , 1995 .

[2]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[3]  Polina Golland Discriminative Direction for Kernel Classifiers , 2001, NIPS.

[4]  Pedro M. Domingos,et al.  Adversarial classification , 2004, KDD.

[5]  Christopher Meek,et al.  Adversarial learning , 2005, KDD '05.

[6]  Wenke Lee,et al.  Polymorphic Blending Attacks , 2006, USENIX Security Symposium.

[7]  Blaine Nelson,et al.  Can machine learning be secure? , 2006, ASIACCS '06.

[8]  Amir Globerson,et al.  Nightmare at test time: robust learning by feature deletion , 2006, ICML.

[9]  Blaine Nelson,et al.  Exploiting Machine Learning to Subvert Your Spam Filter , 2008, LEET.

[10]  Ohad Shamir,et al.  Learning to classify with missing and corrupted features , 2008, ICML '08.

[11]  Marius Kloft,et al.  A framework for quantitative security analysis of machine learning , 2009, AISec '09.

[12]  Aleksander Kolcz,et al.  Feature Weighting for Improved Classifier Robustness , 2009, CEAS 2009.

[13]  Marius Kloft,et al.  Online Anomaly Detection under Adversarial Impact , 2010, AISTATS.

[14]  Fabio Roli,et al.  Multiple classifier systems for robust classifier design in adversarial environments , 2010, Int. J. Mach. Learn. Cybern..

[15]  Tobias Scheffer,et al.  Stackelberg games for adversarial prediction problems , 2011, KDD.

[16]  J. Doug Tygar,et al.  Adversarial machine learning , 2019, AISec '11.

[17]  Fabio Roli,et al.  Design of robust classifiers for adversarial environments , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[18]  Tobias Scheffer,et al.  Static prediction games for adversarial learning problems , 2012, J. Mach. Learn. Res..

[19]  Giorgio Giacinto,et al.  A Pattern Recognition System for Malicious PDF Files Detection , 2012, MLDM.

[20]  Blaine Nelson,et al.  Poisoning Attacks against Support Vector Machines , 2012, ICML.

[21]  Ling Huang,et al.  Query Strategies for Evading Convex-Inducing Classifiers , 2010, J. Mach. Learn. Res..

[22]  Angelos Stavrou,et al.  Malicious PDF detection using metadata and structural features , 2012, ACSAC '12.

[23]  Pavel Laskov,et al.  Detection of Malicious PDF Files Based on Hierarchical Document Structure , 2013, NDSS.

[24]  Fabio Roli,et al.  Security Evaluation of Pattern Classifiers under Attack , 2014, ArXiv.