Data Driven Exploratory Attacks on Black Box Classifiers in Adversarial Domains

New data driven framework for simulating exploratory attacks on black box classifiers.Algorithms for simple evasion attacks, to more sophisticated reverse engineering attacks.Formal adversarial model and metrics for baseline evaluation of secure learning strategies.Experimental evaluation on 10 datasets, with linear and non-linear defender models.Experimental evaluation on the black box Google Cloud Platform classifier system. While modern day web applications aim to create impact at the civilization level, they have become vulnerable to adversarial activity, where the next cyber-attack can take any shape and can originate from anywhere. The increasing scale and sophistication of attacks, has prompted the need for a data driven solution, with machine learning forming the core of many cybersecurity systems. Machine learning was not designed with security in mind and the essential assumption of stationarity, requiring that the training and testing data follow similar distributions, is violated in an adversarial domain. In this paper, an adversarys view point of a classification based system, is presented. Based on a formal adversarial model, the Seed-Explore-Exploit framework is presented, for simulating the generation of data driven and reverse engineering attacks on classifiers. Experimental evaluation, on 10 real world datasets and using the Google Cloud Prediction Platform, demonstrates the innate vulnerability of classifiers and the ease with which evasion can be carried out, without any explicit information about the classifier type, the training data or the application domain. The proposed framework, algorithms and empirical evaluation, serve as a white hat analysis of the vulnerabilities, and aim to foster the development of secure machine learning frameworks.