Robustness as Inherent Property of Datapoints

Characterizing how effective a machine learning algorithm is while being trained and tested on slightly different data is a widespread matter. The property of models which perform well under this general framework is commonly known as robustness. We propose a class of model-agnostic empirical robustness measures for image classification tasks. To any random image perturbation scheme, we attach a robustness measure that empirically checks how easy it is to perturb a labelled image and cause the model to misclassify it. We also introduce a methodology for training more robust models using the information gained about the empirical robustness measure of the training set. We only keep a fraction of datapoints that are robust according to our robustness measure and retrain the model using it. Our methodology validates that the robustness of the model increases by measuring its empirical robustness on test data.