Poking the bear: lessons learned from probing three Android malware datasets

To counter the continuous threat posed by Android malware, we attempted to devise a novel method based on active learning. Nonetheless, evaluating our active learning based method on three different Android malware datasets resulted in performance discrepancies. In an attempt to explain such inconsistencies, we postulated research questions and designed corresponding experiments to answer them. The results of our experiments unveiled the reasons behind the struggles of our method and, more importantly, revealed some limitations with the current Android malware detection methods that, we fear, can be leveraged by malware authors to evade detection. In this paper, we share with the research community our research questions, experiments, and findings to instigate researchers to devise methods to tackle such limitations.