Bayesian Active Malware Analysis

We propose a novel technique for Active Malware Analysis (AMA) formalized as a Bayesian game between an analyzer agent and a malware agent, focusing on the decision making strategy for the analyzer. In our model, the analyzer performs an action on the system to trigger the malware into showing a malicious behavior, i.e., by activating its payload. The formalization is built upon the link between malware families and the notion of types in Bayesian games. A key point is the design of the utility function, which reflects the amount of uncertainty on the type of the adversary after the execution of an analyzer action. This allows us to devise an algorithm to play the game with the aim of minimizing the entropy of the analyzer’s belief at every stage of the game in a myopic fashion. Empirical evaluation indicates that our approach results in a significant improvement both in terms of learning speed and classification score when compared to other state-of-the-art AMA techniques. ACM Reference Format: Riccardo Sartea, Georgios Chalkiadakis, Alessandro Farinelli, and Matteo Murari. 2020. Bayesian Active Malware Analysis. In Proc. of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020), Auckland, New Zealand, May 9–13, 2020, IFAAMAS, 9 pages.

[1]  Alessandro Farinelli,et al.  SECUR-AMA: Active Malware Analysis Based on Monte Carlo Tree Search for Android Systems , 2020, Eng. Appl. Artif. Intell..

[2]  A. Azzouz.  2011 , 2020, City.

[3]  G. Tian,et al.  Dirichlet and Related Distributions: Theory, Methods and Applications , 2011 .

[4]  Craig Boutilier,et al.  Coalitional Bargaining with Agent Type Uncertainty , 2007, IJCAI.

[5]  Debin Gao,et al.  Active malware analysis using stochastic games , 2012, AAMAS.

[6]  Gianluca Stringhini,et al.  MaMaDroid: Detecting Android Malware by Building Markov Chains of Behavioral Models (Extended Version) , 2016, NDSS 2017.

[7]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[8]  Vincent Conitzer,et al.  Signaling in Bayesian Stackelberg Games , 2016, AAMAS.

[9]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[10]  Sankardas Roy,et al.  Deep Ground Truth Analysis of Current Android Malware , 2017, DIMVA.

[11]  Wiem Tounsi,et al.  A survey on technical threat intelligence in the age of sophisticated cyber attacks , 2018, Comput. Secur..

[12]  Piotr Garbaczewski,et al.  Differential Entropy and Dynamics of Uncertainty , 2004 .

[13]  Alessandro Farinelli,et al.  Agent Behavioral Analysis Based on Absorbing Markov Chains , 2019, AAMAS.

[14]  Kazimierz Sobczyk,et al.  Information dynamics : Premises, challenges and results , 2001 .

[15]  Sarit Kraus,et al.  Bayesian stackelberg games and their application for security at Los Angeles international airport , 2008, SECO.

[16]  Niki Pissinou,et al.  Modeling cooperative, selfish and malicious behaviors for Trajectory Privacy Preservation using Bayesian game theory , 2013, 38th Annual IEEE Conference on Local Computer Networks.

[17]  Christopher Krügel,et al.  Exploring Multiple Execution Paths for Malware Analysis , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[18]  Craig Boutilier,et al.  Coalition formation under uncertainty: bargaining equilibria and the Bayesian core stability concept , 2007, AAMAS '07.

[19]  John C. Harsanyi,et al.  Games with Incomplete Information Played by "Bayesian" Players, I-III: Part I. The Basic Model& , 2004, Manag. Sci..

[20]  J. Harsanyi.  Games with Incomplete Information Played by 'Bayesian' Players, Part III. The Basic Probability Distribution of the Game , 1968 .

[21]  N. Ebrahimi,et al.  Information measures of Dirichlet distribution with applications , 2011 .

[22]  Jason Upchurch,et al.  Malware provenance: code reuse detection in malicious software at scale , 2016, 2016 11th International Conference on Malicious and Unwanted Software (MALWARE).

[23]  Juliane Hahn,et al.  Security And Game Theory Algorithms Deployed Systems Lessons Learned , 2016 .

[24]  Mauro Conti,et al.  Detecting Targeted Smartphone Malware with Behavior-Triggering Stochastic Models , 2014, ESORICS.

[25]  Mangal Sain,et al.  Survey on malware evasion techniques: State of the art and challenges , 2012, 2012 14th International Conference on Advanced Communication Technology (ICACT).

[26]  Heng Yin,et al.  DroidAPIMiner: Mining API-Level Features for Robust Malware Detection in Android , 2013, SecureComm.

[27]  Andrew Walenstein,et al.  The Software Similarity Problem in Malware Analysis , 2006, Duplication, Redundancy, and Similarity in Software.

[28]  Andrew Walenstein,et al.  Exploiting Similarity Between Variants to Defeat Malware “ Vilo ” Method for Comparing and Searching Binary Programs , 2007 .

[29]  Alessandro Farinelli,et al.  A Monte Carlo Tree Search approach to Active Malware Analysis , 2017, IJCAI.

[30]  International Foundation for Autonomous Agents and MultiAgent Systems ( IFAAMAS ) , 2007 .

[31]  David Camacho,et al.  CANDYMAN: Classifying Android malware families by modelling dynamic traces with Markov chains , 2018, Eng. Appl. Artif. Intell..