Using Entropy and Mutual Information to Extract Threat Actions from Cyber Threat Intelligence

With the rapid growth of the cyber attacks, cyber threat intelligence (CTI) sharing becomes essential for providing advance threat notice and enabling timely response to cyber attacks. Our goal in this paper is to develop an approach to extract low-level cyber threat actions from publicly available CTI sources in an automated manner to enable timely defense decision making. Specifically, we innovatively and successfully used the metrics of entropy and mutual information from Information Theory to analyze the text in the cybersecurity domain. Combined with some basic NLP techniques, our framework, called ActionMiner has achieved higher precision and recall than the state-of-the-art Stanford typed dependency parser, which usually works well in general English but not cybersecurity texts.

[1]  Timothy W. Finin,et al.  Extracting Information about Security Vulnerabilities from Web Text , 2011, 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[2]  Tudor Dumitras,et al.  FeatureSmith: Automatically Engineering Features for Malware Detection by Mining the Security Literature , 2016, CCS.

[3]  Tudor Dumitras,et al.  Vulnerability Disclosure in the Age of Social Media: Exploiting Twitter for Predicting Real-World Exploits , 2015, USENIX Security Symposium.

[4]  Tudor Dumitras,et al.  ChainSmith: Automatically Learning the Semantics of Malicious Campaigns by Mining Threat Intelligence Reports , 2018, 2018 IEEE European Symposium on Security and Privacy (EuroS&P).

[5]  Christopher D. Manning,et al.  Stanford typed dependencies manual , 2010 .

[6]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[7]  Zhou Li,et al.  Acing the IOC Game: Toward Automatic Discovery and Analysis of Open-Source Cyber Threat Intelligence , 2016, CCS.

[8]  Timothy W. Finin,et al.  Extracting Cybersecurity Related Linked Data from Text , 2013, 2013 IEEE Seventh International Conference on Semantic Computing.

[9]  Gang Wang,et al.  Crowdsourcing Cybersecurity: Cyber Attack Detection using Social Media , 2017, CIKM.

[10]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[11]  Flora S. Tsai,et al.  Detecting Cyber Security Threats in Weblogs Using Probabilistic Models , 2007, PAISI.

[12]  Ehab Al-Shaer,et al.  TTPDrill: Automatic and Accurate Extraction of Threat Actions from Unstructured Text of CTI Sources , 2017, ACSAC.