MADE: Security Analytics for Enterprise Threat Detection

Enterprises are targeted by various malware activities at a staggering rate. To counteract the increased sophistication of cyber attacks, most enterprises deploy within their perimeter a number of security technologies, including firewalls, antivirus software, and web proxies, as well as specialized teams of security analysts forming Security Operations Centers (SOCs). In this paper we address the problem of detecting malicious activity in enterprise networks and prioritizing the detected activities according to their risk. We design a system called MADE using machine learning applied to data extracted from security logs. MADE leverages an extensive set of features for enterprise malicious communication and uses supervised learning in a novel way for prioritization, rather than detection, of enterprise malicious activities. MADE has been deployed in a large enterprise and used by SOC analysts. Over one month, MADE successfully prioritizes the most risky domains contacted by enterprise hosts, achieving a precision of 97% in 100 detected domains, at a very small false positive rate. We also demonstrate MADE's ability to identify new malicious activities (18 out of 100) overlooked by state-of-the-art security technologies.

[1]  Damon McCoy,et al.  To Catch a Ratter: Monitoring the Behavior of Amateur DarkComet RAT Operators in the Wild , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[2]  Christopher Krügel,et al.  Anomaly detection of web-based attacks , 2003, CCS '03.

[3]  Lujo Bauer,et al.  Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition , 2016, CCS.

[4]  Kevin M. Carter,et al.  Probabilistic Threat Propagation for Network Security , 2014, IEEE Transactions on Information Forensics and Security.

[5]  Kevin C. Almeroth,et al.  FIRE: FInding Rogue nEtworks , 2009, 2009 Annual Computer Security Applications Conference.

[6]  Jiyong Jang,et al.  BAYWATCH: Robust Beaconing Detection to Identify Infected Hosts in Large-Scale Enterprise Networks , 2016, 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[7]  Christopher Krügel,et al.  BotFinder: finding bots in network traffic without deep packet inspection , 2012, CoNEXT '12.

[8]  Steven C. H. Hoi,et al.  Cost-sensitive online active learning with application to malicious URL detection , 2013, KDD.

[9]  Christopher Krügel,et al.  Nazca: Detecting Malware Distribution in Large-Scale Networks , 2014, NDSS.

[10]  Roberto Perdisci,et al.  WebWitness: Investigating, Categorizing, and Mitigating Malware Download Paths , 2015, USENIX Security Symposium.

[11]  Nick Feamster,et al.  Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces , 2010, NSDI.

[12]  Lawrence K. Saul,et al.  From .academy to .zone: An Analysis of the New TLD Land Rush , 2015, Internet Measurement Conference.

[13]  Nicolas Christin,et al.  Automatically Detecting Vulnerable Websites Before They Turn Malicious , 2014, USENIX Security Symposium.

[14]  Nick Feamster,et al.  Building a Dynamic Reputation System for DNS , 2010, USENIX Security Symposium.

[15]  Vern Paxson,et al.  On the Potential of Proactive Domain Blacklisting , 2010, LEET.

[16]  Fang Yu,et al.  Finding the Linchpins of the Dark Web: a Study on Topologically Dedicated Hosts on Malicious Web Infrastructures , 2013, 2013 IEEE Symposium on Security and Privacy.

[17]  Leyla Bilge,et al.  Measuring PUP Prevalence and PUP Distribution through Pay-Per-Install Services , 2016, USENIX Security Symposium.

[18]  Sandeep Yadav,et al.  Detecting Malicious Domains via Graph Inference , 2014, AISec '14.

[19]  Jianping Wu,et al.  When HTTPS Meets CDN: A Case of Authentication in Delegated Service , 2014, 2014 IEEE Symposium on Security and Privacy.

[20]  Leyla Bilge,et al.  Disclosure: detecting botnet command and control servers through large-scale NetFlow analysis , 2012, ACSAC '12.

[21]  Karel Bartos,et al.  Optimized Invariant Representation of Network Traffic for Detecting Unseen Malware Variants , 2016, USENIX Security Symposium.

[22]  Nick Feamster,et al.  PREDATOR: Proactive Recognition and Elimination of Domain Abuse at Time-Of-Registration , 2016, CCS.

[23]  Gianluca Stringhini,et al.  The Dark Alleys of Madison Avenue: Understanding Malicious Advertisements , 2014, Internet Measurement Conference.

[24]  Niels Provos,et al.  CAMP: Content-Agnostic Malware Protection , 2013, NDSS.

[25]  Lawrence K. Saul,et al.  Beyond blacklists: learning to detect malicious web sites from suspicious URLs , 2009, KDD.

[26]  Leyla Bilge,et al.  EXPOSURE: Finding Malicious Domains Using Passive DNS Analysis , 2011, NDSS.

[27]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[28]  Gianluca Stringhini,et al.  Shady paths: leveraging surfing crowds to detect malicious web pages , 2013, CCS.

[29]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[30]  Emi Kalita WannaCry Ransomware Attack: Protect yourself from WannaCry Ransomware: Cyber Risk and Cyber War , 2017 .

[31]  Fabrício Benevenuto,et al.  Phi.sh/$oCiaL: the phishing landscape through short URLs , 2011, CEAS '11.

[32]  Wenke Lee,et al.  Detecting Malware Domains at the Upper DNS Hierarchy , 2011, USENIX Security Symposium.

[33]  Roberto Perdisci,et al.  From Throw-Away Traffic to Bots: Detecting the Rise of DGA-Based Malware , 2012, USENIX Security Symposium.

[34]  William K. Robertson,et al.  Beehive: large-scale log analysis for detecting suspicious activity in enterprise networks , 2013, ACSAC.

[35]  Roberto Perdisci,et al.  ExecScent: Mining for New C&C Domains in Live Networks with Adaptive Control Protocol Templates , 2013, USENIX Security Symposium.

[36]  Patrick D. McDaniel,et al.  Adversarial Examples for Malware Detection , 2017, ESORICS.

[37]  Babak Rahbarinia,et al.  Segugio: Efficient Behavior-Based Tracking of Malware-Control Domains in Large ISP Networks , 2015, 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[38]  Felix C. Freiling,et al.  Measuring and Detecting Fast-Flux Service Networks , 2008, NDSS.

[39]  Zhou Li,et al.  Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data , 2014, 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[40]  Vlad-Mihai Cotenescu SIEM (SECURITY INFORMATION AND EVENT MANAGEMENT SOLUTIONS) IMPLEMENTATIONS IN PRIVATE OR PUBLIC CLOUDS , 2017 .

[41]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.