DECANTeR: DEteCtion of Anomalous outbouNd HTTP TRaffic by Passive Application Fingerprinting

We present DECANTeR, a system to detect anomalous outbound HTTP communication, which passively extracts fingerprints for each application running on a monitored host. The goal of our system is to detect unknown malware and backdoor communication indicated by unknown fingerprints extracted from a host's network traffic. We evaluate a prototype with realistic data from an international organization and datasets composed of malicious traffic. We show that our system achieves a false positive rate of 0.9% for 441 monitored host machines, an average detection rate of 97.7%, and that it cannot be evaded by malware using simple evasion techniques such as using known browser user agent values. We compare our solution with DUMONT [24], the current state-of-the-art IDS which detects HTTP covert communication channels by focusing on benign HTTP traffic. The results show that DECANTeR outperforms DUMONT in terms of detection rate, false positive rate, and even evasion-resistance. Finally, DECANTeR detects 96.8% of information stealers in our dataset, which shows its potential to detect data exfiltration.

[1]  Yuval Elovici,et al.  CoBAn: A context based model for data leakage prevention , 2014, Inf. Sci..

[2]  Christopher Krügel,et al.  Anomaly detection of web-based attacks , 2003, CCS '03.

[3]  Michalis Faloutsos,et al.  ReSurf: Reconstructing web-surfing activity from network traffic , 2013, 2013 IFIP Networking Conference.

[4]  Kevin Borders,et al.  Web tap: detecting covert web traffic , 2004, CCS '04.

[5]  Areej Al-Bataineh,et al.  Analysis and detection of malicious data exfiltration in web traffic , 2012, 2012 7th International Conference on Malicious and Unwanted Software.

[6]  Kevin Borders,et al.  Quantifying Information Leaks in Outbound Web Traffic , 2009, 2009 30th IEEE Symposium on Security and Privacy.

[7]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[8]  Engin Kirda,et al.  Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security , 2011, Eurosys 2011.

[9]  Francesco Versaci,et al.  A Novel Method to Detect Encrypted Data Exfiltration , 2013, ICPADS 2013.

[10]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.

[11]  Dawn Xiaodong Song,et al.  Fig: Automatic Fingerprint Generation , 2007, NDSS.

[12]  Juan Caballero,et al.  FIRMA: Malware Clustering and Network Signature Generation with Mixed Network Behaviors , 2013, RAID.

[13]  Dan Schnackenberg,et al.  Statistical approaches to DDoS attack detection and response , 2003, Proceedings DARPA Information Survivability Conference and Exposition.

[14]  Apostolis Zarras,et al.  Automated generation of models for fast and precise detection of HTTP-based malware , 2014, 2014 Twelfth Annual International Conference on Privacy, Security and Trust.

[15]  Elisa Bertino,et al.  Privacy-Preserving Detection of Sensitive Data Exposure , 2015, IEEE Transactions on Information Forensics and Security.

[16]  Danfeng Yao,et al.  Data Leak Detection as a Service , 2012, SecureComm.

[17]  Vallipuram Muthukkumarasamy,et al.  Detecting Data Semantic: A Data Leakage Prevention Approach , 2015, 2015 IEEE Trustcom/BigDataSE/ISPA.

[18]  Nizar Kheir,et al.  Analyzing HTTP User Agent Anomalies for Malware Detection , 2012, DPM/SETOP.

[19]  Naren Ramakrishnan,et al.  Causality reasoning about network events for detecting stealthy malware activities , 2016, Comput. Secur..

[20]  Nick Sullivan,et al.  The Security Impact of HTTPS Interception , 2017, NDSS.

[21]  Alessandro Barenghi,et al.  ShieldFS: a self-healing, ransomware-aware filesystem , 2016, ACSAC.

[22]  Christopher Krügel,et al.  Extracting probable command and control signatures for detecting botnets , 2014, SAC.

[23]  Nick Feamster,et al.  Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces , 2010, NSDI.

[24]  Roberto Perdisci,et al.  ExecScent: Mining for New C&C Domains in Live Networks with Adaptive Control Protocol Templates , 2013, USENIX Security Symposium.

[25]  Karel Bartos,et al.  Optimized Invariant Representation of Network Traffic for Detecting Unseen Malware Variants , 2016, USENIX Security Symposium.

[26]  Rob Johnson,et al.  Text Classification for Data Loss Prevention , 2011, PETS.

[27]  Wojciech Mazurczyk,et al.  Trends in steganography , 2014, Commun. ACM.

[28]  Sherali Zeadally,et al.  An Empirical Study of HTTP-based Financial Botnets , 2016, IEEE Transactions on Dependable and Secure Computing.

[29]  Tao Zhang,et al.  A Novel Method to Detect Encrypted Data Exfiltration , 2014, 2014 Second International Conference on Advanced Cloud and Big Data.

[30]  Felix C. Freiling,et al.  Sandnet: network traffic analysis of malicious software , 2011, BADGERS '11.

[31]  Jing Zhang,et al.  Fast Detection of Transformed Data Leaks , 2016, IEEE Transactions on Information Forensics and Security.

[32]  Guofei Gu,et al.  BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic , 2008, NDSS.

[33]  Leyla Bilge,et al.  Disclosure: detecting botnet command and control servers through large-scale NetFlow analysis , 2012, ACSAC '12.

[34]  Kang Li,et al.  ClickMiner: Towards Forensic Reconstruction of User-Browser Interactions from Network Traces , 2014, CCS.

[35]  Roberto Perdisci,et al.  Scalable fine-grained behavioral clustering of HTTP-based malware , 2013, Comput. Networks.

[36]  Konrad Rieck,et al.  Adaptive Detection of Covert Communication in HTTP Requests , 2011, 2011 Seventh European Conference on Computer Network Defense.