Implementation of a RAT Detection System using HTTP-Based Communication Behavior

In recent APT campaigns it is hard to detect RAT activity, unless we know the C&C server address or the distinctive communication pattern. As a countermeasure, we proposed how to detect RAT activity without pattern matching, which uses the http-based behavior such as the sizes of the object returned to the client or the intervals of the logged time. Our method uses statistical classifiers based on support vector machine and random forests. Our method can detect unknown RAT activity in proxy server logs. This paper proposes the operation concept using the blacklist of C&C servers, and implements the RAT detection system using http-based communication behavior. Moreover, the experimental result on a real network shows the system can run in real time.

[1]  Vinod Yegneswaran,et al.  BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation , 2007, USENIX Security Symposium.

[2]  Guofei Gu,et al.  BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic , 2008, NDSS.

[3]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.

[4]  Nick Feamster,et al.  Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces , 2010, NSDI.

[5]  Nick Feamster,et al.  Building a Dynamic Reputation System for DNS , 2010, USENIX Security Symposium.

[6]  Michael K. Reiter,et al.  Are Your Hosts Trading or Plotting? Telling P2P File-Sharing and Bots Apart , 2010, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[7]  Xiapu Luo,et al.  Detecting stealthy P2P botnets using statistical traffic fingerprints , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[8]  Christopher Krügel,et al.  JACKSTRAWS: Picking Command and Control Connections from Bot Traffic , 2011, USENIX Security Symposium.

[9]  Wenke Lee,et al.  Detecting Malware Domains at the Upper DNS Hierarchy , 2011, USENIX Security Symposium.

[10]  Leyla Bilge,et al.  Disclosure: detecting botnet command and control servers through large-scale NetFlow analysis , 2012, ACSAC '12.

[11]  Christopher Krügel,et al.  BotFinder: finding bots in network traffic without deep packet inspection , 2012, CoNEXT '12.

[12]  Roberto Perdisci,et al.  From Throw-Away Traffic to Bots: Detecting the Rise of DGA-Based Malware , 2012, USENIX Security Symposium.

[13]  Roberto Perdisci,et al.  ExecScent: Mining for New C&C Domains in Live Networks with Adaptive Control Protocol Templates , 2013, USENIX Security Symposium.

[14]  Juan Caballero,et al.  FIRMA: Malware Clustering and Network Signature Generation with Mixed Network Behaviors , 2013, RAID.

[15]  Yasuhiro Nakamura,et al.  A Supplementary Method for Malicious Detection Based on HTTP-Activity Similarity Features , 2014 .

[16]  Yashar Ganjali,et al.  Beehive: Towards a Simple Abstraction for Scalable Software-Defined Networking , 2014, HotNets.

[17]  Toru Sato,et al.  The method of detecting malware-infected hosts analyzing firewall and proxy logs , 2015, 2015 10th Asia-Pacific Symposium on Information and Telecommunication Technologies (APSITT).

[18]  Zhou Li,et al.  Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data , 2014, 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[19]  Babak Rahbarinia,et al.  Segugio: Efficient Behavior-Based Tracking of Malware-Control Domains in Large ISP Networks , 2015, 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.