A General Framework of Trojan Communication Detection Based on Network Traces

Because of the widespread Trojan, Internet users become more and more vulnerable to the threat of information leakage. Traditional techniques of Trojan detection were classified into two main categories: host-based and network-based. Unfortunately, existing techniques are insufficient and limited, because of the following reasons: (1)only uncover the known Trojan while inefficiently detecting novel samples, (2) should be adjusted in a timely fashion even a trivial change is applied, and (3)become computationally more expensive. In our work, we focus on a network behavior based method to address the limitations of previous network-based approaches. We analyze the profile of network behavior at two levels: (i)flow-level, (ii)IP-level. Our approach present two main advantages: (1)capture more detailed information to describe the network behavior profile, (2)consume lower computational overhead. We proposed a system, Manto, which detects Trojan communication with high accuracy using clustering technique. We implement Manto on real-world traces. The evaluation results exhibit that Manto is suitable for detecting Trojan communication amongst the vast amount of network traffic, with over 91% accuracy and less than 3.2% false positive ratio. We confidently regard our approach as a complementary way to the existing network-based techniques for we could address their main shortcomings.

[1]  Christopher Krügel,et al.  Behavior-based Spyware Detection , 2006, USENIX Security Symposium.

[2]  Felix C. Freiling,et al.  Sandnet: network traffic analysis of malicious software , 2011, BADGERS '11.

[3]  Thorsten Holz,et al.  Rishi: Identify Bot Contaminated Hosts by IRC Nickname Evaluation , 2007, HotBots.

[4]  Heng Yin,et al.  Panorama: capturing system-wide information flow for malware detection and analysis , 2007, CCS '07.

[5]  Engin Kirda,et al.  A View on Current Malware Behaviors , 2009, LEET.

[6]  Kevin Borders,et al.  Quantifying Information Leaks in Outbound Web Traffic , 2009, 2009 30th IEEE Symposium on Security and Privacy.

[7]  Yi-Min Wang,et al.  Detecting stealth software with Strider GhostBuster , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[8]  Marcus A. Maloof,et al.  Learning to Detect and Classify Malicious Executables in the Wild , 2006, J. Mach. Learn. Res..

[9]  Martin Boldt,et al.  Analysing Privacy-Invasive Software Using Computer Forensic Methods , 2006 .

[10]  Kevin Borders,et al.  Web tap: detecting covert web traffic , 2004, CCS '04.

[11]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[12]  U. Bayer,et al.  TTAnalyze: A Tool for Analyzing Malware , 2006 .

[13]  Zhuoqing Morley Mao,et al.  Automated Classification and Analysis of Internet Malware , 2007, RAID.

[14]  Engin Kirda,et al.  Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security , 2011, Eurosys 2011.

[15]  Debin Gao,et al.  BinHunt: Automatically Finding Semantic Differences in Binary Programs , 2008, ICICS.

[16]  Carey L. Williamson,et al.  Offline/realtime traffic classification using semi-supervised learning , 2007, Perform. Evaluation.

[17]  Grenville J. Armitage,et al.  A survey of techniques for internet traffic classification using machine learning , 2008, IEEE Communications Surveys & Tutorials.

[18]  Zheng Wang,et al.  BMAT - A Binary Matching Tool for Stale Profile Propagation , 2000, J. Instr. Level Parallelism.

[19]  Michael K. Reiter,et al.  Traffic Aggregation for Malware Detection , 2008, DIMVA.

[20]  Taskin Koçak,et al.  Low-power bloom filter architecture for deep packet inspection , 2006, IEEE Communications Letters.

[21]  Anirban Mahanti,et al.  Traffic classification using clustering algorithms , 2006, MineNet '06.

[22]  John C. Mitchell,et al.  Characterizing Bots' Remote Control Behavior , 2007, DIMVA.

[23]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[24]  Nick Feamster,et al.  Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces , 2010, NSDI.

[25]  T. Dullien,et al.  Graph-based comparison of Executable Objects ( English Version ) , 2005 .