Combining Dynamic Passive Analysis and Active Fingerprinting for Effective Bot Malware Detection in Virtualized Environments

We propose a detection mechanism that takes the advantage of virtualized environment and combines both passive and active detection approaches for detecting bot malware. Our proposed passive detection agent lies in the virtual machine monitor to profile the bot behavior and check against it with other hosts. The proposed active detection agent that performs active bot fingerprinting can send specific stimulus to a host and examine if there exists expected triggered behavior. In our experiments, our system can distinguish bots and the benign process with low false alarm. The active fingerprinting technique can detect a bot even when a bot does not do its malicious jobs.

[1]  Felix C. Freiling,et al.  Toward Automated Dynamic Malware Analysis Using CWSandbox , 2007, IEEE Secur. Priv..

[2]  Heng Yin,et al.  Panorama: capturing system-wide information flow for malware detection and analysis , 2007, CCS '07.

[3]  П. Довгалюк,et al.  Два способа организации механизма полносистемного детерминированного воспроизведения в симуляторе QEMU , 2012 .

[4]  Claudia Eckert,et al.  A formal model for virtual machine introspection , 2009, VMSec '09.

[5]  Vinod Yegneswaran,et al.  BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation , 2007, USENIX Security Symposium.

[6]  Brian D. Noble,et al.  When Virtual Is Better Than Real , 2001 .

[7]  Lei Liu,et al.  BotTracer: Execution-Based Bot-Like Malware Detection , 2008, ISC.

[8]  Samuel T. King,et al.  ReVirt: enabling intrusion analysis through virtual-machine logging and replay , 2002, OPSR.

[9]  Zhenkai Liang,et al.  BitBlaze: A New Approach to Computer Security via Binary Analysis , 2008, ICISS.

[10]  Zhenkai Liang,et al.  Automatically Identifying Trigger-based Behavior in Malware , 2008, Botnet Detection.

[11]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.

[12]  Guofei Gu,et al.  BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic , 2008, NDSS.

[13]  Tal Garfinkel,et al.  A Virtual Machine Introspection Based Architecture for Intrusion Detection , 2003, NDSS.