Environment-Reactive Malware Behavior: Detection and Categorization

Present malicious threats have been consolidated in past few years by incorporating diverse stealthy techniques. Detecting these malwares on the basis of their dynamic behavior has become a potential approach as it suppresses the shortcomings of static approaches raised due to the obfuscated malware binaries. Additionally, existing behavior based malware detection approaches are resilient to zero–day malware attacks. These approaches rely on isolated analysis environment to monitor and capture the run–time malware behavior. Malware bundled with environment–aware payload may degrade detection accuracy of such approaches. These malicious programs detect the presence of execution environment and thus inspite of having their malicious payload they mimic a benign behavior to avoid detection. In this paper, we have presented an approach using system–calls to identify a malware on the basis of their malignant and environment–reactive behavior. The proposed approach offers an automated screening mechanism to segregate malware samples on the basis of aforementioned behaviors. We have built a decision model which is based on multi–layer perceptron learning with back propagation algorithm. Our proposed model decides the candidacy of a sample to be put into one of the four classes (clean, malignant, guest–crashing and infinite–running). Clean behavior denotes benign sample and rest of the behaviors denote the presence of malware sample. The proposed technique has been evaluated with known and unknown instances of real malware and benign programs.

[1]  Wenke Lee,et al.  Ether: malware analysis via hardware virtualization extensions , 2008, CCS.

[2]  Carsten Willems,et al.  Learning and Classification of Malware Behavior , 2008, DIMVA.

[3]  Christopher Krügel,et al.  Effective and Efficient Malware Detection at the End Host , 2009, USENIX Security Symposium.

[4]  Mark Stamp,et al.  Deriving common malware behavior through graph clustering , 2013, Comput. Secur..

[5]  Zhenkai Liang,et al.  Automatically Identifying Trigger-based Behavior in Malware , 2008, Botnet Detection.

[6]  Levente Buttyán,et al.  nEther: in-guest detection of out-of-the-guest malware analyzers , 2011, EUROSEC '11.

[7]  Curtis B. Storlie,et al.  Graph-based malware detection using dynamic analysis , 2011, Journal in Computer Virology.

[8]  Christopher Krügel,et al.  JACKSTRAWS: Picking Command and Control Connections from Bot Traffic , 2011, USENIX Security Symposium.

[9]  Christopher Krügel,et al.  Efficient Detection of Split Personalities in Malware , 2010, NDSS.

[10]  Abhinav Srivastava,et al.  System Call API Obfuscation (Extended Abstract) , 2008, RAID.

[11]  Xu Chen,et al.  Towards an understanding of anti-virtualization and anti-debugging behavior in modern malware , 2008, 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN).

[12]  Lorie M. Liebrock,et al.  Improving antivirus accuracy with hypervisor assisted analysis , 2010, Journal in Computer Virology.

[13]  Yi Lu Murphey,et al.  Multi-class pattern classification using neural networks , 2007, Pattern Recognit..

[14]  Martina Lindorfer,et al.  Detecting Environment-Sensitive Malware , 2011, RAID.

[15]  Brent Waters,et al.  Analysis-Resistant Malware , 2008, NDSS.

[16]  Chi-Sung Laih,et al.  Malware Virtualization-Resistant Behavior Detection , 2011, 2011 IEEE 17th International Conference on Parallel and Distributed Systems.

[17]  Min Gyung Kang,et al.  Emulating emulation-resistant malware , 2009, VMSec '09.

[18]  Christopher Krügel,et al.  Exploring Multiple Execution Paths for Malware Analysis , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[19]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.