Detecting encrypted metamorphic viruses by hidden Markov Models

Virus writers make their viruses undetectable by using obfuscation methods, which ends in metamorphic viruses. We propose a method named detection circle which is based on the hidden Markov Model theory. We have used three elements to characterize a family of viruses: string occurrence probability, specifically-located character occurrence probability, and the amount of virus similarities. For the evaluation, we have created viruses and tested them by our method and four anti-virus software packages. The experimental results show that our detection rate was much higher in the first stage without obfuscation. Then we have encrypted the detected viruses and tested the proposed algorithm again. At this stage none of the four anti-viruses software packages detected viruses while our method found 70% of them.

[1]  Mark Stamp,et al.  Profile hidden Markov models and metamorphic virus detection , 2009, Journal in Computer Virology.

[2]  Wanlei Zhou,et al.  Malwise—An Effective and Efficient Classification System for Packed and Polymorphic Malware , 2013, IEEE Transactions on Computers.

[3]  Jiankun Hu,et al.  A Semantic Approach to Host-Based Intrusion Detection Systems Using Contiguousand Discontiguous System Call Patterns , 2014, IEEE Transactions on Computers.

[4]  Wanlei Zhou,et al.  Locating Defense Positions for Thwarting the Propagation of Topological Worms , 2012, IEEE Communications Letters.

[5]  Mark Stamp,et al.  Practical Detection of Metamorphic Computer Viruses , 2008 .

[6]  Jiankun Hu,et al.  Modeling Oscillation Behavior of Network Traffic by Nested Hidden Markov Model with Variable State-Duration , 2013, IEEE Transactions on Parallel and Distributed Systems.

[7]  Yi Xie,et al.  A Forward-Backward Algorithm for Nested Hidden semi-Markov Model and Application to Network Traffic , 2013, Comput. J..

[8]  Sami Khuri,et al.  ANALYSIS AND DETECTION OF METAMORPHIC COMPUTER VIRUSES , 2006 .

[10]  Jun Zhang,et al.  Modeling Propagation Dynamics of Social Network Worms , 2013, IEEE Transactions on Parallel and Distributed Systems.

[11]  Mark Stamp,et al.  Detecting Undetectable Metamorphic Viruses , 2011 .

[12]  T. Vinay Kumar M. Tech.  Malwise-An Effective and Efficient Classification System for Packed and Polymorphic Malware , 2014 .

[13]  Grant Malcolm,et al.  Detection of metamorphic computer viruses using algebraic specification , 2006, Journal in Computer Virology.

[14]  W. Marsden.  I and J , 2012 .

[15]  Wanlei Zhou,et al.  Eliminating Errors in Worm Propagation Models , 2011, IEEE Communications Letters.

[16]  Jiankun Hu,et al.  A program-based anomaly intrusion detection scheme using multiple detection engines and fuzzy inference , 2009, J. Netw. Comput. Appl..