Detecting Self-mutating Malware Using Control-Flow Graph Matching

Next generation malware will by be characterized by the intense use of polymorphic and metamorphic techniques aimed at circumventing the current malware detectors, based on pattern matching. In order to deal with this new kind of threat, novel techniques have to be devised for the realization of malware detectors. Recent papers started to address such an issue and this paper represents a further contribution in such a field. More precisely in this paper we propose a strategy for the detection of metamorphic malicious code inside a program P based on the comparison of the control flow graphs of P against the set of control flow graphs of known malware. We also provide experimental data supporting the validity of our strategy

[1]  Arun Lakhotia,et al.  A method for detecting obfuscated calls in malicious binaries , 2005, IEEE Transactions on Software Engineering.

[2]  Christopher Krügel,et al.  Static Disassembly of Obfuscated Binaries , 2004, USENIX Security Symposium.

[3]  Somesh Jha,et al.  Static Analysis of Executables to Detect Malicious Patterns , 2003, USENIX Security Symposium.

[4]  Christopher Krügel,et al.  Polymorphic Worm Detection Using Structural Information of Executables , 2005, RAID.

[5]  Somesh Jha,et al.  Semantics-aware malware detection , 2005, 2005 IEEE Symposium on Security and Privacy (S&P'05).

[6]  Somesh Jha,et al.  Testing malware detectors , 2004, ISSTA '04.

[7]  Steve R. White,et al.  An Undetectable Computer Virus , 2000 .

[8]  Saumya K. Debray,et al.  Obfuscation of executable code to improve resistance to static disassembly , 2003, CCS '03.

[9]  James Newsome,et al.  Polygraph: automatically generating signatures for polymorphic worms , 2005, 2005 IEEE Symposium on Security and Privacy (S&P'05).

[10]  Steven S. Muchnick,et al.  Advanced Compiler Design and Implementation , 1997 .

[11]  Peter Szor,et al.  HUNTING FOR METAMORPHIC , 2001 .

[12]  Christian S. Collberg,et al.  A Taxonomy of Obfuscating Transformations , 1997 .

[13]  Bjorn De Sutter,et al.  Compiler techniques for code compaction , 2000, TOPL.

[14]  Frederick B. Cohen,et al.  A short course on computer viruses (2nd ed.) , 1994 .

[15]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[16]  Mattia Monga,et al.  Using Code Normalization for Fighting Self-Mutating Malware , 2006, ISSSE.