Modelling Metamorphism by Abstract Interpretation

Metamorphic malware apply semantics-preserving transformations to their own code in order to foil detection systems based on signature matching. In this paper we consider the problem of automatically extract metamorphic signatures from these malware. We introduce a semantics for self-modifying code, later called phase semantics, and prove its correctness by showing that it is an abstract interpretation of the standard trace semantics. Phase semantics precisely models the metamorphic code behavior by providing a set of traces of programs which correspond to the possible evolutions of the metamorphic code during execution. We show that metamorphic signatures can be automatically extracted by abstract interpretation of the phase semantics, and that regular metamorphism can be modelled as finite state automata abstraction of the phase semantics.

[1]  Bart Demoen,et al.  Abstract Interpretation: Towards the Global Optimization of Prolog Programs , 1987, SLP.

[2]  Helmut Veith,et al.  Using Verification Technology to Specify and Detect Malware , 2007, EUROCAST.

[3]  Nicolas Halbwachs,et al.  Automatic discovery of linear restraints among variables of a program , 1978, POPL.

[4]  Patrick Cousot,et al.  Formal language, grammar and set-constraint-based program analysis by abstract interpretation , 1995, FPCA '95.

[5]  Eric Filiol,et al.  Metamorphism, Formal Grammars and Undecidable Code Mutation , 2007 .

[6]  Arun Lakhotia,et al.  Static verification of worm and virus behavior in binary executables using model checking , 2003, IEEE Systems, Man and Cybernetics SocietyInformation Assurance Workshop, 2003..

[7]  R. M. Díaz,et al.  On first-passage problems for asymmetric one-dimensional diffusions , 2007 .

[8]  Pavel V. Zbitskiy.  Code mutation techniques by means of formal grammars and automatons , 2009, Journal in Computer Virology.

[9]  Peter Szor,et al.  HUNTING FOR METAMORPHIC , 2001 .

[10]  Somesh Jha,et al.  A semantics-based approach to malware detection , 2007, POPL '07.

[11]  Somesh Jha,et al.  Static Analysis of Executables to Detect Malicious Patterns , 2003, USENIX Security Symposium.

[12]  Patrick Cousot,et al.  Constructive design of a hierarchy of semantics of a transition system by abstract interpretation , 2002, MFPS.

[13]  Thomas W. Reps,et al.  CodeSurfer/x86-A Platform for Analyzing x86 Executables , 2005, CC.

[14]  Somesh Jha,et al.  Testing malware detectors , 2004, ISSTA '04.

[15]  Roberto Giacobazzi,et al.  Making abstract interpretations complete , 2000, JACM.

[16]  Somesh Jha,et al.  Semantics-aware malware detection , 2005, 2005 IEEE Symposium on Security and Privacy (S&P'05).

[17]  Patrick Cousot,et al.  Systematic design of program analysis frameworks , 1979, POPL.

[18]  Laurie J. Hendren,et al.  Context-sensitive interprocedural points-to analysis in the presence of function pointers , 1994, PLDI '94.

[19]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[20]  Amey Karkare,et al.  Heap reference analysis using access graphs , 2006, ACM Trans. Program. Lang. Syst..

[21]  Stefan Katzenbeisser,et al.  Detecting Malicious Code by Model Checking , 2005, DIMVA.

[22]  Zhong Shao,et al.  Certified self-modifying code , 2007, PLDI '07.

[23]  Hisao Tamaki,et al.  Program transformation through meta-shifting , 2009, New Generation Computing.

[24]  Thomas W. Reps,et al.  Analyzing Memory Accesses in x86 Executables , 2004, CC.

[25]  Peter Szor,et al.  The Art of Computer Virus Research and Defense , 2005 .