Unconditional self-modifying code elimination with dynamic compiler optimizations

This paper deals with the issue of self-modifying code and packed programs, a long-standing problem commonly addressed by emulation techniques and memory dumps. We propose an original semantics-based approach to simplify dynamic code analysis, by using compiler optimization techniques to get rid of code-generating instructions. For this, we use classic slicing techniques to identify code dependencies. As it is semantics-based, our approach allows us to rely on strongly established formal methods and is a promising approach for handling packed programs.

[1]  Heng Yin,et al.  Renovo: a hidden code extractor for packed executables , 2007, WORM '07.

[2]  Wenke Lee,et al.  PolyUnpack: Automating the Hidden-Code Extraction of Unpack-Executing Malware , 2006, 2006 22nd Annual Computer Security Applications Conference (ACSAC'06).

[3]  Zhenkai Liang,et al.  BitBlaze: A New Approach to Computer Security via Binary Analysis , 2008, ICISS.

[4]  Somesh Jha,et al.  OmniUnpack: Fast, Generic, and Safe Unpacking of Malware , 2007, Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007).

[5]  Yang Xiang,et al.  A Fast Flowgraph Based Classification System for Packed and Polymorphic Malware on the Endhost , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[6]  M. Eliantonio,et al.  Private Parties and the Annulment Procedure: Can the Gap in the European System of Judicial Protection Be Closed? , 2010 .

[7]  Derek Bruening,et al.  Efficient, transparent, and comprehensive runtime code manipulation , 2004 .

[8]  Frank Tip,et al.  A survey of program slicing techniques , 1994, J. Program. Lang..

[9]  Sébastien Josse.  Secure and advanced unpacking using computer emulation , 2007, Journal in Computer Virology.

[10]  Tzi-cker Chiueh,et al.  A Study of the Packer Problem and Its Solutions , 2008, RAID.

[11]  Christopher Krügel,et al.  Cross Site Scripting Prevention with Dynamic Data Tainting and Static Analysis , 2007, NDSS.

[12]  Alessandro Orso,et al.  Dytan: a generic dynamic taint analysis framework , 2007, ISSTA '07.

[13]  Heng Yin,et al.  Panorama: capturing system-wide information flow for malware detection and analysis , 2007, CCS '07.

[14]  Peter Szor,et al.  The Art of Computer Virus Research and Defense , 2005 .

[15]  Peter Sestoft,et al.  Partial evaluation and automatic program generation , 1993, Prentice Hall international series in computer science.

[16]  Magnus O. Myreen.  Verified just-in-time compiler on x86 , 2010, POPL '10.

[17]  Armin Rigo,et al.  Representation-based just-in-time specialization and the psyco prototype for python , 2004, PEPM '04.

[18]  Harish Patil,et al.  Pin: building customized program analysis tools with dynamic instrumentation , 2005, PLDI '05.

[19]  Christopher Krügel,et al.  Exploring Multiple Execution Paths for Malware Analysis , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[20]  Wenke Lee,et al.  Ether: malware analysis via hardware virtualization extensions , 2008, CCS.

[21]  James Newsome,et al.  Dynamic Taint Analysis for Automatic Detection, Analysis, and SignatureGeneration of Exploits on Commodity Software , 2005, NDSS.

[22]  Akihiko Takano.  Y.Futamura : Partial Evaluation of Computation Process : An Approach to a Compiler-Compiler , 2002 .

[23]  Yoshihiko Futamura,et al.  Partial Evaluation of Computation Process--An Approach to a Compiler-Compiler , 1999, High. Order Symb. Comput..