An Approach towards Disassembly of Malicious Binary Executables

Analysis of a binary executable starts with disassembly. Current methods start disassembling a program from an entry point, and continue disassembling either by a simple linear scan or by following the execution paths using static analysis. A hostile programmer can easily fool such methods for disassembly and cause confusion between code and data. A method for disassembling that does not leave any code behind is proposed. The method does not depend on an entry point; instead, it treats every byte as a potential entry point. It partitions the code into segments where a segment is a largest sequence of instructions terminated by an instruction, such as JUMP, that explicitly modifies the program counter. A segment is invalid if its execution will always lead into data. The method filters out invalid segments. It finally connects all the code segments to give a “precise disassembly.” Results from using the method on ten programs are presented. We have implemented Binary Disassembler for Malicious Executables (BDME). BDME is a disassembler that is written specifically to target malicious code and obfuscated binaries. For our test data, we are able to reduce up to 87% of false positives while getting no false negatives. BDME is able to find all the potential instructions in the code, thus making it very hard to hide malicious code from analysis through disassembly. The average reduction of 46% of false positives further helps the disassembler in keeping the disassembly precise.

[1]  Arun Lakhotia,et al.  CHALLENGES IN GETTING ‘FORMAL’ WITH VIRUSES , 2003 .

[2]  Thomas W. Reps,et al.  Analyzing Memory Accesses in x86 Executables , 2004, CC.

[3]  Christopher Krügel,et al.  Static Disassembly of Obfuscated Binaries , 2004, USENIX Security Symposium.

[4]  Saumya K. Debray,et al.  Obfuscation of executable code to improve resistance to static disassembly , 2003, CCS '03.

[5]  Tzi-cker Chiueh,et al.  A Binary Rewriting Defense Against Stack based Buffer Overflow Attacks , 2003, USENIX Annual Technical Conference, General Track.

[6]  Gregory R. Andrews,et al.  Disassembly of executable code revisited , 2002, Ninth Working Conference on Reverse Engineering, 2002. Proceedings..

[7]  Evgeniy Gabrilovich,et al.  JNI-C++ integration made easy , 2001 .

[8]  Richard L. Sites,et al.  Binary translation , 1993, CACM.

[9]  Mike Van,et al.  UQBT: Adaptable Binary Translation at Low Cost , 2000 .

[10]  S. Fouzi Husaini Using the Java Native Interface , 1997, CROS.

[11]  Koen De Bosschere,et al.  alto: a link-time optimizer for the Compaq Alpha , 2001, Softw. Pract. Exp..

[12]  R. Nigel Horspool,et al.  An Approach to the Problem of Detranslation of Computer Programs , 1980, Comput. J..

[13]  Linda M. Wills,et al.  An experimentation framework for evaluating disassembly and decompilation tools for C++ and java , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..