A New Detection Method for Stack Overflow Vulnerability Based on Component Binary Code for Third-Party Component

Security testing of Component Object Model (COM) is an active area of research in the software engineering community. This is partly due to the increase in security related issues (referred to as vulnerabilities) reported by users. Although many papers have been published on Component Object Model, very little attention has been paid to the detection of stack overflow vulnerability in software component. This paper presents a method to detect stack overflow vulnerability of binary code of a component. We first convert the buffer overflow problem as an integer constraints problem. We then scan the different buffers according to the types of risk function parameters to establish a function library for all the risks COM components. Finally, we compare the used buffer size and the declared buffer size to identify the stack overflow vulnerability. The experimental result shows that the proposed method is capable of detecting COM components' stack overflow vulnerability. The introduced SBOD (stack buffer overflow detection) algorithm is a promising direction to assist software engineers who seek to detect stack overflow vulnerability in order to improve software quality.

[1]  Crispan Cowan,et al.  StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks , 1998, USENIX Security Symposium.

[2]  David Evans,et al.  Statically Detecting Likely Buffer Overflow Vulnerabilities , 2001, USENIX Security Symposium.

[3]  Stefan D. Bruda,et al.  Counter-Measures against Stack Buffer Overflows in GNU/Linux Operating Systems , 2016, ANT/SEIT.

[4]  Daniel M. Roy,et al.  A dynamic technique for eliminating buffer overflow vulnerabilities (and other memory errors) , 2004, 20th Annual Computer Security Applications Conference.

[5]  Tzi-cker Chiueh,et al.  RAD: a compile-time solution to buffer overflow attacks , 2001, Proceedings 21st International Conference on Distributed Computing Systems.

[6]  Sahel Alouneh,et al.  A software tool to protect executable files from buffer overflow attacks , 2016 .

[7]  Zhao Lei.  STUDY AND REALIZATION OF A LIBRARY FUNCTIONS RECOGNITION ALGORITHM IN C DECOMPILER , 1997 .

[8]  Kyung-Suk Lhee,et al.  Buffer overflow and format string overflow vulnerabilities , 2003, Softw. Pract. Exp..

[9]  Dong Liu,et al.  A highly automated binary software vulnerability risk evaluation method for off-by-one stack based buffer overflow , 2015, 2015 IEEE International Conference on Computer and Communications (ICCC).

[10]  Soo-Hyun Oh,et al.  Detection Mechanism against Code Re-use Attack in Stack region , 2014 .

[11]  Calton Pu,et al.  Buffer overflows: attacks and defenses for the vulnerability of the decade , 2000, Foundations of Intrusion Tolerant Systems, 2003 [Organically Assured and Survivable Information Systems].

[12]  Angelos D. Keromytis,et al.  A Dynamic Mechanism for Recovering from Buffer Overflow Attacks , 2005, ISC.

[13]  John Wilander,et al.  A Comparison of Publicly Available Tools for Dynamic Buffer Overflow Prevention , 2003, NDSS.

[14]  Babak Sadeghiyan,et al.  Smart fuzzing method for detecting stack-based buffer overflow in binary codes , 2016, IET Softw..

[15]  Malcolm I. Heywood,et al.  Evolving successful stack overflow attacks for vulnerability testing , 2005, 21st Annual Computer Security Applications Conference (ACSAC'05).

[16]  Gary McGraw,et al.  ITS4: a static vulnerability scanner for C and C++ code , 2000, Proceedings 16th Annual Computer Security Applications Conference (ACSAC'00).

[17]  Paul H. J. Kelly,et al.  Backwards-Compatible Bounds Checking for Arrays and Pointers in C Programs , 1997, AADEBUG.

[18]  John Wilander,et al.  A Comparison of Publicly Available Tools for Static Intrusion Prevention , 2002 .

[19]  Sahel Alouneh,et al.  Stack Memory Buffer Overflow Protection based on Duplication and Randomization , 2013, EUSPN/ICTH.

[20]  Dawson R. Engler,et al.  ARCHER: using symbolic, path-sensitive analysis to detect memory access errors , 2003, ESEC/FSE-11.