Characterizing the 'security vulnerability likelihood' of software functions

Software maintainers and auditors would benefit from a tool to help them focus their attention on functions that are likely to be the source of security vulnerabilities. However, the existence of such a tool is predicated on the ability to characterize a function's 'security vulnerability likelihood'. Our hypothesis is that functions near a source of input are most likely to contain security vulnerability. These functions should be a small percentage of the total number of functions in the system. To validate this hypothesis, we performed an experiment involving thirty one vulnerabilities in thirty open source systems. This paper describes the experiment, its outcome, and the tools used to conduct it. It also describes the FLF (front line functions) finder, which is a tool that was developed using knowledge gathered from the outcome of the experiment. This tool automates the detection of high-risk functions. To demonstrate the effectiveness of the FLF finder, three open source applications with known vulnerabilities were tested. In addition to this test, a case study was performed on the privilege separation code in the OpenSSH server daemon.

[1]  Crispan Cowan,et al.  StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks , 1998, USENIX Security Symposium.

[2]  David S. Rosenblum,et al.  Representing Semantically Analyzed C++ Code with Reprise , 1991, C++ Conference.

[3]  Matt Bishop,et al.  A Taxonomy of UNIX System and Network Vulnerabilities , 1997 .

[4]  Eugene H. Spafford,et al.  Software vulnerability analysis , 1998 .

[5]  Ratul Mahajan,et al.  Controlling high bandwidth aggregates in the network , 2002, CCRV.

[6]  Ivan Krsul,et al.  Computer Vulnerability Analysis: Thesis Proposal , 1997 .

[7]  A. One,et al.  Smashing The Stack For Fun And Profit , 1996 .

[8]  Ratul Mahajan,et al.  Controlling High Bandwidth Aggregates in the Network (Extended Version) , 2001 .

[9]  Emden R. Gansner,et al.  A C++ data model supporting reachability analysis and dead code detection , 1997, ESEC '97/FSE-5.

[10]  Crispin Cowan,et al.  FormatGuard: Automatic Protection From printf Format String Vulnerabilities , 2001, USENIX Security Symposium.

[11]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[12]  Wojtek Kozaczynski,et al.  Automated support for legacy code understanding , 1994, CACM.

[13]  David Evans,et al.  Statically Detecting Likely Buffer Overflow Vulnerabilities , 2001, USENIX Security Symposium.

[14]  Emden R. Gansner,et al.  REportal: a Web-based portal site for reverse engineering , 2001, Proceedings Eighth Working Conference on Reverse Engineering.

[15]  David A. Wagner,et al.  This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. Detecting Format String Vulnerabilities with Type Qualifiers , 2001 .

[16]  Jeffrey L. Korn,et al.  Chava: reverse engineering and tracking of Java applets , 1999, Sixth Working Conference on Reverse Engineering (Cat. No.PR00303).

[17]  Vitaly Osipov,et al.  Format String Attacks , 2005 .

[18]  David A. Wagner,et al.  MOPS: an infrastructure for examining security properties of software , 2002, CCS '02.

[19]  Robert E. Tarjan,et al.  Fast Algorithms for Finding Nearest Common Ancestors , 1984, SIAM J. Comput..

[20]  Niels Provos,et al.  Preventing Privilege Escalation , 2003, USENIX Security Symposium.