Report on the third static analysis tool exposition (SATE 2010)

The NIST Software Assurance Metrics And Tool Evaluation (SAMATE) project conducted the third Static Analysis Tool Exposition (SATE) in 2010 to advance research in static analysis tools that find security defects in source code. The main goals of SATE were to enable empirical research based on large test sets, encourage improvements to tools, and promote broader and more rapid adoption of tools by objectively demonstrating their use on production software. Briefly, participating tool makers ran their tool on a set of programs. Researchers led by NIST performed a partial analysis of tool reports. The results and experiences were reported at the SATE 2010 Workshop in Gaithersburg, MD, in October, 2010. The tool reports and analysis were made publicly available in 2011. This special publication consists of the following three papers. “The Third Static Analysis Tool Exposition (SATE 2010),” by Vadim Okun, Aurelien Delaitre, and Paul E. Black, describes the SATE procedure and provides observations based on the data collected. The other two papers are written by participating tool makers. “Goanna Static Analysis at the NIST Static Analysis Tool Exposition,” by Mark Bradley, Ansgar Fehnker, Ralf Huuck, and Paul Steckler, introduces Goanna, which uses a combination of static analysis with model checking, and describes its SATE experience, tool results, and some of the lessons learned in the process. Serguei A. Mokhov introduces a machine learning approach to static analysis and presents MARFCAT’s SATE 2010 results in “The use of machine learning with signaland NLP processing of source code to fingerprint, detect, and classify vulnerabilities and weaknesses with MARFCAT.”