The Juliet test suite is a systematic set of thousands of small test programs in C/C++ and Java, exhibiting over 100 classes of errors, such as buffer overflow, OS injection, hardcoded password, absolute path traversal, NULL pointer dereference, uncaught exception, deadlock, and missing release of resource. These test programs should be helpful in determining capabilities of software assurance tools, particularly static analyzers, in Unix, Microsoft Windows, and other environments. Juliet was developed by the National Security Agency’s Center for Assured Software and first released in December 2010. It has been enhanced twice since then. Version 1.2 was released in May 2013 with a total of 86 864 test cases. In the years after its release, many problems and deficiencies in Version 1.2 came to our attention. Released in October 2017, Version 1.3 fixes about fourteen systematic problems in Version 1.2 and adds tests for prefix and postfix increment integer overflow and decrement integer underflow. This technical note details the changes from Version 1.2 to 1.3. This note also lists known problems remaining in Juliet 1.3.
[1]
Paul E. Black.
SARD: Thousands of Reference Programs for Software Assurance | NIST
,
2017
.
[2]
Paul E. Black,et al.
SATE V Ockham Sound Analysis Criteria
,
2016
.
[3]
Paul E. Black,et al.
Juliet 1.1 C/C++ and Java Test Suite
,
2012,
Computer.
[4]
Irena Bojanova,et al.
Cryptography classes in bugs framework (BF): Encryption bugs (ENC), verification bugs (VRF), and key management bugs (KMN)
,
2017,
2017 IEEE 28th Annual Software Technology Conference (STC).
[5]
Yan Wu,et al.
The Bugs Framework (BF): A Structured Approach to Express Bugs
,
2016,
2016 IEEE International Conference on Software Quality, Reliability and Security (QRS).
[6]
Elizabeth Fong,et al.
Large Scale Generation of Complex and Faulty PHP Test Cases
,
2016,
2016 IEEE International Conference on Software Testing, Verification and Validation (ICST).
[7]
Elizabeth N. Fong,et al.
Improving Software Assurance through Static Analysis Tool Expositions | NIST
,
2017
.