Reliable and Reproducible Competition Results with BenchExec and Witnesses (Report on SV-COMP 2016)

The 5$$^{\text {th}}$$ Competition on Software Verification SV-COMP 2016 continues the tradition of a thorough comparative evaluation of fully-automatic software verifiers. This report presents the results of the competition and includes a special section that describes how SV-COMP ensures that the experiments are reliably executed, precisely measured, and organized such that the results can be reproduced later. SV-COMP uses BenchExec for controlling and measuring the verification runs, and requires violation witnesses in an exchangeable format, whenever a verifier reports that a property is violated. Each witness was validated by two independent and publicly-available witness validators. The tables report the state of the art in software verification in terms of effectiveness and efficiency. The competition used 6i¾?661 verification tasks that each consisted of a C program and a property reachability, memory safety, termination. SV-COMP 2016 had 35i¾?participating verification systems 22i¾?in 2015 from 16i¾?countries.