Reliability and risk analysis for software that must be safe

Remaining failures, total failures, test time required to attain a given fraction of remaining failures, and time to next failure are useful reliability metrics for: providing confidence that the software has achieved reliability goals; rationalizing how long to test a piece of software; and analyzing the risk of not achieving remaining failure and time to next failure goals. Having predictions of the extent that the software is not fault free (remaining failures) and whether it is likely to survive a mission (time to next failure) provide criteria for assessing the risk of deploying the software. Furthermore, the fraction of remaining failures can be used as both a program quality goal in predicting test time requirements and, conversely as an indicator of program quality as a function of test time expended. We show how these software reliability predictions can increase confidence in the reliability of safety critical software such as the NASA Space Shuttle Primary Avionics Software.