Toward a software testing and reliability early warning metric suite

The field reliability is measured too late for affordably guiding corrective action to improve the quality of the software. Software developers can benefit from an early warning of their reliability while they can still affordably react. This early warning can be built from a collection of internal metrics. An internal metric, such as the number of lines of code, is a measure derived from the product itself. An external measure is a measure of a product derived from assessment of the behavior of the system. For example, the number of defects found in test is an external measure. The ISO/IEC standard states that [i]nternal metrics are of little value unless there is evidence that they are related to external quality. Internal metrics can be collected in-process and more easily than external metrics. Additionally, internal metrics have been shown to be useful as early indicators of externally-visible product quality. For these early indicators to be meaningful, they must be related (in a statistically significant and stable way) to the field quality/reliability of the product. The validation of such metrics requires the convincing demonstration that (1) the metric measures what it purports to measure and (2) the metric is associated with an important external metric, such as field reliability, maintainability, or fault-proneness. Software metrics have been used as indicators of software quality and fault proneness. There is a growing body of empirical results that supports the theoretical validity of the use of higher-order early metrics, such as OO metrics defined by Chidamber-Kemerer (CK) and the MOOD OO metric suites as predictors of field quality. However, general validity of these metrics (which are often unrelated to the actual operational profile of the product) is still open to criticism.