Estimating Defect Density Using Test Coverage

Defect density is one of the most important factors that allow one to decide if a piece of software is ready to be released. In theory, one can find all the defects and count them, however it is impossible to find all the defects within any reasonable amount of time. Estimating defect density can become difficult for high reliability software, since the remaining defects can be extremely hard to test. Defect seeding will work only if the distribution of seeded defects is similar to the existing defects. One possible way is to apply the exponential SRGM and thus estimate the total number of defects present at the beginning of testing. Here we show the problems with this approach and present a new approach based on software test coverage. Software test coverage directly measures the thoroughness of testing avoiding the problem of variations of test effectiveness. Here we present interpretations of the parameters of the coveragedefect-density model presented by Malaiya et al. We apply this model to actual test data to project the residual defect density. The results show that this method results in estimates that are more stable than the existing methods. This method is easier to understand and the convergence to the estimate can be visually observed.

[1]  Pradip K. Srimani,et al.  An Examination of Fault Exposure Ratio , 1993, IEEE Trans. Software Eng..

[2]  Bev Littlewood,et al.  N-version design Versus one Good Version , 2000 .

[3]  Michael R. Lyu,et al.  A coverage analysis tool for the effectiveness of software testing , 1994 .

[4]  Michael R. Lyu,et al.  What is software reliability? , 1994, Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance.

[5]  James M. Bieman,et al.  The relationship between test coverage and reliability , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[6]  Hoang Pham Software Reliability , 1999 .

[7]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[8]  G. B. Finelli,et al.  The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software , 1993, IEEE Trans. Software Eng..

[9]  Yashwant K. Malaiya,et al.  What do the software reliability growth model parameters represent? , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[10]  Nader B. Ebrahimi,et al.  On the Statistical Analysis of the Number of Errors Remaining in a Software Design Document after Inspection , 1997, IEEE Trans. Software Eng..

[11]  Yashwant K. Malaiya,et al.  Fault exposure ratio estimation and applications , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[12]  Steve McConnell,et al.  Best Practices: Gauging Software Readiness with Defect Tracking , 1997, IEEE Softw..

[13]  Les Hatton,et al.  N-Version Design vs. One Good Version , 1997, IEEE Softw..

[14]  Paul Piwowarski,et al.  Coverage measurement experience during function test , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[15]  Alberto Pasquini,et al.  Sensitivity of reliability growth models to operational profile errors , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[16]  Yashwant K. Malaiya,et al.  An Integrated Software Reliability Tool , 1999 .

[17]  Yashwant K. Malaiya,et al.  The Coverage Problem for Random Testing , 1984, ITC.

[18]  Michael R. Lyu,et al.  An empirical study of the correlation between code coverage and reliability estimation , 1996, Proceedings of the 3rd International Software Metrics Symposium.