A New Challenge for Applying Time Series Metrics Data to Software Quality Estimation

In typical software development, a software reliability growth model (SRGM) is applied in each testing activity to determine the time to finish the testing. However, there are some cases in which the SRGM does not work correctly. That is, the SRGM sometimes mistakes quality for poor quality products. In order to tackle this problem, we focussed on the trend of time series data of software defects among successive testing phases and tried to estimate software quality using the trend. First, we investigate the characteristics of the time series data on the detected faults by observing the change of the number of detected faults. Using the rank correlation coefficient, the data are classified into four kinds of trends. Next, with the intention of estimating software quality, we investigate the relationship between the trends of the time series data and software quality. Here, software quality is defined by the number of faults detected during six months after shipment. Finally, we find a relationship between the trends and metrics data collected in the software design phase. Using logistic regression, we statistically show that two review metrics in the design and coding phase can determine the trend.

[1]  Standard Glossary of Software Engineering Terminology , 1990 .

[2]  Taghi M. Khoshgoftaar,et al.  The Detection of Fault-Prone Programs , 1992, IEEE Trans. Software Eng..

[3]  Daniel Horch,et al.  Practical guide to software quality management , 1996 .

[4]  Michael E. Fagan.  Advances in software inspections , 1986, IEEE Transactions on Software Engineering.

[5]  G. J. Knafl,et al.  Residual fault density prediction using regression methods , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[6]  Maurice H. Halstead,et al.  Elements of software science (Operating and programming systems series) , 1977 .

[7]  Bart M. Broekman,et al.  Testing Enbredded Software , 2002 .

[8]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[9]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[10]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[11]  Mark C. Paulk,et al.  Capability Maturity Model , 1991 .

[12]  E. Lehmann,et al.  Nonparametrics: Statistical Methods Based on Ranks , 1976 .

[13]  Shinji Kusumoto,et al.  Analysis of review's effectiveness based on software metrics , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[14]  Mitchell J. Mergenthaler.  Nonparametrics: Statistical Methods Based on Ranks , 1979 .

[15]  Shinji Kusumoto,et al.  Improvement of Software Process by Process Description and Benefit Estimation , 1995, 1995 17th International Conference on Software Engineering.

[16]  John D. Musa,et al.  Software reliability measurement , 1984, J. Syst. Softw..

[17]  Taghi M. Khoshgoftaar,et al.  Tree-based software quality estimation models for fault prediction , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[18]  Brian Marick,et al.  The craft of software testing - subsystem testing including object-based and object-oriented testing , 1994 .

[19]  David M. Marks.  Testing Very Big Systems , 1991 .

[20]  Bart Broekman,et al.  Testing Embedded Software , 2002 .

[21]  M. Kendall,et al.  Rank Correlation Methods (5th ed.). , 1992 .

[22]  Yooichi Yokoyama,et al.  Software cost and quality analysis by statistical approaches , 1998, Proceedings of the 20th International Conference on Software Engineering.

[23]  Amrit L. Goel,et al.  Software Reliability Models: Assumptions, Limitations, and Applicability , 1985, IEEE Transactions on Software Engineering.

[24]  Carol Withrow,et al.  Prediction and control of ADA software defects , 1990, J. Syst. Softw..

[25]  Norman F. Schneidewind.  Software metrics model for integrating quality control and prediction , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[26]  Carol S. Smidts,et al.  Software reliability models: an approach to early reliability prediction , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[27]  Thomas Ragg,et al.  Using machine learning for estimating the defect content after an inspection , 2004, IEEE Transactions on Software Engineering.

[28]  James R. Lyle,et al.  A Two-Person Inspection Method to Improve Prog ramming Productivity , 1989, IEEE Transactions on Software Engineering.

[29]  Yasunari Takagi,et al.  On estimating testing effort needed to assure field quality in software development , 2002, 13th International Symposium on Software Reliability Engineering, 2002. Proceedings..