An Empirical Assessment on Program Module-Order Models

Abstract Software reliability can be improved by detecting latent software defects prior to deployment. A practical approach is to target available resources toward program modules likely to have faults, i.e., low-reliability modules. An empirical investigation on program module-order models (MOM) is presented. A module-order model assists management in prioritizing verification and validation tasks toward defective modules, providing a goal-oriented software reliability enhancement. The output of a MOM is a quality-based ranking of program modules where quality is estimated by an underlying software defect prediction model. We present an empirical investigation on module-order models designed using different underlying defect prediction models. Based on case studies of various industrial systems, it is observed that there is no single fault prediction model that always results in the preferred MOM. In fact, performance of a MOM was dependent on the cutoff range of interest, i.e., how many modules can be inspected with the given resources. Consequently, a practitioner can select a MOM based on a given cutoff range of interest.

[1]  Stephen G. MacDonell,et al.  Software Metrics Data Analysis—Exploring the Relative Performance of Some Commonly Used Modeling Techniques , 1999, Empirical Software Engineering.

[2]  Taghi M. Khoshgoftaar,et al.  Emerald: Software Metrics and Models on the Desktop , 1996, IEEE Softw..

[3]  Jeffrey J. P. Tsai,et al.  Machine Learning and Software Engineering , 2004, Software Quality Journal.

[4]  Taghi M. Khoshgoftaar,et al.  Predicting testability of program modules using a neural network , 2000, Proceedings 3rd IEEE Symposium on Application-Specific Systems and Software Engineering Technology.

[5]  Witold Pedrycz,et al.  Software quality analysis with the use of computational intelligence , 2003, Inf. Softw. Technol..

[6]  Barry W. Boehm,et al.  What we have learned about fighting defects , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[7]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[8]  C. S. George Lee,et al.  Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems , 1996 .

[9]  William E. Perry,et al.  Effective methods for software testing , 1995 .

[10]  N. E. Schneidewind,et al.  Body of Knowledge for Software Quality Measurement , 2002, Computer.

[11]  Khaled El Emam,et al.  Comparing case-based reasoning classifiers for predicting high risk software components , 2001, J. Syst. Softw..

[12]  Taghi M. Khoshgoftaar,et al.  Tree-based software quality estimation models for fault prediction , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[13]  Martin J. Shepperd,et al.  Comparing Software Prediction Techniques Using Simulation , 2001, IEEE Trans. Software Eng..

[14]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[15]  Daryl Pregibon,et al.  Tree-based models , 1992 .

[16]  Taghi M. Khoshgoftaar,et al.  Ordering Fault-Prone Software Modules , 2003, Software Quality Journal.

[17]  Adam A. Porter,et al.  Experimental Software Engineering: A Report on the State of the Art , 1995, 1995 17th International Conference on Software Engineering.

[18]  Alberto Suárez,et al.  Globally Optimal Fuzzy Decision Trees for Classification and Regression , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[20]  Taghi M. Khoshgoftaar,et al.  Fault Prediction Modeling for Software Quality Estimation: Comparing Commonly Used Techniques , 2003, Empirical Software Engineering.

[21]  W. Pedrycz,et al.  Self organizing maps as a tool for software analysis , 2001, Canadian Conference on Electrical and Computer Engineering 2001. Conference Proceedings (Cat. No.01TH8555).

[22]  David M. Levine,et al.  Intermediate Statistical Methods and Applications: A Computer Package Approach , 1982 .

[23]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[24]  Taghi M. Khoshgoftaar,et al.  An empirical study of the impact of count models predictions on module-order models , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[25]  Taghi M. Khoshgoftaar,et al.  A Comparative Study of Ordering and Classification of Fault-Prone Software Modules , 1999, Empirical Software Engineering.

[26]  Taghi M. Khoshgoftaar,et al.  Software quality estimation with case-based reasoning , 2004, Adv. Comput..