Analyzing the effect of imperfect debugging on software fault detection and correction processes via a simulation framework

Abstract During a debugging operation, there is a high probability that an additional fault will be introduced into the program when removing an existing fault. Thus, perfect debugging is an ideal but impractical assumption when modeling software reliability. If the debugging of a software system is imperfect, more faults may be introduced and detected. In such cases, it may be necessary to add more staff to the debugging team to share the load and ensure the quality of the software. To investigate the effects of imperfect debugging, we simulate the fault detection and correction processes by a single-queue multichannel queuing model with feedback. In this paper, two debugging procedures are discussed. The first, called Procedure_perfect_debugging, is based on a single-queue multichannel queuing model and deals with the case of perfect debugging. Then, we relax the restriction on perfect debugging, and further propose Procedure_imperfect_ debugging based on a queuing model with feedback to address the case of imperfect debugging. We demonstrate the implementation of the procedures via two case studies in which we quantify the effects of imperfect debugging in terms of throughput, time consumption, and debugger utilization. Finally, based on the measurement results, we determine the most suitable staffing level (i.e., the number of debuggers required) for a debugging system under different degrees of imperfect debugging.

[1]  Tadashi Dohi,et al.  On the Effect of Fault Removal in Software Testing - Bayesian Reliability Estimation Approach , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[2]  Liang Tian,et al.  Modeling distributed software defect removal effectiveness in the presence of code churn , 2005, Math. Comput. Model..

[3]  Mitsuru Ohba,et al.  Does imperfect debugging affect software reliability growth? , 1989, ICSE '89.

[4]  C.-T. Lin,et al.  Software Reliability Analysis by Considering Fault Dependency and Debugging Time Lag , 2006, IEEE Transactions on Reliability.

[5]  Chin-Yu Huang,et al.  Quantifying the Influences of Imperfect Debugging on Software Development Using Simulation Approach , 2009, FGIT-ASEA.

[6]  Jun Zheng,et al.  Predicting software reliability with neural network ensembles , 2009, Expert Syst. Appl..

[7]  Hoang Pham,et al.  A software cost model with imperfect debugging, random life cycle and penalty cost , 1996, Int. J. Syst. Sci..

[8]  Q. P. Hu,et al.  Early Software Reliability Prediction with ANN Models , 2006, 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06).

[9]  Amrit L. Goel,et al.  An Analysis Of Recurrent Software Errors In A Real-Time Control System , 1978, ACM Annual Conference.

[10]  Kishor S. Trivedi Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[11]  Norman F. Schneidewind Fault correction profiles , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[12]  Chin-Yu Huang,et al.  Neural-network-based approaches for software reliability estimation using dynamic weighted combinational models , 2007, J. Syst. Softw..

[13]  Liang Tian,et al.  On-line prediction of software reliability using an evolutionary connectionist model , 2005, J. Syst. Softw..

[14]  Swapna S. Gokhale,et al.  A simulation approach to structure-based software reliability analysis , 2005, IEEE Transactions on Software Engineering.

[15]  S.H. Ng,et al.  Simulation-based FDP & FCP analysis with queueing models , 2008, 2008 IEEE International Conference on Industrial Engineering and Engineering Management.

[16]  Shigeru Yamada SOFTWARE RELIABILITY GROWTH MODELS INCORPORATING IMPERFECT DEBUGGING WITH INTRODUCED FAULTS , 1998 .

[17]  Q. P. Hu,et al.  Modeling and Analysis of Software Fault Detection and Correction Process by Considering Time Dependency , 2007, IEEE Transactions on Reliability.

[18]  Shigeru Yamada,et al.  An imperfect debugging model with two types of hazard rates for software reliability measurement and assessment , 2000 .

[19]  Ping-Feng Pai,et al.  System reliability forecasting by support vector machines with genetic algorithms , 2006, Math. Comput. Model..

[20]  P. K. Kapur,et al.  Modelling an imperfect debugging phenomenon in software reliability , 1996 .

[21]  Michael R. Lyu,et al.  A Generalized Technique for Simulating Software Reliability , 1996, IEEE Softw..

[22]  Jun Zhu,et al.  A novel method for real parameter optimization based on Gene Expression Programming , 2009, Appl. Soft Comput..

[23]  Eric V. Slud,et al.  Testing for Imperfect Debugging in Software Reliability , 1997 .

[24]  Mauro Pezzè,et al.  Software testing and analysis - process, principles and techniques , 2007 .

[25]  Kai-Yuan Cai,et al.  Software Defect and Operational Profile Modeling , 1998, The Kluwer International Series in Software Engineering.

[26]  Xuemei Zhang,et al.  An NHPP Software Reliability Model and Its Comparison , 1997 .

[27]  Gregory Levitin,et al.  Robust recurrent neural network modeling for software fault detection and correction prediction , 2007, Reliab. Eng. Syst. Saf..

[28]  Chin-Yu Huang,et al.  Staffing Level and Cost Analyses for Software Debugging Activities Through Rate-Based Simulation Approaches , 2009, IEEE Transactions on Reliability.

[29]  Swapna S. Gokhale,et al.  Analysis of Software Fault Removal Policies Using a Non-Homogeneous Continuous Time Markov Chain , 2004, Software Quality Journal.

[30]  Chin-Yu Huang,et al.  Reliability prediction and assessment of fielded software based on multiple change-point models , 2005, 11th Pacific Rim International Symposium on Dependable Computing (PRDC'05).

[31]  Shigeru Yamada,et al.  Markovian software reliability measurement with a geometrically decreasing perfect debugging rate , 2003 .

[32]  Michael R. Lyu Software Reliability Engineering: A Roadmap , 2007, Future of Software Engineering (FOSE '07).

[33]  Simon P. Wilson,et al.  Software Reliability Modeling , 1994 .

[34]  Hoang Pham Software Reliability , 1999 .

[35]  Michael Haug,et al.  Software Quality Approaches: Testing, Verification, and Validation , 2001, Springer Berlin Heidelberg.

[36]  Swapna S. Gokhale,et al.  Incorporating fault debugging activities into software reliability models: a simulation approach , 2006, IEEE Transactions on Reliability.

[37]  P. R. Jayashree,et al.  Transient solutions of a software model with imperfect debugging and generation of errors by two servers , 1998, Int. J. Comput. Math..

[38]  Carl M. Harris,et al.  Fundamentals of queueing theory , 1975 .

[39]  Min Xie,et al.  A Study of the Effect of Imperfect Debugging on Software Development Cost , 2003, IEEE Trans. Software Eng..

[40]  Giuliano Antoniol,et al.  Assessing staffing needs for a software maintenance project through queuing simulation , 2004, IEEE Transactions on Software Engineering.

[41]  L. Darrell Whitley,et al.  Prediction of Software Reliability Using Connectionist Models , 1992, IEEE Trans. Software Eng..

[42]  Tadashi Dohi,et al.  An infinite server queueing approach for describing software reliability growth: unified modeling and estimation framework , 2004, 11th Asia-Pacific Software Engineering Conference.

[43]  Thong Ngee Goh,et al.  A study of the connectionist models for software reliability prediction , 2003 .