Code-based prioritization: a pre-testing effort to minimize post-release failures

Improving the efficiency of the testing process is a challenging goal. Prior work has shown that often a small number of errors account for the majority of software failures; and often, most errors are found in a small portion of a source code. We argue that prioritizing code elements before conducting testing can help testers focus their testing effort on the parts of the code most likely to expose errors. This can, in turn, promote more efficient testing of software. Keeping this in view, we propose a testing effort prioritization method to guide tester during software development life cycle. Our approach considers five factors of a component such as Influence value, Average execution time, Structural complexity, Severity and Value as inputs and produce the priority value of the component as an output. Once all components of a program have been prioritized, testing effort can be apportioned so that the components causing more frequent and/or more severe failures will be tested more thoroughly. Our proposed approach is effective in guiding testing effort as it is linked to external measure of defect severity and business value, internal measure of frequency and complexity. As a result, the failure rate is decreased and the chance of severe type of failures is also decreased in the operational environment. We have conducted experiments to compare our scheme with a related scheme. The results establish that our proposed approach that prioritizes the testing effort within the source code is able to minimize highly severed types of failures and also number of failures at the post-release time of a software system.

[1]  Edward N. Adams,et al.  Optimizing Preventive Service of Software Products , 1984, IBM J. Res. Dev..

[2]  Durga Prasad Mohapatra,et al.  Source Code Prioritization Using Forward Slicing for Exposing Critical Elements in a Program , 2011, Journal of Computer Science and Technology.

[3]  J. Jenny Li,et al.  Code-coverage guided prioritized test generation , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[4]  Barry W. Boehm Value-based software engineering: reinventing , 2003, SOEN.

[5]  Taylor L. Booth,et al.  Performance Optimization of Software Systems Processing Information Sequences Modeled by Probabilistic Languages , 1979, IEEE Transactions on Software Engineering.

[6]  John D. McGregor,et al.  An extensible program representation for object-oriented software , 1994, SIGP.

[7]  Barry Boehm,et al.  Top 10 list [software development] , 2001 .

[8]  Gregg Rothermel,et al.  Test Case Prioritization: A Family of Empirical Studies , 2002, IEEE Trans. Software Eng..

[9]  Aditya P. Mathur,et al.  Interface Mutation Test Adequacy Criterion: An Empirical Evaluation , 2001, Empirical Software Engineering.

[10]  J.H. Andrews,et al.  Is mutation an appropriate tool for testing experiments? [software testing] , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[11]  Roger C. Cheung,et al.  A User-Oriented Software Reliability Model , 1978, IEEE Transactions on Software Engineering.

[12]  Barry W. Boehm,et al.  Value-Based Software Engineering: A Case Study , 2003, Computer.

[13]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[14]  Lionel C. Briand,et al.  A comprehensive and systematic methodology for client-server class integration testing , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[15]  Carl S. Carlson,et al.  Failure Mode Effects and Criticality Analysis (FMECA) , 2012 .

[16]  Harlan D. Mills,et al.  Engineering software under statistical quality control , 1990, IEEE Software.

[17]  Mary Jean Harrold,et al.  Slicing object-oriented software , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[18]  Barry W. Boehm,et al.  Bridge the Gap between Software Test Process and Business Value: A Case Study , 2009, ICSP.

[19]  K. Goseva-Popstojanova,et al.  UML based severity analysis methodology , 2005, Annual Reliability and Maintainability Symposium, 2005. Proceedings..

[20]  Priti Shankar,et al.  The Compiler Design Handbook: Optimizations and Machine Code Generation , 2002, The Compiler Design Handbook.

[21]  Eugene Miya,et al.  On "Software engineering" , 1985, SOEN.

[22]  Barry W. Boehm,et al.  How Much Software Quality Investment Is Enough: A Value-Based Approach , 2006, IEEE Software.

[23]  Neelam Gupta,et al.  Experiments with test case prioritization using relevant slices , 2008, J. Syst. Softw..

[24]  John D. Musa,et al.  Software Reliability Engineering: More Reliable Software Faster and Cheaper , 2004 .

[25]  Barry W. Boehm,et al.  Value-based software engineering: reinventing , 2003, SOEN.

[26]  Gregg Rothermel,et al.  Incorporating varying test costs and fault severities into test case prioritization , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[27]  Michael R. Lyu,et al.  Optimal resource allocation and reliability analysis for component-based software applications , 2002, Proceedings 26th Annual International Computer Software and Applications.

[28]  Donglin Liang,et al.  Slicing objects using system dependence graphs , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[29]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[30]  J. Jenny Li,et al.  Prioritize code for testing to improve code coverage of complex software , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[31]  John D. Musa,et al.  Operational profiles in software-reliability engineering , 1993, IEEE Software.

[32]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[33]  Barry W. Boehm,et al.  Software Defect Reduction Top 10 List , 2001, Computer.

[34]  Bev Littlewood,et al.  A Reliability Model for Systems with Markov Structure , 1975 .

[35]  Stefan Biffl,et al.  Value-Based Management of Software Testing , 2006, Value-Based Software Engineering.

[36]  John A. Clark,et al.  Class Mutation : Mutation Testing for Object-Oriented Programs , 2000 .

[37]  Durga Prasad Mohapatra,et al.  Reliability Improvement Based on Prioritization of Source Code , 2010, ICDCIT.

[38]  Ian Sommerville,et al.  Software engineering (5th ed.) , 1995 .

[39]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[40]  Linda H. Rosenberg,et al.  Risk-Based Object Oriented Testing , 2000 .

[41]  Barry W. Boehm,et al.  Cost models for future software life cycle processes: COCOMO 2.0 , 1995, Ann. Softw. Eng..

[42]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[43]  Durga Prasad Mohapatra,et al.  A scheme to prioritize classes at the early stage for improving observable reliability , 2010, ISEC.