A Systematic Review of Software Testability Measurement Techniques

As the software applications have become important in our daily lives, the importance of the quality of the software has increased. To have guarantee about the quality of the software, it is essential for it to be tested properly. However, testing the software is very costly phase of the software development life cycle and it becomes more difficult with the growing dimension and complications involved with the software. Therefore, there is a need to make the testing process more effective. Testability improvement is one such method that helps in reducing the testing effort. Numerous methodologies for testability assessment have been anticipated in the literature. This paper provides the result of systematic review of the studies associated with software testability. In this paper, we review several journal and conference papers of software testability. In each study, researchers have used different approaches to measure software testability like controllability, observability etc. Thus this review focuses on various software testabilities theories and studies performed till date. In this paper, we have selected 55 primary studies after following a rigorous selection criteria. Results indicate that controllability and observability concept is widely explored for the measurement of software testability. Also, most of the work is based on object-oriented paradigm at the code level.

[1]  Sungyoung Lee,et al.  Increasing the Testability of Object-Oriented Frameworks with Built-in Tests , 2002, AISA.

[2]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..

[3]  Mourad Badri,et al.  An Empirical Analysis of Lack of Cohesion Metrics for Predicting Testability of Classes , 2011 .

[4]  Joachim Wegener,et al.  Evaluating evolutionary testability for structure-oriented testing with software measurements , 2008, Appl. Soft Comput..

[5]  S. C. Gupta,et al.  Impact of software testability considerations on software development life cycle , 1994, Proceedings of 1994 1st International Conference on Software Testing, Reliability and Quality Assurance (STRQA'94).

[6]  Anju Saha,et al.  Assessment of Class Testability - An Empirical Study , 2009 .

[7]  Myra B. Cohen,et al.  Improving the Testing and Testability of Software Product Lines , 2010, SPLC.

[8]  Keith W. Miller,et al.  An empirical comparison of a dynamic software testability metric to static cyclomatic complexity , 1993 .

[9]  Jeffrey M. Voas,et al.  Improving the software development process using testability research , 1992, [1992] Proceedings Third International Symposium on Software Reliability Engineering.

[10]  Wasif Afzal,et al.  Testability and software performance: a systematic mapping study , 2016, SAC.

[11]  Jin-Cherng Lin,et al.  Software testability measurements derived from data flow analysis , 1998, Proceedings of the Second Euromicro Conference on Software Maintenance and Reengineering.

[12]  Taghi M. Khoshgoftaar,et al.  Detecting program modules with low testability , 1995, Proceedings of International Conference on Software Maintenance.

[13]  Luc Lamontagne,et al.  Towards a Unified Metrics Suite for JUnit Test Cases , 2014, SEKE.

[14]  Jeffrey M. Voas,et al.  Designing programs that are less likely to hide faults , 1993, J. Syst. Softw..

[15]  Martin R. Woodward,et al.  Testability, fault size and the domain-to-range ratio: An eternal triangle , 2000, ISSTA '00.

[16]  Roy S. Freedman,et al.  Testability of Software Components , 1991, IEEE Trans. Software Eng..

[17]  Wasif Afzal,et al.  Testability and Software Robustness: A Systematic Literature Review , 2015, 2015 41st Euromicro Conference on Software Engineering and Advanced Applications.

[18]  Thanh Binh Nguyen,et al.  Testability Analysis of Data-Flow Software , 2005, Electron. Notes Theor. Comput. Sci..

[19]  Yves Le Traon,et al.  Measuring design testability of a UML class diagram , 2005, Information and Software Technology.

[20]  Anita Goel,et al.  COTT – A Testability Framework for Object-Oriented Software Testing , 2008 .

[21]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[22]  Taghi M. Khoshgoftaar,et al.  Predicting testability of program modules using a neural network , 2000, Proceedings 3rd IEEE Symposium on Application-Specific Systems and Software Engineering Technology.

[23]  Mourad Badri,et al.  An empirical analysis of a testability model for object-oriented programs , 2011, SOEN.

[24]  Lydie du Bousquet,et al.  A New Approach for Software Testability , 2010, TAIC PART.

[25]  Arie van Deursen,et al.  An empirical study into class testability , 2006, J. Syst. Softw..

[26]  John D. McGregor,et al.  A Measure of Testing Effort , 1996, COOTS.

[27]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[28]  Mourad Badri,et al.  Exploring Empirically the Relationship between Lack of Cohesion and Testability in Object-Oriented Systems , 2010, FGIT-ASEA.

[29]  Alok Mishra,et al.  A discussion on design factors of client-server software testability , 2009, SOEN.

[30]  K. K. Aggarwal,et al.  A Fuzzy Approach for Integrated Measure of Object-Oriented Software Testability , 2005 .

[31]  Lionel C. Briand,et al.  A measurement framework for object-oriented software testability , 2005, Inf. Softw. Technol..

[32]  Anju Saha,et al.  Prediction of testability using the design metrics for object-oriented software , 2012, Int. J. Comput. Appl. Technol..

[33]  Wanchai Rivepiboon,et al.  Class-component testability analysis , 2006, ICSE 2006.

[34]  Jerry Zeyu Gao,et al.  On Building Testable Software Components , 2002, ICCBSS.

[35]  Anju Saha,et al.  Improving the testability of object oriented software through software contracts , 2010, ACM SIGSOFT Softw. Eng. Notes.

[36]  Khurram Mustafa,et al.  Metric based testability model for object oriented design (MTMOOD) , 2009, SOEN.

[37]  Yuming Zhou,et al.  An in-depth investigation into the relationships between structural metrics and unit testability in object-oriented systems , 2012, Science China Information Sciences.

[38]  Poong-Hyun Seong,et al.  Quantitative evaluation of safety critical software testability based on fault tree analysis and entropy , 2004, J. Syst. Softw..

[39]  Eunjee Song,et al.  An aspect-oriented testability framework , 2012, RACS.

[40]  Mark Harman,et al.  Testability Transformation - Program Transformation to Improve Testability , 2008, Formal Methods and Testing.

[41]  Jeffrey M. Voas,et al.  Semantic metrics for software testability , 1993, J. Syst. Softw..

[42]  Yves Le Traon,et al.  Testability measurements for data flow designs , 1997, Proceedings Fourth International Software Metrics Symposium.

[43]  Jerry Zeyu Gao,et al.  A component testability model for verification and measurement , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[44]  Dirk Muthig,et al.  Making testing product lines more efficient by improving the testability of product line architectures , 2006, ROSATEA '06.

[45]  Yves Le Traon,et al.  Usage and testability of AOP: An empirical study of AspectJ , 2013, Inf. Softw. Technol..

[46]  Jeffrey M. Voas,et al.  Predicting where faults can hide from testing , 1991, IEEE Software.

[47]  Myungchul Kim,et al.  Design for testability of protocols based on formal specifications , 1996 .

[48]  Stefan Jungmayr.  Identifying test-critical dependencies , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[49]  Liang Zhao.  A new approach for software testability analysis , 2006, ICSE '06.

[50]  Mourad Badri,et al.  Empirical Analysis of Object-Oriented Design Metrics for Predicting Unit Testing Effort of Classes , 2012 .

[51]  Poong-Hyun Seong,et al.  Testing digital safety system software with a testability measure based on a software fault tree , 2006, Reliab. Eng. Syst. Saf..

[52]  Richard Bache,et al.  Measures of testability as a basis for quality assurance , 1990, Softw. Eng. J..