Fault Detection Likelihood of Test Sequence Length

Testing of graphical user interfaces is important due to its potential to reveal faults in operation and performance of the system under consideration. Most existing test approaches generate test cases as sequences of events of different length. The cost of the test process depends on the number and total length of those test sequences. One of the problems to be encountered is the determination of the test sequence length. Widely accepted hypothesis is that the longer the test sequences, the higher the chances to detect faults. However, there is no evidence that an increase of the test sequence length really affect the fault detection. This paper introduces a reliability theoretical approach to analyze the problem in the light of real-life case studies. Based on a reliability growth model the expected number of additional faults is predicted that will be detected when increasing the length of test sequences.

[1]  W. Nelson.  Statistical Methods for Reliability Data , 1998 .

[2]  Atif M. Memon,et al.  Empirical evaluation of the fault-detection effectiveness of smoke regression test cases for GUI-based software , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[3]  Fevzi Belli,et al.  Test minimization for human-computer interaction , 2006, Applied Intelligence.

[4]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[5]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[6]  Bogdan Korel,et al.  Automated test data generation for programs with procedures , 1996, ISSTA '96.

[7]  Daniel P. Siewiorek,et al.  A method to automate user interface testing using variable finite state machines , 1997, Proceedings of IEEE 27th International Symposium on Fault Tolerant Computing.

[8]  J. Bert Keats,et al.  Statistical Methods for Reliability Data , 1999 .

[9]  John D. Musa,et al.  Software reliability measurement , 1984, J. Syst. Softw..

[10]  Gregory Levitin,et al.  Software reliability modelling and optimization for multi-release software development processes , 2011, 2011 IEEE International Conference on Industrial Engineering and Engineering Management.

[11]  Fevzi Belli,et al.  Finite state testing and analysis of graphical user interfaces , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[12]  Alexandre Petrenko,et al.  Protocol testing: review of methods and relevance for software testing , 1994, ISSTA '94.

[13]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[14]  Aditya P. Mathur,et al.  Foundations of Software Testing , 2007 .

[15]  F. Belli,et al.  On “ Negative ” Tests of Web Applications , 2007 .

[16]  David Vere-Jones,et al.  Point Processes , 2011, International Encyclopedia of Statistical Science.

[17]  Mary Lou Soffa,et al.  Hierarchical GUI Test Case Generation Using Automated Planning , 2001, IEEE Trans. Software Eng..

[18]  Yu Lei,et al.  A Test Generation Strategy for Pairwise Testing , 2002, IEEE Trans. Software Eng..

[19]  Krishan K. Sabnani,et al.  A Protocol Test Generation Procedure , 1988, Comput. Networks.

[20]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[21]  Alessandro Birolini.  Reliability Engineering: Theory and Practice , 1999 .

[22]  Mary Lou Soffa,et al.  Coverage criteria for GUI testing , 2001, ESEC/FSE-9.

[23]  Harald A. Stieber,et al.  A Family of Software Reliability Growth Models , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[24]  Fevzi Belli,et al.  Event‐based modelling, analysis and testing of user interactions: approach and case study , 2006, Softw. Test. Verification Reliab..

[25]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.