Parameterized GUI Tests

GUI testing is a form of system testing where test cases are based on user interactions. A user interaction may be encoded by a sequence of events (e.g., mouse clicks) together with input data (e.g., string values for text boxes). For selecting event sequences, one can use the black-box approach based on Event Flow Graphs. For selecting input data, one can use the white-box approach based on parameterized unit tests and symbolic execution. The contribution of this paper is an approach to make the principle of parameterized unit testing available to black-box GUI testing. The approach is based on the new notion of parameterized GUI tests. We have implemented the approach in a new tool. In order to evaluate whether parameterized GUI tests have the potential to achieve high code coverage, we apply the tool to four open source GUI applications. The results are encouraging.

[1]  Atif M. Memon,et al.  GUI ripping: reverse engineering of graphical user interfaces for testing , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..

[2]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[3]  Andreas Zeller,et al.  EXSYST: Search-based GUI testing , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[4]  Nikolai Tillmann,et al.  Achieving both model and code coverage with automated gray-box testing , 2007, A-MOST '07.

[5]  Ajitha Rajan,et al.  Requirements Coverage as an Adequacy Measure for Conformance Testing , 2008, ICFEM.

[6]  Andreas Podelski,et al.  Lightweight Static Analysis for GUI Testing , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[7]  Atif M. Memon,et al.  What test oracle should I use for effective GUI testing? , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[8]  Koushik Sen.  DART: Directed Automated Random Testing , 2009, Haifa Verification Conference.

[9]  Fevzi Belli,et al.  Finite state testing and analysis of graphical user interfaces , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[10]  Michael D. Ernst,et al.  Combined static and dynamic automated test generation , 2011, ISSTA '11.

[11]  Lee J. White,et al.  User-based testing of GUI sequences and their interactions , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[12]  Myra B. Cohen,et al.  Covering array sampling of input event sequences for automated gui testing , 2007, ASE.

[13]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[14]  Sarfraz Khurshid,et al.  Event Listener Analysis and Symbolic Execution for Testing GUI Applications , 2009, ICFEM.

[15]  Michael R. Lowry,et al.  Combining unit-level symbolic execution and system-level concrete execution for testing nasa software , 2008, ISSTA '08.

[16]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[17]  Nikolai Tillmann,et al.  Parameterized Unit Testing with Pex , 2008, TAP.

[18]  Andreas Zeller,et al.  Generating parameterized unit tests , 2011, ISSTA '11.

[19]  Nikolai Tillmann,et al.  Parameterized unit tests , 2005, ESEC/FSE-13.

[20]  Atif M. Memon,et al.  Using GUI Run-Time State as Feedback to Generate Test Cases , 2007, 29th International Conference on Software Engineering (ICSE'07).