Grey-box GUI Testing: Efficient Generation of Event Sequences

Graphical user interfaces (GUIs) encode, as event sequences, potentially unbounded ways to interact with software. During testing it becomes necessary to effectively sample the GUI’s event space. Ideally, for increasing the efficiency and effectiveness of GUI testing, one would like to sample the GUI’s event space by only generating sequences that (1) are allowed by the GUI’s structure, and (2) chain together only those events that have data dependencies between their event handlers. We propose a new model, called an eventdependency graph (EDG) of the GUI that captures data dependencies between the code of event handlers. We develop a mapping between an EDG and an existing black-box model of the GUI’s structure, called an event-flow graph (EFG). We automate the EDG construction in a tool that analyzes the bytecode of each event handler. We evaluate our“grey-box”approach using four open-source applications and compare it with the EFG approach. Our results show that using the EDG reduces the number of event sequences with respect to the EFG, while still achieving at least the same coverage. Furthermore, we are able to detect 2 new bugs in the subject applications.

[1]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[2]  Frank Yellin,et al.  The Java Virtual Machine Specification , 1996 .

[3]  Mary Lou Soffa,et al.  Plan Generation for GUI Testing , 2000, AIPS.

[4]  Fevzi Belli,et al.  Finite state testing and analysis of graphical user interfaces , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[5]  Atif M. Memon,et al.  GUI ripping: reverse engineering of graphical user interfaces for testing , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..

[6]  Alex Groce,et al.  Heuristics for model checking Java programs , 2004, International Journal on Software Tools for Technology Transfer.

[7]  Ondrej Lhoták,et al.  Integrating the Soot Compiler Infrastructure into an IDE , 2004, CC.

[8]  Atif M. Memon,et al.  Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software , 2005, IEEE Transactions on Software Engineering.

[9]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[10]  Nikolai Tillmann,et al.  Modeling and Testing Hierarchical GUIs , 2005, Abstract State Machines.

[11]  Sergiu M. Dascalu,et al.  Challenges and opportunities for improving code-based testing of graphical user interfaces , 2006, J. Comput. Methods Sci. Eng..

[12]  Hassan Reza,et al.  A Model-Based Approach for Testing GUI Using Hierarchical Predicate Transition Nets , 2007, Fourth International Conference on Information Technology (ITNG'07).

[13]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[14]  Nikolai Tillmann,et al.  Achieving both model and code coverage with automated gray-box testing , 2007, A-MOST '07.

[15]  Atif M. Memon,et al.  Call-Stack Coverage for GUI Test Suite Reduction , 2008, IEEE Transactions on Software Engineering.

[16]  Marcelo d'Amorim,et al.  An Empirical Evaluation of Automated Black Box Testing Techniques for Crashing GUIs , 2009, 2009 International Conference on Software Testing Verification and Validation.

[17]  Mika Katara,et al.  Automatic GUI test generation for smartphone applications - an evaluation , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[18]  Sarfraz Khurshid,et al.  Event Listener Analysis and Symbolic Execution for Testing GUI Applications , 2009, ICFEM.

[19]  Luciano Baresi,et al.  TestFul: automatic unit-test generation for Java classes , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[20]  Eduardo Aranha,et al.  GUI Testing Techniques Evaluation by Designed Experiments , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[21]  João Saraiva,et al.  The GUISurfer tool: towards a language independent approach to reverse engineering GUI code , 2010, EICS '10.

[22]  Atif M. Memon,et al.  Iterative execution-feedback model-directed GUI testing , 2010, Inf. Softw. Technol..

[23]  Ana C. R. Paiva,et al.  Reverse engineering of GUI models for testing , 2010, 5th Iberian Conference on Information Systems and Technologies.

[24]  Gordon Fraser,et al.  EvoSuite: automatic test suite generation for object-oriented software , 2011, ESEC/FSE '11.

[25]  Andreas Zeller,et al.  Generating parameterized unit tests , 2011, ISSTA '11.

[26]  Leonardo Mariani,et al.  AutoBlackTest: a tool for automatic black-box testing , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[27]  Myra B. Cohen,et al.  GUI Interaction Testing: Incorporating Event Context , 2011, IEEE Transactions on Software Engineering.

[28]  Atif M. Memon,et al.  Developing a Single Model and Test Prioritization Strategies for Event-Driven Software , 2011, IEEE Transactions on Software Engineering.

[29]  Leonardo Mariani,et al.  AutoBlackTest: Automatic Black-Box Testing of Interactive Applications , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.