The first decade of GUI ripping: Extensions, applications, and broader impacts

This paper provides a retrospective examination of GUI Ripping - reverse engineering a workflow model of the graphical user interface of a software application - born a decade ago out of recognition of the severe need for improving the then largely manual state-of-the-practice of functional GUI testing. In these last 10 years, GUI ripping has turned out to be an enabler for much research, both within our group at Maryland and other groups. Researchers have found new and unique applications of GUI ripping, ranging from measuring human performance to re-engineering legacy user interfaces. GUI ripping has also enabled large-scale experimentation involving millions of test cases, thereby helping to understand the nature of GUI faults and characteristics of test cases to detect them. It has resulted in large multi-institutional Government-sponsored research projects on test automation and benchmarking. GUI ripping tools have been ported to many platforms, including Java AWT and Swing, iOS, Android, UNO, Microsoft Windows, and web. In essence, the technology has transformed the way researchers and practitioners think about the nature of GUI testing, no longer considered a manual activity; rather, thanks largely to GUI Ripping, automation has become the primary focus of current GUI testing techniques.

[1]  Andreas Podelski,et al.  Lightweight Static Analysis for GUI Testing , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[2]  Myra B. Cohen,et al.  Easing the generation of predictive human performance models from legacy systems , 2012, CHI.

[3]  Atif M. Memon,et al.  Relationships between Test Suites, Faults, and Fault Detection in GUI Testing , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[4]  Andreas Zeller,et al.  Automatically Generating Test Cases for Specification Mining , 2012, IEEE Transactions on Software Engineering.

[5]  Mika Katara,et al.  Model-Based Testing with a General Purpose Keyword-Driven Test Automation Framework , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[6]  José Creissac Campos,et al.  Model-based User Interface Testing With Spec Explorer and ConcurTaskTrees , 2008, Electron. Notes Theor. Comput. Sci..

[7]  Iulian Neamtiu,et al.  Targeted and depth-first exploration for systematic testing of android apps , 2013, OOPSLA.

[8]  Atif M. Memon,et al.  Accounting for defect characteristics in evaluations of testing techniques , 2012, TSEM.

[9]  Hongseok Yang,et al.  Automated concolic testing of smartphone apps , 2012, SIGSOFT FSE.

[10]  Atif M. Memon,et al.  GUI ripping: reverse engineering of graphical user interfaces for testing , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..

[11]  Myra B. Cohen,et al.  Human performance regression testing , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[12]  Sarfraz Khurshid,et al.  Event Listener Analysis and Symbolic Execution for Testing GUI Applications , 2009, ICFEM.

[13]  Michael D. Ernst,et al.  Automated diagnosis of software configuration errors , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[14]  Atif M. Memon,et al.  GUITAR: an innovative tool for automated testing of GUI-driven software , 2014, Automated Software Engineering.

[15]  Leonardo Mariani,et al.  AutoBlackTest: a tool for automatic black-box testing , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[16]  Stefan Staiger,et al.  Reverse Engineering of Graphical User Interfaces Using Static Analyses , 2007, 14th Working Conference on Reverse Engineering (WCRE 2007).

[17]  Nikolai Tillmann,et al.  A Model-to-Implementation Mapping Tool for Automated Model-Based GUI Testing , 2005, ICFEM.

[18]  Fevzi Belli,et al.  Event-Based GUI Testing and Reliability Assessment Techniques -- An Experimental Insight and Preliminary Results , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[19]  Porfirio Tramontana,et al.  Experimenting a reverse engineering technique for modelling the behaviour of rich internet applications , 2009, 2009 IEEE International Conference on Software Maintenance.

[20]  Arie van Deursen,et al.  Invariant-Based Automatic Testing of Modern Web Applications , 2012, IEEE Transactions on Software Engineering.

[21]  Atif M. Memon,et al.  Call-Stack Coverage for GUI Test Suite Reduction , 2008, IEEE Transactions on Software Engineering.

[22]  Atif M. Memon,et al.  Designing and comparing automated test oracles for GUI-based software applications , 2007, TSEM.

[23]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[24]  Tao Xie,et al.  A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications , 2013, FASE.

[25]  Ali Mesbah,et al.  Automated cross-browser compatibility testing , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[26]  Atif M. Memon,et al.  Automated gui testing guided by usage profiles , 2007, ASE.

[27]  Myra B. Cohen,et al.  GUI Interaction Testing: Incorporating Event Context , 2011, IEEE Transactions on Software Engineering.

[28]  Fadi A. Zaraket,et al.  GUICOP: Specification-Based GUI Testing , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[29]  Ana C. R. Paiva,et al.  Reverse Engineered Formal Models for GUI Testing , 2007, FMICS.

[30]  Porfirio Tramontana,et al.  Reverse Engineering Finite State Machines from Rich Internet Applications , 2008, 2008 15th Working Conference on Reverse Engineering.

[31]  Atif M. Memon,et al.  Test case generator for GUITAR , 2008, ICSE Companion '08.

[32]  Donald Kossmann,et al.  AJAX Crawl: Making AJAX Applications Searchable , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[33]  Myra B. Cohen,et al.  Towards Dynamic Adaptive Automated Test Generation for Graphical User Interfaces , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[34]  Mateusz Ujma,et al.  JPF-AWT: Model checking GUI applications , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[35]  Porfirio Tramontana,et al.  Using GUI ripping for automated testing of Android applications , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[36]  Atif M. Memon,et al.  Generating Event Sequence-Based Test Cases Using GUI Runtime State Feedback , 2010, IEEE Transactions on Software Engineering.

[37]  Myra B. Cohen,et al.  Repairing GUI Test Suites Using a Genetic Algorithm , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[38]  Atif M. Memon,et al.  Rapid "crash testing" for continuously evolving GUI-based software applications , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[39]  Atif M. Memon,et al.  Using Methods & Measures from Network Analysis for GUI Testing , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[40]  Porfirio Tramontana,et al.  A GUI Crawling-Based Technique for Android Mobile Application Testing , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[41]  Rob Miller,et al.  GUI testing using computer vision , 2010, CHI.

[42]  Xun Yuan,et al.  Alternating GUI Test Generation and Execution , 2008, Testing: Academic & Industrial Conference - Practice and Research Techniques (taic part 2008).

[43]  Atif M. Memon,et al.  Using a pilot study to derive a GUI model for automated testing , 2008, TSEM.

[44]  Myra B. Cohen,et al.  Automated testing of GUI applications: Models, tools, and controlling flakiness , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[45]  Jesús Sánchez Cuadrado,et al.  Model-driven reverse engineering of legacy graphical user interfaces , 2010, ASE.

[46]  Atif M. Memon,et al.  Automating regression testing for evolving GUI software , 2005, J. Softw. Maintenance Res. Pract..

[47]  Myra B. Cohen,et al.  AutoInSpec: Using Missing Test Coverage to Improve Specifications in GUIs , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[48]  Ali Mesbah,et al.  Reverse Engineering iOS Mobile Applications , 2012, 2012 19th Working Conference on Reverse Engineering.

[49]  Mary Lou Soffa,et al.  A comprehensive framework for testing graphical user interfaces , 2001 .

[50]  Atif M. Memon,et al.  Model-Based Testing of Community-Driven Open-Source GUI Applications , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[51]  Lu Lu,et al.  Apply ant colony to event-flow model for graphical user interface test case generation , 2012, IET Softw..

[52]  Atif M. Memon,et al.  Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software , 2005, IEEE Transactions on Software Engineering.

[53]  Chin-Yu Huang,et al.  Design and analysis of GUI test-case prioritization using weight-based methods , 2010, J. Syst. Softw..

[54]  Michael D. Ernst,et al.  Combined static and dynamic automated test generation , 2011, ISSTA '11.

[55]  Atif M. Memon,et al.  Fault Detection Probability Analysis for Coverage-Based Test Suite Reduction , 2007, 2007 IEEE International Conference on Software Maintenance.

[56]  Andreas Zeller,et al.  Search-based system testing: high coverage, no false alarms , 2012, ISSTA 2012.

[57]  Andreas Zeller,et al.  EXSYST: Search-based GUI testing , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[58]  Vahid Garousi,et al.  Graphical user interface (GUI) testing: Systematic mapping and repository , 2013, Inf. Softw. Technol..

[59]  David S. Rosenblum,et al.  Automated Generation of Context-Aware Tests , 2007, 29th International Conference on Software Engineering (ICSE'07).

[60]  Atif M. Memon,et al.  Iterative execution-feedback model-directed GUI testing , 2010, Inf. Softw. Technol..

[61]  Siddhartha R. Dalal,et al.  Model-based testing in practice , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[62]  Myra B. Cohen,et al.  Covering array sampling of input event sequences for automated gui testing , 2007, ASE.

[63]  Lee J. White,et al.  Generating test cases for GUI responsibilities using complete interaction sequences , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[64]  Arie van Deursen,et al.  Crawling Ajax-Based Web Applications through Dynamic Analysis of User Interface State Changes , 2012, TWEB.

[65]  Atif M. Memon.  GUI Testing: Pitfalls and Process , 2002, Computer.

[66]  Tao Xie,et al.  Helping users avoid bugs in GUI applications , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[67]  Martin Schäf,et al.  Grey-box GUI Testing: Efficient Generation of Event Sequences , 2012, ArXiv.

[68]  David S. Rosenblum,et al.  Context-Aware Adaptive Applications: Fault Patterns and Their Automated Identification , 2010, IEEE Transactions on Software Engineering.

[69]  Atif M. Memon,et al.  Automatically repairing event sequence-based GUI test suites for regression testing , 2008, TSEM.

[70]  Atif M. Memon,et al.  An Extensible Heuristic-Based Framework for GUI Test Case Maintenance , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[71]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[72]  Atif M. Memon,et al.  Introducing a test suite similarity metric for event sequence-based test cases , 2009, 2009 IEEE International Conference on Software Maintenance.

[73]  Atif M. Memon,et al.  Studying the Characteristics of a "Good" GUI Test Suite , 2006, 2006 17th International Symposium on Software Reliability Engineering.