Change-Based Test Script Maintenance for Android Apps

In regression GUI testing for Android apps, test scripts often fail due to changes to, rather than faults in, those apps. To avoid such false positives while still retaining the value of the old test scripts as much as possible, programmers need an automatic way to maintain the tests after the corresponding GUI has evolved. In this paper, we propose the CHATEM approach to automate GUI test script maintenance for Android apps. Taking as input the models for the GUIs of the base and updated version app and the original test scripts, CHATEM automatically extracts the changes between the two GUIs and generates maintenance actions for each change, which are then combined to form the maintenance actions for affected test scripts. In an experimental evaluation on 16 Android apps, CHATEM was able to automatically maintain the test scripts so that overall more than 95% of the remaining behaviors tested before are still tested, and almost 80% of the reusable test actions are retained in the result tests.

[1]  Gregory M. Kapfhammer,et al.  Test suite reduction methods that decrease regression testing costs by identifying irreplaceable tests , 2014, Inf. Softw. Technol..

[2]  Arun Prakash Agarwal,et al.  An automated tool for regression testing in web applications , 2013, SOEN.

[3]  Porfirio Tramontana,et al.  A GUI Crawling-Based Technique for Android Mobile Application Testing , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[4]  Xiaolin Wang,et al.  History-based dynamic test case prioritization for requirement properties in regression testing , 2016, CSED@ICSE.

[5]  Wei-Tek Tsai,et al.  Mobile Application Testing: A Tutorial , 2014, Computer.

[6]  Atif M. Memon,et al.  Automatically repairing event sequence-based GUI test suites for regression testing , 2008, TSEM.

[7]  Shingo Takada,et al.  GUI and usage model-based test case generation for Android applications with change analysis , 2016, Mobile!@SPLASH.

[8]  Alessandra Gorla,et al.  Automated Test Input Generation for Android: Are We There Yet? (E) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[9]  Yan Wang,et al.  Static Window Transition Graphs for Android (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[10]  Xiao Li,et al.  ATOM: Automatic Maintenance of GUI Test Scripts for Evolving Mobile Applications , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[11]  Andreas Zeller,et al.  EXSYST: Search-based GUI testing , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[12]  Guowei Yang,et al.  Regression Test Selection for Android Applications , 2016, 2016 IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft).

[13]  Atif M. Memon,et al.  GUI ripping: reverse engineering of graphical user interfaces for testing , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..

[14]  Zhenyu Chen,et al.  SITAR: GUI Test Script Repair , 2016, IEEE Transactions on Software Engineering.

[15]  Alessandro Orso,et al.  WATER: Web Application TEst Repair , 2011, ETSE '11.

[16]  Glen D. Rodriguez Rafael,et al.  15 Years of Software Regression Testing Techniques - A Survey , 2016, Int. J. Softw. Eng. Knowl. Eng..

[17]  Eleni Stroulia,et al.  UMLDiff: an algorithm for object-oriented design differencing , 2005, ASE.

[18]  Jian Lu,et al.  Testing Android Apps via Guided Gesture Event Generation , 2016, 2016 23rd Asia-Pacific Software Engineering Conference (APSEC).

[19]  Iulian Neamtiu,et al.  Automating GUI testing for Android applications , 2011, AST '11.

[20]  Alessandro Orso,et al.  MINTS: A general framework and tool for supporting test-suite minimization , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[21]  Chen Fu,et al.  Maintaining and evolving GUI-directed test scripts , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[22]  Yan Wang,et al.  Static window transition graphs for Android , 2018, Automated Software Engineering.