Understanding Ineffective Events and Reducing Test Sequences for Android Applications

Monkey, which is integrated with the Android system, becomes the most widely used test input generation tool, owing to the simplicity, effectiveness and good compatibility. However, Monkey is based on coordinates of screen and oblivious to the widgets and the GUI states, which results in a great many ineffective events that have no contribution to the test. To address the major drawbacks, this paper parses the events of 200 test sequences generated by Monkey into human-readable scripts and manually investigate the effects of these events. We find three types of patterns on the ineffective events, including no-ops, single and combination of effect-free ones, and summarize them into ten rules for sequence reduction. Then, we implement a tool CHARD to match these patterns in real-world traces and prune the redundant events. The evaluation on 923 traces from various apps covering 16 categories shows that CHARD can process 1,000 events in a few seconds and identifies 41.3% events as ineffective ones. Meanwhile, the reduced sequence keeps the same functionality with the original one that can trigger the same behaviors. Our work can be applied to lessen the diagnose effort for record-and-replay, and as a preprocessing step for other works on analyzing sequences. For instance, CHARD can remove 72.6% ineffective events and saves 67.6% time of delta debugging in our experiments.

[1]  Junfeng Yang,et al.  Efficiently, effectively detecting mobile app bugs with AppDoctor , 2014, EuroSys '14.

[2]  Xiangyu Zhang,et al.  Dynamic slicing long running programs through execution fast forwarding , 2006, SIGSOFT '06/FSE-14.

[3]  Xiangyu Zhang,et al.  Precise dynamic slicing algorithms , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[4]  Jie Wang,et al.  Fast reproducing web application errors , 2015, 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE).

[5]  Alessandra Gorla,et al.  Automated Test Input Generation for Android: Are We There Yet? (E) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[6]  Gregg Rothermel,et al.  On the use of delta debugging to reduce recordings and facilitate debugging of web applications , 2015, ESEC/SIGSOFT FSE.

[7]  Bo Jiang,et al.  SimplyDroid: Efficient event sequence simplification for android application , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[8]  Alexander Aiken,et al.  Minimizing GUI event traces , 2016, SIGSOFT FSE.

[9]  Vangalur S. Alagar,et al.  Specification of Software Systems , 1998, Graduate Texts in Computer Science.

[10]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[11]  Mark Weiser,et al.  Programmers use slices when debugging , 1982, CACM.

[12]  Mayur Naik,et al.  Dynodroid: an input generation system for Android apps , 2013, ESEC/FSE 2013.

[13]  Lei Huang,et al.  PUMA: An Improved Realization of MODE for DOA Estimation , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[14]  Xi Deng,et al.  InsDal: A safe and extensible instrumentation tool on Dalvik byte-code for Android applications , 2017, 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[15]  Michael Sipser,et al.  Introduction to the Theory of Computation , 1996, SIGA.

[16]  Sam Malek,et al.  EvoDroid: segmented evolutionary testing of Android apps , 2014, SIGSOFT FSE.

[17]  Yue Jia,et al.  Sapienz: multi-objective automated testing for Android applications , 2016, ISSTA.

[18]  Christopher Vendome,et al.  Automatically Discovering, Reporting and Reproducing Android Application Crashes , 2016, 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[19]  Tibor Gyimóthy,et al.  An efficient relevant slicing method for debugging , 1999, ESEC/FSE-7.