Practical GUI Testing of Android Applications Via Model Abstraction and Refinement

This paper introduces a new, fully automated modelbased approach for effective testing of Android apps. Different from existing model-based approaches that guide testing with a static GUI model (i.e., the model does not evolve its abstraction during testing, and is thus often imprecise), our approach dynamically optimizes the model by leveraging the runtime information during testing. This capability of model evolution significantly improves model precision, and thus dramatically enhances the testing effectiveness compared to existing approaches, which our evaluation confirms.We have realized our technique in a practical tool, APE. On 15 large, widely-used apps from the Google Play Store, APE outperforms the state-of-the-art Android GUI testing tools in terms of both testing coverage and the number of detected unique crashes. To further demonstrate APE’s effectiveness and usability, we conduct another evaluation of APE on 1,316 popular apps, where it found 537 unique crashes. Out of the 38 reported crashes, 13 have been fixed and 5 have been confirmed.

[1]  Mayur Naik,et al.  Dynodroid: an input generation system for Android apps , 2013, ESEC/FSE 2013.

[2]  Alessandra Gorla,et al.  Automated Test Input Generation for Android: Are We There Yet? (E) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[3]  Jian Lu,et al.  AimDroid: Activity-Insulated Multi-level Automated Testing for Android Applications , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[4]  Matthew B. Dwyer,et al.  Model checking graphical user interfaces using abstractions , 1997, ESEC '97/FSE-5.

[5]  Junfeng Yang,et al.  Efficiently, effectively detecting mobile app bugs with AppDoctor , 2014, EuroSys '14.

[6]  Yepang Liu,et al.  Taming Android fragmentation: Characterizing and detecting compatibility issues for Android apps , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[7]  Christoffer Quist Adamsen,et al.  Systematic execution of Android test suites in adverse conditions , 2015, ISSTA.

[8]  Iulian Neamtiu,et al.  Finding resume and restart errors in Android applications , 2016, OOPSLA.

[9]  Ranveer Chandra,et al.  Caiipa: automated large-scale mobile app testing through contextual fuzzing , 2014, MobiCom.

[10]  Philippe Kruchten,et al.  Real Challenges in Mobile App Development , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[11]  Iulian Neamtiu,et al.  Targeted and depth-first exploration for systematic testing of android apps , 2013, OOPSLA.

[12]  Mukul R. Prasad,et al.  Automated testing with targeted event sequence generation , 2013, ISSTA.

[13]  Ellen J. Bass,et al.  Using Formal Verification to Evaluate Human-Automation Interaction: A Review , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[14]  Yingjun Lyu,et al.  Automated Energy Optimization of HTTP Requests for Mobile Applications , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[15]  Brink van der Merwe,et al.  Execution and property specifications for JPF-android , 2014, SOEN.

[16]  Christopher Vendome,et al.  CrashScope: A Practical Tool for Automated Testing of Android Applications , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[17]  Tao Xie,et al.  A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications , 2013, FASE.

[18]  Doo-Hwan Bae,et al.  Automated model-based Android GUI testing using multi-level GUI comparison criteria , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[19]  Michael Pradel,et al.  Monkey see, monkey do: effective generation of GUI tests with inferred macro events , 2017, Software Engineering.

[20]  Alexander Aiken,et al.  Minimizing GUI event traces , 2016, SIGSOFT FSE.

[21]  Jian Lu,et al.  AATT+: Effectively manifesting concurrency bugs in Android apps , 2018, Sci. Comput. Program..

[22]  Hongseok Yang,et al.  Automated concolic testing of smartphone apps , 2012, SIGSOFT FSE.

[23]  Sam Malek,et al.  EvoDroid: segmented evolutionary testing of Android apps , 2014, SIGSOFT FSE.

[24]  Ahmed E. Hassan,et al.  Prioritizing the devices to test your app on: a case study of Android game apps , 2014, SIGSOFT FSE.

[25]  Edmund M. Clarke,et al.  Counterexample-guided abstraction refinement , 2003, 10th International Symposium on Temporal Representation and Reasoning, 2003 and Fourth International Conference on Temporal Logic. Proceedings..

[26]  Jeff Huang,et al.  EHBDroid: Beyond GUI testing for Android applications , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[27]  Porfirio Tramontana,et al.  Using GUI ripping for automated testing of Android applications , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[28]  Mark Harman,et al.  Crowd intelligence enhances automated mobile testing , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[29]  Enhong Chen,et al.  Systematically testing background services of mobile apps , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[30]  Yang Liu,et al.  Efficiently Manifesting Asynchronous Programming Errors in Android Apps , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[31]  Yang Liu,et al.  Guided, stochastic model-based GUI testing of Android apps , 2017, ESEC/SIGSOFT FSE.

[32]  John Regehr,et al.  Intent fuzzer: crafting intents of death , 2014, WODA+PERTEA 2014.

[33]  Elizabeth Schroeder Activity , 2005 .

[34]  Gabriele Bavota,et al.  Optimizing energy consumption of GUIs in Android apps: a multi-objective approach , 2015, ESEC/SIGSOFT FSE.

[35]  Yang Liu,et al.  Large-Scale Analysis of Framework-Specific Exceptions in Android Apps , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[36]  Jian Lu,et al.  GreenDroid: Automated Diagnosis of Energy Inefficiency for Smartphone Applications , 2014, IEEE Transactions on Software Engineering.

[37]  George C. Necula,et al.  Guided GUI testing of android apps with minimal restart and approximate learning , 2013, OOPSLA.

[38]  Xia Zeng,et al.  Automated test input generation for Android: are we really there yet in an industrial case? , 2016, SIGSOFT FSE.

[39]  Yue Jia,et al.  Sapienz: multi-objective automated testing for Android applications , 2016, ISSTA.

[40]  George C. Necula,et al.  DetReduce: Minimizing Android GUI Test Suites for Regression Testing , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[41]  S. Malek,et al.  PATDroid: permission-aware GUI testing of Android , 2017, ESEC/SIGSOFT FSE.

[42]  Mark Harman,et al.  Deploying Search Based Software Engineering with Sapienz at Facebook , 2018, SSBSE.