PATDroid: permission-aware GUI testing of Android

Recent introduction of a dynamic permission system in Android, allowing the users to grant and revoke permissions after the installation of an app, has made it harder to properly test apps. Since an app's behavior may change depending on the granted permissions, it needs to be tested under a wide range of permission combinations. At the state-of-the-art, in the absence of any automated tool support, a developer needs to either manually determine the interaction of tests and app permissions, or exhaustively re-execute tests for all possible permission combinations, thereby increasing the time and resources required to test apps. This paper presents an automated approach, called PATDroid, for efficiently testing an Android app while taking the impact of permissions on its behavior into account. PATDroid performs a hybrid program analysis on both an app under test and its test suite to determine which tests should be executed on what permission combinations. Our experimental results show that PATDroid significantly reduces the testing effort, yet achieves comparable code coverage and fault detection capability as exhaustively testing an app under all permission combinations.

[1]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[2]  Gregg Rothermel,et al.  Software testing: a research travelogue (2000–2014) , 2014, FOSE.

[3]  Charles J. Colbourn,et al.  Prioritized interaction testing for pair-wise coverage with seeding and constraints , 2006, Inf. Softw. Technol..

[4]  Laurie Hendren,et al.  Soot: a Java bytecode optimization framework , 2010, CASCON.

[5]  Sarfraz Khurshid,et al.  Shared Execution for Efficiently Testing Product Lines , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[6]  Iulian Neamtiu,et al.  Targeted and depth-first exploration for systematic testing of android apps , 2013, OOPSLA.

[7]  Wenke Lee,et al.  CHEX: statically vetting Android apps for component hijacking vulnerabilities , 2012, CCS.

[8]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[9]  Matthew L. Dering,et al.  Composite Constant Propagation: Application to Android Inter-Component Communication Analysis , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[10]  Zhen Huang,et al.  PScout: analyzing the Android permission specification , 2012, CCS.

[11]  Cemal Yilmaz.  Test Case-Aware Combinatorial Interaction Testing , 2013, IEEE Transactions on Software Engineering.

[12]  Myra B. Cohen,et al.  Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach , 2008, IEEE Transactions on Software Engineering.

[13]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[14]  Gregg Rothermel,et al.  A safe, efficient regression test selection technique , 1997, TSEM.

[15]  Sarfraz Khurshid,et al.  Reducing combinatorics in testing product lines , 2011, AOSD '11.

[16]  J. Czerwonka.  Pairwise Testing in the Real World : Practical Extensions to Test-Case Scenarios , 2011 .

[17]  Iulian Neamtiu,et al.  Automating GUI testing for Android applications , 2011, AST '11.

[18]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[19]  Frank Tip,et al.  Chianti: a tool for change impact analysis of java programs , 2004, OOPSLA.

[20]  Lukasz Ziarek,et al.  Information flows as a permission mechanism , 2014, ASE.

[21]  Porfirio Tramontana,et al.  Using GUI ripping for automated testing of Android applications , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[22]  Sven Apel,et al.  Toward variability-aware testing , 2012, FOSD '12.

[23]  Jacques Klein,et al.  FlowDroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps , 2014, PLDI.

[24]  Kari J. Nurmela,et al.  Upper bounds for covering arrays by tabu search , 2004, Discret. Appl. Math..

[25]  Darko Marinov,et al.  Practical regression test selection with dynamic file dependencies , 2015, ISSTA.

[26]  Sarfraz Khurshid,et al.  SPLat: lightweight dynamic analysis for reducing combinatorics in testing configurable systems , 2013, ESEC/FSE 2013.

[27]  Paul C. van Oorschot,et al.  A methodology for empirical analysis of permission-based security models and its application to android , 2010, CCS '10.

[28]  Mayur Naik,et al.  Dynodroid: an input generation system for Android apps , 2013, ESEC/FSE 2013.

[29]  Myra B. Cohen,et al.  A deterministic density algorithm for pairwise interaction coverage , 2004, IASTED Conf. on Software Engineering.

[30]  Alessandra Gorla,et al.  Automated Test Input Generation for Android: Are We There Yet? (E) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[31]  Myra B. Cohen,et al.  Moving Forward with Combinatorial Interaction Testing , 2014, Computer.

[32]  Tao Xie,et al.  A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications , 2013, FASE.

[33]  Adam A. Porter,et al.  iTree: Efficiently Discovering High-Coverage Configurations Using Interaction Trees , 2014, IEEE Transactions on Software Engineering.

[34]  Sam Malek,et al.  EvoDroid: segmented evolutionary testing of Android apps , 2014, SIGSOFT FSE.

[35]  Sarfraz Khurshid,et al.  Localizing failure-inducing program edits based on spectrum information , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[36]  Porfirio Tramontana,et al.  A GUI Crawling-Based Technique for Android Mobile Application Testing , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[37]  Erik Derr,et al.  On Demystifying the Android Application Framework: Re-Visiting Android Permission Specification Analysis , 2016, USENIX Security Symposium.

[38]  Jacques Klein,et al.  IccTA: Detecting Inter-Component Privacy Leaks in Android Apps , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[39]  Alireza Sadeghi,et al.  Reducing Combinatorics in GUI Testing of Android Applications , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).