A Survey of Context Simulation for Testing Mobile Context-Aware Applications

Equipped with an abundance of small-scale microelectromechanical sensors, modern mobile devices such as smartphones and smartwatches can now offer context-aware services to users in mobile environments. Although advances in mobile context-aware applications have made our everyday environments increasingly intelligent, these applications are prone to bugs that are highly difficult to reproduce and repair. Compared to conventional computer software, mobile context-aware applications often have more complex structures to process a wide variety of dynamic context data in specific scenarios. Accordingly, researchers have proposed diverse context simulation techniques to enable low-cost and effective tests instead of conducting costly and time-consuming real-world experiments. This article aims to give a comprehensive overview of the state-of-the-art context simulation methods for testing mobile context-aware applications. In particular, this article highlights the technical distinctions and commonalities in previous research conducted across multiple disciplines, particularly at the intersection of software testing, ubiquitous computing, and mobile computing. This article also discusses how each method can be implemented and deployed by testing tool developers and mobile application testers. Finally, this article identifies several unexplored issues and directions for further advancements in this field.

[1]  Ralf Tönjes,et al.  Automated Testing of Context-Aware Applications , 2015, 2015 IEEE 82nd Vehicular Technology Conference (VTC2015-Fall).

[2]  Jun Sun,et al.  Towards Model Checking Android Applications , 2018, IEEE Transactions on Software Engineering.

[3]  Vinny Cahill,et al.  A framework for developing mobile, context-aware applications , 2004, Second IEEE Annual Conference on Pervasive Computing and Communications, 2004. Proceedings of the.

[4]  Porfirio Tramontana,et al.  Using GUI ripping for automated testing of Android applications , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[5]  Kazi Sakib,et al.  MobiCoMonkey - Context Testing of Android Apps , 2018, 2018 IEEE/ACM 5th International Conference on Mobile Software Engineering and Systems (MOBILESoft).

[6]  Yichen Wang,et al.  Model-Based Simulation Testing for Embedded Software , 2011, 2011 Third International Conference on Communications and Mobile Computing.

[7]  Rajaram Regupathy,et al.  Android Debug Bridge (ADB) , 2014 .

[8]  Junfeng Yang,et al.  Efficiently, effectively detecting mobile app bugs with AppDoctor , 2014, EuroSys '14.

[9]  Henry Muccini,et al.  Software testing of mobile applications: Challenges and future research directions , 2012, 2012 7th International Workshop on Automation of Software Test (AST).

[10]  Brink van der Merwe,et al.  Verifying android applications using Java PathFinder , 2012, ACM SIGSOFT Softw. Eng. Notes.

[11]  Enhong Chen,et al.  Systematically testing background services of mobile apps , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[12]  T. Y. Chen,et al.  Adaptive Random Testing , 2004, ASIAN.

[13]  Christopher Vendome,et al.  CrashScope: A Practical Tool for Automated Testing of Android Applications , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[14]  Ranveer Chandra,et al.  Contextual Fuzzing: Automated Mobile App Testing Under Dynamic Device and Environment Conditions , 2013 .

[15]  Mario Linares-Vasquez,et al.  Automated Extraction of Augmented Models for Android Apps , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[16]  Yongjian Hu,et al.  Fuzzy and Cross-App Replay for Smartphone Apps , 2016, 2016 IEEE/ACM 11th International Workshop in Automation of Software Test (AST).

[17]  Galen C. Hunt,et al.  Debugging in the (very) large: ten years of implementation and experience , 2009, SOSP '09.

[18]  Qun Li,et al.  MobiPlay: A Remote Execution Based Record-and-Replay Tool for Mobile Applications , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[19]  Luigi Rizzo,et al.  Dummynet: a simple approach to the evaluation of network protocols , 1997, CCRV.

[20]  Ranveer Chandra,et al.  Caiipa: automated large-scale mobile app testing through contextual fuzzing , 2014, MobiCom.

[21]  Denzil Ferreira,et al.  AWARE: Mobile Context Instrumentation Framework , 2015, Front. ICT.

[22]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[23]  Yongjian Hu,et al.  VALERA: An Effective and Efficient Record-and-Replay Tool for Android , 2016, 2016 IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft).

[24]  Wei Zhang,et al.  Towards A Contextual and Scalable Automated-testing Service for Mobile Apps , 2017, HotMobile.

[25]  Jorge Gonçalves,et al.  How to validate mobile crowdsourcing design? leveraging data integration in prototype testing , 2016, UbiComp Adjunct.

[26]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[27]  Mario Linares Vásquez,et al.  Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mobile App Testing , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[28]  Jorge Gonçalves,et al.  CamTest: A laboratory testbed for camera-based mobile sensing applications , 2019, Pervasive Mob. Comput..

[29]  Zoltán Szatmári,et al.  A Concept for Testing Robustness and Safety of the Context-Aware Behaviour of Autonomous Systems , 2012, KES-AMSTA.

[30]  Yi Qin,et al.  SIT: Sampling-based interactive testing for self-adaptive apps , 2016, J. Syst. Softw..

[31]  Wei-Tek Tsai,et al.  Generating Test Cases for Context-Aware Applications Using Bigraphs , 2014, 2014 Eighth International Conference on Software Security and Reliability.

[32]  Seungho Lim,et al.  x86‐Android performance improvement for x86 smart mobile devices , 2016, Concurr. Comput. Pract. Exp..

[33]  Christoffer Quist Adamsen,et al.  Systematic execution of Android test suites in adverse conditions , 2015, ISSTA.

[34]  John Grundy,et al.  A systematic mapping study of mobile application testing techniques , 2016, J. Syst. Softw..

[35]  Matthias Warkentin,et al.  Exchange Graphs via Quiver Mutation , 2014 .

[36]  Nigel H. Lovell,et al.  Low-power technologies for wearable telecare and telehealth systems: A review , 2015 .

[37]  Xiang Long,et al.  Adaptive random testing of mobile application , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[38]  Ivar Jacobson,et al.  The unified modeling language reference manual , 2010 .

[39]  Li Jiang,et al.  Smartphone technology can be transformative to the deployment of lab-on-chip diagnostics. , 2014, Lab on a chip.

[40]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[41]  Volker Gruhn,et al.  A model-based approach to test automation for context-aware mobile applications , 2014, SAC.

[42]  罗国昭 Wi-Fi Direct技术发布 , 2010 .

[43]  Volker Gruhn,et al.  Towards Automated UI-Tests for Sensor-Based Mobile Applications , 2015, SoMeT.

[44]  Mayur Naik,et al.  Dynodroid: an input generation system for Android apps , 2013, ESEC/FSE 2013.

[45]  Ivar Jacobson,et al.  Unified Modeling Language Reference Manual, The (2nd Edition) , 2004 .

[46]  Sehun Jeong,et al.  Generating various contexts from permissions for testing Android applications , 2015, SEKE.

[47]  Martin T. Vechev,et al.  Scalable race detection for Android applications , 2015, OOPSLA.

[48]  Ranjan Kumar,et al.  Understanding the Challenges in Mobile Computation Offloading to Cloud through Experimentation , 2015, 2015 2nd ACM International Conference on Mobile Software Engineering and Systems.

[49]  Jie Yang,et al.  Accurate WiFi Based Localization for Smartphones Using Peer Assistance , 2014, IEEE Transactions on Mobile Computing.

[50]  Grzegorz J. Nalepa,et al.  Understanding Context with ContextViewer - Tool for Visualization and Initial Preprocessing of Mobile Sensors Data , 2015, CONTEXT.

[51]  Lorenzo Torresani,et al.  CarSafe: a driver safety app that detects dangerous driving behavior using dual-cameras on smartphones , 2012, UbiComp.

[52]  Tetsuya Yoshida,et al.  Using a Virtual Machine Monitor to Slow Down CPU Speed for Embedded Time-Sensitive Software Testing , 2009 .

[53]  Wei-Tek Tsai,et al.  Mobile Application Testing: A Tutorial , 2014, Computer.

[54]  Klaus Havelund,et al.  Model checking JAVA programs using JAVA PathFinder , 2000, International Journal on Software Tools for Technology Transfer.

[55]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[56]  Porfirio Tramontana,et al.  Considering Context Events in Event-Based Testing of Mobile Applications , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[57]  Saurabh Bagchi,et al.  How Reliable is My Wearable: A Fuzz Testing-Based Study , 2018, 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[58]  Kwang-Ting Cheng,et al.  Automatic Functional Test Generation Using The Extended Finite State Machine Model , 1993, 30th ACM/IEEE Design Automation Conference.

[59]  Bo Jiang,et al.  MobileTest: A Tool Supporting Automatic Black Box Test for Software on Smart Mobile Devices , 2007, AST.

[60]  Wei Pan,et al.  SoundSense: scalable sound sensing for people-centric applications on mobile phones , 2009, MobiSys '09.

[61]  Lionel C. Briand,et al.  Testing Vision-Based Control Systems Using Learnable Evolutionary Algorithms , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[62]  Jacques Klein,et al.  Automated Testing of Android Apps: A Systematic Literature Review , 2019, IEEE Transactions on Reliability.

[63]  Pankaj Mudholkar,et al.  Software Testing , 2002, Computer.

[64]  Emiliano Miluzzo,et al.  A survey of mobile phone sensing , 2010, IEEE Communications Magazine.

[65]  Hongfei Yan,et al.  DroidForensics: Accurate Reconstruction of Android Attacks via Multi-layer Forensic Logging , 2017, AsiaCCS.

[66]  Todd D. Millstein,et al.  RERAN: Timing- and touch-sensitive record and replay for Android , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[67]  James Davis,et al.  Node.fz: Fuzzing the Server-Side Event-Driven Architecture , 2017, EuroSys.

[68]  Porfirio Tramontana,et al.  Automated functional testing of mobile applications: a systematic mapping study , 2018, Software Quality Journal.

[69]  Robin Milner,et al.  Pure bigraphs: Structure and dynamics , 2006, Inf. Comput..

[70]  Jun Wei,et al.  Generating test cases to expose concurrency bugs in android applications , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[71]  Yongjian Hu,et al.  Automatically verifying and reproducing event-based races in Android apps , 2016, ISSTA.

[72]  Antonio Ken Iannillo,et al.  Chizpurfle: A Gray-Box Android Fuzzer for Vendor Service Customizations , 2017, 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE).

[73]  David S. Rosenblum,et al.  Automated Generation of Context-Aware Tests , 2007, 29th International Conference on Software Engineering (ICSE'07).

[74]  Denys Poshyvanyk,et al.  On-Device Bug Reporting for Android Applications , 2017, 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft).

[75]  Thorsten Holz,et al.  Slicing droids: program slicing for smali code , 2013, SAC '13.

[76]  Axel Küpper,et al.  CATLES: A Crowdsensing-Supported Interactive World-Scale Environment Simulator for Context-Aware Systems , 2016, 2016 IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft).

[77]  Andres J. Ramirez,et al.  A taxonomy of uncertainty for dynamically adaptive systems , 2012, 2012 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[78]  Sergiy Vilkomir,et al.  Testing-as-a-Service for Mobile Applications: State-of-the-Art Survey , 2015 .

[79]  小林 明大,et al.  楽々!Android Studioはじめの一歩 , 2015 .

[80]  Jorge Gonçalves,et al.  TestAWARE: A Laboratory-Oriented Testing Tool for Mobile Context-Aware Applications , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[81]  Matthias Baldauf,et al.  A survey on context-aware systems , 2007, Int. J. Ad Hoc Ubiquitous Comput..

[82]  B. P. Ziegler,et al.  Theory of Modeling and Simulation , 1976 .

[83]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[84]  Alessandra Gorla,et al.  Automated Test Input Generation for Android: Are We There Yet? (E) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[85]  Tao Zhang,et al.  Testing Location-Based Function Services for Mobile Applications , 2015, 2015 IEEE Symposium on Service-Oriented System Engineering.

[86]  Ian Warren,et al.  Hermes: A Tool for Testing Mobile Device Applications , 2009, 2009 Australian Software Engineering Conference.

[87]  Rodrigo Castro,et al.  STDEVS. A NOVEL FORMALISM FOR MODELING AND SIMULATION OF STOCHASTIC DISCRETE EVENT SYSTEMS , 2006 .

[88]  Yongjian Hu,et al.  Versatile yet lightweight record-and-replay for Android , 2015, OOPSLA.

[89]  Pankaj Jalote,et al.  A Concise Introduction to Software Engineering , 2008, Undergraduate Topics in Computer Science.

[90]  Denzil Ferreira,et al.  Social-aware device-to-device communication: a contribution for edge and fog computing? , 2016, UbiComp Adjunct.

[91]  Matthias Hauswirth,et al.  Automating performance testing of interactive Java applications , 2010, AST '10.

[92]  Dieter Hogrefe,et al.  An introduction to the testing and test control notation (TTCN-3) , 2003, Comput. Networks.

[93]  Wasif Afzal,et al.  Automating Test Data Generation for Testing Context-Aware Applications , 2018, 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS).

[94]  Jian Lu,et al.  Dynamic fault detection in context-aware adaptation , 2012, Internetware.

[95]  Suman Nath,et al.  Automatic and scalable fault detection for mobile applications , 2014, MobiSys.

[96]  Sandro Rodriguez Garzon,et al.  Model-based generation of scenario-specific event sequences for the simulation of recurrent user behavior within context-aware applications (WIP) , 2012, SpringSim.

[97]  Chang Xu,et al.  Facilitating Reusable and Scalable Automated Testing and Analysis for Android Apps , 2015, Internetware.

[98]  Somesh Jha,et al.  Retargeting Android applications to Java bytecode , 2012, SIGSOFT FSE.

[99]  Tim A. Majchrzak,et al.  Context-Dependent Testing of Applications for Mobile Devices , 2015, Open J. Web Technol..

[100]  Shing-Chi Cheung,et al.  Partial constraint checking for context consistency in pervasive computing , 2010, TSEM.

[101]  Romain Rouvoy,et al.  ANDROFLEET: Testing WiFi peer-to-peer mobile apps in the large , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[102]  Herbert Bos,et al.  Paranoid Android: versatile protection for smartphones , 2010, ACSAC '10.

[103]  Noraini Ibrahim,et al.  Test Case Generation from Android Mobile Applications Focusing on Context Events , 2018, ICSCA.

[104]  Suman Nath,et al.  PUMA: programmable UI-automation for large-scale dynamic analysis of mobile apps , 2014, MobiSys.

[105]  David S. Rosenblum,et al.  Model-based fault detection in context-aware adaptive applications , 2008, SIGSOFT '08/FSE-16.