Probabilistic Formal Analysis of App Usage to Inform Redesign

Evaluation and redesign of user-intensive mobile applications is challenging because users are often heterogeneous, adopting different patterns of activity, at different times. We set out a process of integrating statistical, longitudinal analysis of actual logged behaviours, formal, probabilistic discrete state models of activity patterns, and hypotheses over those models expressed as probabilistic temporal logic properties to inform redesign. We employ formal methods not to the design of the mobile application, but to characterise the different probabilistic patterns of actual use over various time cuts within a population of users. We define the whole process from identifying questions that give us insight into application usage, to event logging, data abstraction from logs, model inference, temporal logic property formulation, visualisation of results, and interpretation in the context of redesign. We illustrate the process through a real-life case study, which results in a new and principled way for selecting content for an extension to the mobile application.

[1]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[2]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[3]  Matthew Chalmers,et al.  Adapting ubicomp software and its evaluation , 2009, EICS '09.

[4]  Muffy Calder,et al.  Probabilistic Model Checking of DTMC Models of User Activity Patterns , 2014, QEST.

[5]  Marta Z. Kwiatkowska,et al.  Stochastic Model Checking , 2007, SFM.

[6]  Carlo Ghezzi,et al.  Mining behavior models from user-intensive web applications , 2014, ICSE.

[7]  Wilhelm Hasselbring,et al.  Live trace visualization for comprehending large software landscapes: The ExplorViz approach , 2013, 2013 First IEEE Working Conference on Software Visualization (VISSOFT).

[8]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Ata Kabán,et al.  Simplicial Mixtures of Markov Chains: Distributed Modelling of Dynamic User Profiles , 2003, NIPS.

[11]  Matthew Chalmers,et al.  Experiences in Logging Everyday App Use , 2013 .

[12]  Anneliese Amschler Andrews,et al.  Program Comprehension During Software Maintenance and Evolution , 1995, Computer.

[13]  Jon Howell,et al.  Mugshot: Deterministic Capture and Replay for JavaScript Applications , 2010, NSDI.

[14]  Matthew Chalmers,et al.  Analysing user behaviour through dynamic population models , 2013, CHI Extended Abstracts.

[15]  Christel Baier,et al.  Principles of model checking , 2008 .

[16]  Yuriy Brun,et al.  Inferring models of concurrent systems from logs of their behavior with CSight , 2014, ICSE.

[17]  Andrew Begel,et al.  Analyze this! 145 questions for data scientists in software engineering , 2013, ICSE.

[18]  Todd D. Millstein,et al.  RERAN: Timing- and touch-sensitive record and replay for Android , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[19]  Ezio Bartocci,et al.  Runtime Verification with State Estimation , 2011, RV.