Extending Mobile App Analytics for Usability Test Logging

Mobile application development is characterized by reduced development cycles and high time-to-market pressure. Usability evaluation in mobile applications calls for the application of cost-effective methods, specially adapted to such constraints. We propose extending the Google Analytics for Mobile Applications basic service to store specific low-level user actions of interest for usability evaluation purposes. The solution can serve both for lab usability testing, automating quantitative data gathering, and for logging real use after application release. It is based on identification of relevant user tasks and the detailed events worth gathering, instrumentation of specific code for data gathering, and subsequent data extraction for calculating relevant usability–related variables. We validated our application in a real usability test by comparing the automatically gathered data with the information gathered by the human observer. Results shows both measurements are statistically exchangeable, opening promising new ways to perform usability testing cost-effectively and at greater scale.

[1]  Karl-Heinz Krempels,et al.  Automated Usability Testing for Mobile Applications , 2014, WEBIST.

[2]  Constantinos K. Coursaris,et al.  A Meta-Analytical Review of Empirical Mobile Usability Studies , 2011 .

[3]  Melody Y. Ivory,et al.  Automated web site evaluation - researchers and practitioners perspectives , 2010, Human-computer interaction series.

[4]  Marc Rettig,et al.  Prototyping for tiny fingers , 1994, CACM.

[5]  Van Long Johnny To,et al.  Developing Real Time Tracking of User Behavior, with Google Analytics for Mobile Phone Devices , 2015 .

[6]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[7]  Elisabeth André,et al.  EVAL - an evaluation component for mobile interfaces , 2008, Mobile HCI.

[8]  Natalia Juristo Juzgado,et al.  Usability basics for software developers , 2001, IEEE Software.

[9]  Jackson Feijó Filho,et al.  Automated Usability Tests for Mobile Devices through Live Emotions Logging , 2015, MobileHCI Adjunct.

[10]  Thomas Tiedtke,et al.  AWUSA – A Tool for Automated Website Usability Analysis , 2002 .

[11]  Vagner Figuerêdo de Santana,et al.  Support for remote usability evaluation of web mobile applications , 2011, SIGDOC '11.

[12]  Adrian Mendoza,et al.  Mobile User Experience: Patterns to Make Sense of it All , 2013 .

[13]  Natalia Juristo Juzgado,et al.  Improving Software Engineering Practice with HCI Aspects , 2003, SERA.

[14]  Marti A. Hearst,et al.  The state of the art in automating usability evaluation of user interfaces , 2001, CSUR.

[15]  Clemens Holzmann,et al.  Automated and unsupervised user interaction logging as basis for usability evaluation of mobile applications , 2012, MoMM '12.

[16]  Fabio Paternò,et al.  Remote Evaluation of Mobile Applications , 2007, TAMODIA.

[17]  Xavier Ferré,et al.  A Case Study on Cross-Platform Development Frameworks for Mobile Applications and UX , 2014, Interacción '14.

[18]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[19]  Ivar Jacobson,et al.  Use-Case 2.0 , 2016, ACM Queue.

[20]  Stefanie Seiler.  Software For Use A Practical Guide To The Models And Methods Of Usage Centered Design , 2016 .

[21]  Bracha Shapira,et al.  MATE: a mobile analysis tool for usability experts , 2013, CHI Extended Abstracts.

[22]  Heinrich Hussmann,et al.  A Methodology and Framework to Simplify Usability Analysis of Mobile Applications , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[23]  Philippe Kruchten,et al.  Real Challenges in Mobile App Development , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[24]  Debora Shaw,et al.  Handbook of usability testing: How to plan, design, and conduct effective tests , 1996 .

[25]  Swati V. Chande,et al.  A Review and Analysis on Mobile Application Development Processes using Agile Methodologies , 2013 .

[26]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[27]  Jeffrey Rubin,et al.  Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests , 1994 .