Automating Mockup-Based Usability Testing on the Mobile Device

User interface usability is a very relevant aspect, especially for mobile applications. In this paper we propose a methodology for automating user interface usability testing in the early phases of the development process directly on the mobile device. Mockup interfaces are designed, then the way the designer interacts with the interface is compared with the way the end-user interacts with the same interface and the discrepancies between the two usage models are highlighted. The methodology is supported by a mobile tool named PlatoS, which captures a case of use of a mockup in terms of a sequence of actions. On the end-user side, PlatoS captures the user interaction with the mockups and performs the automatic identification of usability problems on the base of the log data. An example of evaluation of an eCommerce mobile prototypes is also presented.

[1]  Giuseppe Scanniello,et al.  On the Use of Requirements Measures to Predict Software Project and Product Measures in the Context of Android Mobile Apps: A Preliminary Study , 2015, 2015 41st Euromicro Conference on Software Engineering and Advanced Applications.

[2]  Forrest Shull,et al.  Detecting defects in object-oriented designs: using reading techniques to increase software quality , 1999, OOPSLA '99.

[3]  Wan Fatimah Wan Ahmad,et al.  Handling tradeoffs between agile and usability methods , 2014, 2014 International Conference on Computer and Information Sciences (ICCOINS).

[4]  Jean Scholtz,et al.  Common industry format for usability test reports , 2000, CHI Extended Abstracts.

[5]  Venkataraman Ramesh,et al.  Understanding usability in mobile commerce , 2003, CACM.

[6]  Peter Peters,et al.  Klikker: A Method and Infrastructure for Mining, Analysis, and Visualisation of User Behaviour and Usability Issues for Mobile Application Development. , 2018 .

[7]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[8]  Marco Pistoia,et al.  Improving Design Validation of Mobile Application User Interface Implementation , 2016, 2016 IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft).

[9]  Clemens Holzmann,et al.  Automated and unsupervised user interaction logging as basis for usability evaluation of mobile applications , 2012, MoMM '12.

[10]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[11]  Gustavo Rossi,et al.  Mockup-Driven Development: Providing agile support for Model-Driven Web Engineering , 2014, Inf. Softw. Technol..

[12]  Tayana Conte,et al.  Using a Controlled Experiment to Evaluate Usability Inspection Technologies for Improving the Quality of Mobile Web Applications Earlier in their Design , 2014, 2014 Brazilian Symposium on Software Engineering.

[13]  Giuseppe Scanniello,et al.  Mobile App Development and Management: Results from a Qualitative Investigation , 2017, 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft).

[14]  Marco Torchiano,et al.  On the effort of augmenting use cases with screen mockups: results from a preliminary empirical study , 2010, ESEM '10.

[15]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[16]  Jakob Nielsen.  Usability metrics: tracking interface improvements , 1996 .

[17]  Cathleen Wharton,et al.  Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces , 1992, Int. J. Man Mach. Stud..

[18]  Ryan West,et al.  Automated summative usability studies: an empirical evaluation , 2006, CHI.

[19]  R. Rosenthal.  Meta-analytic procedures for social research , 1984 .

[20]  Anu Kankainen,et al.  Usability testing of mobile applications: a comparison between laboratory and field testing , 2005 .

[21]  Saad Masood Butt,et al.  Incorporation of usability evaluation methods in agile software model , 2014, 17th IEEE International Multi Topic Conference 2014.

[22]  Filippo Ricca,et al.  A Method for Requirements Capture and Specification Based on Disciplined Use Cases and Screen Mockups , 2015, PROFES.

[23]  Boyang Li,et al.  Automated Reporting of GUI Design Violations for Mobile Apps , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[24]  Marti A. Hearst,et al.  The State of the Art in Automated Usability Evaluation of User , 2000 .