Empirical Studies on Usability of mHealth Apps: A Systematic Literature Review

The release of smartphones and tablets, which offer more advanced communication and computing capabilities, has led to the strong emergence of mHealth on the market. mHealth systems are being used to improve patients’ lives and their health, in addition to facilitating communication between doctors and patients. Researchers are now proposing mHealth applications for many health conditions such as dementia, autism, dysarthria, Parkinson’s disease, and so on. Usability becomes a key factor in the adoption of these applications, which are often used by people who have problems when using mobile devices and who have a limited experience of technology. The aim of this paper is to investigate the empirical usability evaluation processes described in a total of 22 selected studies related to mHealth applications by means of a Systematic Literature Review. Our results show that the empirical evaluation methods employed as regards usability could be improved by the adoption of automated mechanisms. The evaluation processes should also be revised to combine more than one method. This paper will help researchers and developers to create more usable applications. Our study demonstrates the importance of adapting health applications to users’ need.

[1]  Catarina Silva,et al.  Profile-based system for nutritional information management , 2013, 2013 IEEE 15th International Conference on e-Health Networking, Applications and Services (Healthcom 2013).

[2]  Mike Y. Chen,et al.  iCAN: A tablet-based pedagogical system for improving the user experience of children with autism in the learning process , 2013, 2013 1st International Conference on Orange Technologies (ICOT).

[3]  Linda Little,et al.  E-health , 2008, BCS HCI.

[4]  Duncan Rowland,et al.  Validating a mobile phone application for the everyday, unobtrusive, objective measurement of sleep , 2013, CHI.

[5]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[6]  Sri Hastuti Kurniawan Mobile phone design for older persons , 2007, INTR.

[7]  Arif Raza,et al.  Usability issues for smartphone users with special needs — Autism , 2013, 2013 International Conference on Open Source Systems and Technologies.

[8]  Desmond Lobo,et al.  Web Usability Guidelines For Smartphones: A Synergic Approach , 2011 .

[9]  Borja Martínez-Pérez,et al.  Mobile Health Applications for the Most Prevalent Conditions by the World Health Organization: Review and Analysis , 2013, Journal of medical Internet research.

[10]  Arthur D. Fisk,et al.  Designing for Older Adults: Principles and Creative Human Factors Approaches , 2004 .

[11]  Ted Boren,et al.  Thinking aloud: reconciling theory and practice , 2000 .

[12]  Ali Idri,et al.  Requirements engineering education: a systematic mapping study , 2013, Requirements Engineering.

[13]  Gregory D. Abowd,et al.  Human-Computer Interaction (3rd Edition) , 2003 .

[14]  Anne Collins McLaughlin,et al.  Designing Displays for Older Adults , 2010 .

[15]  Anne R. Haake,et al.  Here, there, anywhere: remote usability testing that works , 2004, CITC5 '04.

[16]  Roman Durikovic,et al.  Automated Usability Measurement of Arbitrary Desktop Application with Eyetracking , 2012, 2012 16th International Conference on Information Visualisation.

[17]  Andreas Holzinger,et al.  Design and development of a mobile computer application to reengineer workflows in the hospital and the methodology to evaluate its effectiveness , 2011, J. Biomed. Informatics.

[18]  Robert L. Robinson,et al.  Tablet Computer Use by Medical Students in the United States , 2012, Journal of Medical Systems.

[19]  Norman M. Sadeh,et al.  Reconciling mobile app privacy and usability on smartphones: could user privacy profiles help? , 2014, WWW.

[20]  Alain Abran,et al.  Usability Meanings and Interpretations in ISO Standards , 2003, Software Quality Journal.

[21]  George Demiris,et al.  Older adults and mobile phones for health: A review , 2013, J. Biomed. Informatics.

[22]  Marti A. Hearst,et al.  The state of the art in automating usability evaluation of user interfaces , 2001, CSUR.

[23]  Paula Alexandra Silva,et al.  A study of novice older adults and gestural interaction on smartphones , 2013 .

[24]  Rogier Barendse,et al.  Portable platform independent patient monitoring , 2013, Computing in Cardiology 2013.

[25]  Rock Leung Improving the learnability of mobile device applications for older adults , 2009, CHI Extended Abstracts.

[26]  Oliver Amft,et al.  USABILITY OF DIGITAL MEDIA IN PATIENTS WITH COPD: A PILOT STUDY , 2012, International Journal of Technology Assessment in Health Care.

[27]  Misha Pavel,et al.  Mobile Health: Revolutionizing Healthcare Through Transdisciplinary Research , 2013, Computer.

[28]  Tonya L. Smith-Jackson,et al.  Reliability and validity of the mobile phone usability questionnaire (MPUQ) , 2006 .

[29]  Jorge Ribeiro,et al.  Design and Evaluation of a Mobile User Interface for Older Adults: Navigation, Interaction and Visual Design Recommendations , 2013, DSAI.

[30]  Bo-Christer Björk,et al.  The publishing delay in scholarly peer-reviewed journals , 2013, J. Informetrics.

[31]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[32]  Emilio Insfrán,et al.  A Systematic Review of Usability Evaluation in Web Development , 2008, WISE Workshops.

[33]  Panos Markopoulos,et al.  On the assessment of usability testing methods for children , 2003, Interact. Comput..

[34]  Abdelkader H. Ouda,et al.  A Conceptual Framework for Measuring the Quality Aspect of Mobile Learning , 2012 .

[35]  K. Kirk Shung,et al.  Protection Circuits for Very High Frequency Ultrasound Systems , 2014, Journal of Medical Systems.

[36]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[37]  Mexhid Ferati,et al.  Usability evaluation of acoustic interfaces for the blind , 2011, SIGDOC '11.

[38]  Arun Rai,et al.  Understanding Determinants of Consumer Mobile Health Usage Intentions, Assimilation, and Channel Preferences , 2013, Journal of medical Internet research.

[39]  Greg Nudelman Android Design Patterns: Interaction Design Solutions for Developers , 2013 .

[40]  Jeongyun Heo,et al.  A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors , 2009, Interact. Comput..

[41]  Eija Kaasinen,et al.  What influences users' decisions to take apps into use?: a framework for evaluating persuasive and engaging design in mobile Apps for well-being , 2012, MUM.

[42]  Gondy Leroy,et al.  A Smart-Phone Application and a Companion Website for the Improvement of the Communication Skills of Children with Autism: Clinical Rationale, Technical Development and Preliminary Results , 2011, Journal of Medical Systems.

[43]  W. Nilsen,et al.  Health behavior models in the age of mobile interventions: are our theories up to the task? , 2011, Translational behavioral medicine.

[44]  Fabio Paternò,et al.  Remote Usability Evaluation of Mobile Web Applications , 2013, HCI.

[45]  Ali Idri,et al.  Mobile PHRs Compliance with Android and iOS Usability Guidelines , 2014, Journal of Medical Systems.

[46]  Kasper Hornbæk,et al.  Current practice in measuring usability: Challenges to usability studies and research , 2006, Int. J. Hum. Comput. Stud..

[47]  Adesina Simon Sodiya,et al.  A Mobile Phone-Based Antenatal Care Support System , 2011, 2011 21st International Conference on Systems Engineering.

[48]  Trine M. Seeberg,et al.  ESUMS: A mobile system for continuous home monitoring of rehabilitation patients , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[49]  Kevin W. Bowyer,et al.  Mentoring Advice on "Conferences Versus Journals" for CSE Faculty , 2012 .

[50]  Jesús Favela,et al.  Usability assessment of a pervasive system to assist caregivers in dealing with repetitive behaviors of patients with dementia , 2011, PETRA '11.

[51]  Fleur Fritz,et al.  Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency , 2012, Int. J. Medical Informatics.

[52]  Alain Abran,et al.  Consolidating the ISO usability models , 2003 .

[53]  Jon A. Sanford,et al.  Mobile Application Concept Development for Remote Patient Monitoring , 2013, 2013 IEEE International Conference on Healthcare Informatics.

[54]  J. M. Christian Bastien,et al.  Usability testing: a review of some methodological and technical aspects of the method , 2010, Int. J. Medical Informatics.

[55]  Hiroyuki Miki,et al.  The Urgent Communication System for Deaf and Language Dysfunction People , 2013, HCI.

[56]  Edward Cutrell,et al.  "Yours is better!": participant response bias in HCI , 2012, CHI.

[57]  C. Yamagata,et al.  Mobile app development and usability research to help dementia and Alzheimer patients , 2013, 2013 IEEE Long Island Systems, Applications and Technology Conference (LISAT).

[58]  José Luis Fernández Alemán,et al.  Security and privacy in electronic health records: A systematic literature review , 2013, J. Biomed. Informatics.

[59]  Giovanni Toffetti Carughi,et al.  Web Usability: Principles and Evaluation Methods , 2006, Web Engineering.

[60]  William J. Norton Part III , 1916, Transactions of the American Institute of Electrical Engineers.

[61]  Petri Pulli,et al.  "Canderoid": A mobile system to remotely monitor travelling status of the elderly with dementia , 2013, 2013 International Joint Conference on Awareness Science and Technology & Ubi-Media Computing (iCAST 2013 & UMEDIA 2013).

[62]  Sanjay Kumar Dubey,et al.  Analytical Comparison of Usability Measurement Methods , 2012 .

[63]  Sarah Waterson,et al.  In the lab and out in the wild: remote web usability testing for mobile devices , 2002, CHI Extended Abstracts.

[64]  Joel J. P. C. Rodrigues,et al.  Mobile-health: A review of current state in 2015 , 2015, J. Biomed. Informatics.

[65]  Marta Rauch Mobile documentation: Usability guidelines, and considerations for providing documentation on Kindle, tablets, and smartphones , 2011, 2011 IEEE International Professional Communication Conference.

[66]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[67]  Sandra I. Woolley,et al.  A user-centered mobile health device to manage life-threatening anaphylactic allergies and provide support in allergic reactions , 2009, 2009 9th International Conference on Information Technology and Applications in Biomedicine.

[68]  Sirpa Riihiaho,et al.  Experiences with Usability Evaluation Methods , 2000 .

[69]  Rozilawati Razali,et al.  Usability Dimensions for Mobile Applications-A Review , 2013 .

[70]  Charmian Reynoldson,et al.  Assessing the quality and usability of smartphone apps for pain self-management. , 2014, Pain medicine.

[71]  Peng Dong,et al.  Publication lag in biomedical journals varies due to the periodical's publishing model , 2006, Scientometrics.

[72]  Gitte Lindgaard,et al.  Attention web designers: You have 50 milliseconds to make a good first impression! , 2006, Behav. Inf. Technol..

[73]  P. Schulz,et al.  Mapping mHealth Research: A Decade of Evolution , 2013, Journal of medical Internet research.

[74]  Constantinos K. Coursaris,et al.  A Meta-Analytical Review of Empirical Mobile Usability Studies , 2011 .

[75]  Onur Cinar,et al.  Android Best Practices , 2014, Apress.

[76]  Janusz Sobecki,et al.  Using Eyetracking in a Mobile Applications Usability Testing , 2012, ACIIDS.

[77]  Silvia Mara Abrahão,et al.  Usability evaluation methods for the web: A systematic mapping study , 2011, Inf. Softw. Technol..

[78]  L. Witte,et al.  Expectations of Middle-aged and Elderly Persons Towards Using Telecare Technologies and eHealth Applications in Primary Care , 2014, eTELEMED 2014.

[79]  Jan Gulliksen,et al.  Key principles for user-centred systems design , 2003, Behav. Inf. Technol..

[80]  Vlado Glavinic,et al.  Need for Usability and Wish for Mobility: Case Study of Client End Applications for Primary Healthcare Providers in Croatia , 2011, USAB.

[81]  Heather Sayers,et al.  Investigating the usability of PDAs with ageing users , 2007 .

[82]  Peter Leijdekkers,et al.  Feasibility trial of a novel mobile cardiac rehabilitation application , 2010, The 12th IEEE International Conference on e-Health Networking, Applications and Services.

[83]  Rachel Benedyk,et al.  A comparison of usability evaluation methods for child participants in a school setting , 2007, IDC.

[84]  I. Dzitac,et al.  How to Write a Good Paper in Computer Science and HowWill It Be Measured by ISI Web of Knowledge , 2010, Int. J. Comput. Commun. Control.

[85]  Juan Manuel Carrillo-de-Gea,et al.  Internationalization requirements for e-learning audit purposes , 2012, Proceedings of the 2012 IEEE Global Engineering Education Conference (EDUCON).

[86]  P. Stone Popping the (PICO) question in research and evidence-based practice. , 2002, Applied nursing research : ANR.

[87]  Nadine Schuurman,et al.  The electronic Trauma Health Record: design and usability of a novel tablet-based tool for trauma care and injury surveillance in low resource settings. , 2014, Journal of the American College of Surgeons.

[88]  Fabio Pianesi,et al.  An exploratory study of a touch-based gestural interface for elderly , 2010, NordiCHI.

[89]  Heyoung Lee,et al.  The SAMS: Smartphone Addiction Management System and Verification , 2013, Journal of Medical Systems.

[90]  Kirsten Risden,et al.  Guidelines for usability testing with children , 1997, INTR.

[91]  Chien-Lung Hsu,et al.  Designing an Intelligent Health Monitoring System and Exploring User Acceptance for the Elderly , 2013, Journal of Medical Systems.

[92]  Minhwa Chung,et al.  Developing a Voice User Interface with Improved Usability for People with Dysarthria , 2012, ICCHP.

[93]  Orrin I. Franko,et al.  Mobile Tablet Use among Academic Physicians and Trainees , 2013, Journal of Medical Systems.

[94]  Jakob Nielsen,et al.  A mathematical model of the finding of usability problems , 1993, INTERCHI.

[95]  Katie A. Siek,et al.  Designing mobile snack application for low socioeconomic status families , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[96]  Ana Sánchez-Henarejos,et al.  Guía de buenas prácticas de seguridad informática en el tratamiento de datos de salud para el personal sanitario en atención primaria , 2014, Atención Primaria.

[97]  Ambrosio Toval,et al.  Learning systems development using reusable standard-based requirements catalogs , 2011, 2011 IEEE Global Engineering Education Conference (EDUCON).

[98]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[99]  Anna L. Cox,et al.  Questionnaires, in-depth interviews and focus groups , 2008 .

[100]  Wilhelm Kirch,et al.  Mobile Applications for Diabetics: A Systematic Review and Expert-Based Usability Evaluation Considering the Special Requirements of Diabetes Patients Age 50 Years or Older , 2014, Journal of medical Internet research.

[101]  Ana Correia de Barros,et al.  User-centred design of a mobile self-management solution for Parkinson's disease , 2013, MUM.

[102]  Alistair G. Sutcliffe,et al.  Assessing the reliability of heuristic evaluation for Web site attractiveness and usability , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[103]  Monique W. M. Jaspers,et al.  A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence , 2009, Int. J. Medical Informatics.

[104]  Stefan Kowalewski,et al.  A real-time extension to the Android platform , 2012, JTRES '12.

[105]  Savoy Court,et al.  Do "Attractive Things Work Better"? An Exploration of Search Tool Visualisations , 2005 .

[106]  J. A. de Bruin,et al.  Saccade deviation indicators for automated eye tracking analysis , 2013, ETSA '13.

[107]  Paul Mc Kevitt,et al.  Investigating the usability of PDAs with ageing users , 2007, BCS HCI.

[108]  Inmaculada Plaza,et al.  Mobile applications in an aging society: Status and trends , 2011, J. Syst. Softw..

[109]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[110]  Adrian Mendoza,et al.  Mobile User Experience: Patterns to Make Sense of it All , 2013 .