Usability of mobile applications: literature review and rationale for a new usability model

The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness has come at the expense of the usability of these devices in some contexts. We conducted a small review of mobile usability models and found that usability is usually measured in terms of three attributes; effectiveness, efficiency and satisfaction. Other attributes, such as cognitive load, tend to be overlooked in the usability models that are most prominent despite their likely impact on the success or failure of an application. To remedy this we introduces the PACMAD (People At the Centre of Mobile Application Development) usability model which was designed to address the limitations of existing usability models when applied to mobile devices. PACMAD brings together significant attributes from different usability models in order to create a more comprehensive model. None of the attributes that it includes are new, but the existing prominent usability models ignore one or more of them. This could lead to an incomplete usability evaluation. We performed a literature search to compile a collection of studies that evaluate mobile applications and then evaluated the studies using our model.

[1]  Martin Hachet,et al.  Navidget for 3D interaction: Camera positioning and further uses , 2009, Int. J. Hum. Comput. Stud..

[2]  Amy Voida,et al.  Mobiphos: a collocated-synchronous mobile photo sharing application , 2008, Mobile HCI.

[3]  Christian Heath,et al.  Swiping paper: the second hand, mundane artifacts, gesture and collaboration , 2010, Personal and Ubiquitous Computing.

[4]  Lorna McKnight,et al.  Children's Interaction with Mobile Touch-Screen Devices: Experiences and Guidelines for Design , 2010, Int. J. Mob. Hum. Comput. Interact..

[5]  Luca Chittaro,et al.  Using mobile devices to support communication between emergency medical responders and deaf people , 2010, Mobile HCI.

[6]  Luca Chittaro Human-Computer Interaction with Mobile Devices and Services: 5th International Symposium, Mobile HCI 2003, Udine, Italy, September 8-11, 2003, Proceedings , 2003 .

[7]  Dimitrios Tzovaras,et al.  3D content-based search using sketches , 2006, Personal and Ubiquitous Computing.

[8]  Pei-Chun Lin,et al.  The effects of gender differences on operational performance and satisfaction with car navigation systems , 2010, Int. J. Hum. Comput. Stud..

[9]  Joachim Meyer,et al.  Benefits and costs of adaptive user interfaces , 2010, Int. J. Hum. Comput. Stud..

[10]  Mark Billinghurst,et al.  A user study of auditory versus visual interfaces for use while driving , 2008, Int. J. Hum. Comput. Stud..

[11]  Joonhwan Lee,et al.  Iterative design of MOVE: A situationally appropriate vehicle navigation system , 2008, Int. J. Hum. Comput. Stud..

[12]  Damianos Gavalas,et al.  An innovative mobile electronic tourist guide application , 2009, Personal and Ubiquitous Computing.

[13]  Yeliz Yesilada,et al.  What input errors do you experience? Typing and pointing errors of mobile Web users , 2010, Int. J. Hum. Comput. Stud..

[14]  Yuan Lu,et al.  Usability benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform , 2008, Mobile HCI.

[15]  Joaquim A. Jorge,et al.  Assessing mobile touch interfaces for tetraplegics , 2010, Mobile HCI.

[16]  Dimitrios Tzovaras,et al.  3D content-based search using sketches , 2006, AIAI.

[17]  Luca Chittaro,et al.  Presenting evacuation instructions on mobile devices by means of location-aware 3D virtual environments , 2008, Mobile HCI.

[18]  Joo-Hwee Lim,et al.  Deploying and evaluating a mixed reality mobile treasure hunt: Snap2Play , 2008, Mobile HCI.

[19]  Miska M. Hannuksela,et al.  Does context matter in quality evaluation of mobile television? , 2008, Mobile HCI.

[20]  Peter Fröhlich,et al.  Exploring the design space of Smart Horizons , 2008, Mobile HCI.

[21]  Mark D. Dunlop,et al.  Investigating five key predictive text entry with combined distance and keystroke modelling , 2008, Personal and Ubiquitous Computing.

[22]  Stephen A. Brewster,et al.  Novel interfaces for digital cameras and camera phones , 2010, Mobile HCI.

[23]  Nicola J. Bidwell,et al.  Field testing mobile digital storytelling software in rural Kenya , 2010, Mobile HCI.

[24]  Mark A. Neerincx,et al.  Field evaluation of a mobile location-based notification system for police officers , 2008, Mobile HCI.

[25]  Dylan D. Schmorrow,et al.  Foundations of Augmented Cognition , 2013, Lecture Notes in Computer Science.

[26]  A. Ant Ozok,et al.  A Comparative Study Between Tablet and Laptop PCs: User Satisfaction and Preferences , 2008, Int. J. Hum. Comput. Interact..

[27]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[28]  Adrian David Cheok,et al.  22nd International Conference on Human-Computer Interaction with Mobile Devices and Services , 2007, Lecture Notes in Computer Science.

[29]  Enrico Rukzio,et al.  Investigating selection and reading performance on a mobile phone while walking , 2010, Mobile HCI.

[30]  Mark D. Dunlop,et al.  A calendar based Internet content pre-caching agent for small computing devices , 2008, Personal and Ubiquitous Computing.

[31]  Luís Carriço,et al.  Lessons from early stages design of mobile applications , 2008, Mobile HCI.

[32]  Alina Hang,et al.  Projector phone: a study of using mobile phones with integrated projector for interaction with maps , 2008, Mobile HCI.

[33]  Elina Vartiainen,et al.  Improving the User Experience of a Mobile Photo Gallery by Supporting Social Interaction , 2009, Int. J. Mob. Hum. Comput. Interact..

[34]  Carsten Sørensen,et al.  Organisational usability of mobile computing - Volatility and control in mobile foreign exchange trading , 2008, Int. J. Hum. Comput. Stud..

[35]  Xiaojun Bi,et al.  RearType: text entry using keys on the back of a device , 2010, Mobile HCI.

[36]  Dag Svanæs,et al.  A comparison of location and token-based interaction techniques for point-of-care access to medical information , 2008, Personal and Ubiquitous Computing.

[37]  Robert Hardy,et al.  Mobile interaction with static and dynamic NFC-based displays , 2010, Mobile HCI.

[38]  KuanTing Liu,et al.  Social playlist: enabling touch points and enriching ongoing relationships through collaborative mobile music listening , 2008, Mobile HCI.

[39]  Andreas Butz,et al.  pieTouch: a direct touch gesture interface for interacting with in-vehicle information systems , 2009, Mobile HCI.

[40]  Paul A. Cairns,et al.  Tlk or txt? Using voice input for SMS composition , 2008, Personal and Ubiquitous Computing.

[41]  Clifford Nass,et al.  Driver safety and information from afar: An experimental driving simulator study of wireless vs. in-car information services , 2008, Int. J. Hum. Comput. Stud..

[42]  François Guimbretière,et al.  Relative role of merging and two-handed operation on command selection speed , 2008, Int. J. Hum. Comput. Stud..

[43]  Mark A. Neerincx,et al.  Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness , 2008, Mobile HCI.

[44]  Rafael Ballagas,et al.  Practices in creating videos with mobile phones , 2009, Mobile HCI.

[45]  Harald Reiterer,et al.  Zoom interaction design for pen-operated portable devices , 2008, Int. J. Hum. Comput. Stud..

[46]  Brad A. Myers,et al.  The performance of hand postures in front- and back-of-device interaction for mobile computing , 2008, Int. J. Hum. Comput. Stud..

[47]  Masanori Sugimoto,et al.  A user-adaptive city guide system with an unobtrusive navigation interface , 2009, Personal and Ubiquitous Computing.

[48]  SørensenCarsten,et al.  Organisational usability of mobile computing-Volatility and control in mobile foreign exchange trading , 2008 .

[49]  Steve Whittaker,et al.  Mobile technologies in mobile spaces: Findings from the context of train travel , 2008, Int. J. Hum. Comput. Stud..

[50]  Xiangshi Ren,et al.  The Optimal Size of Handwriting Character Input Boxes on PDAs , 2009, Int. J. Hum. Comput. Interact..

[51]  Kaisa Väänänen,et al.  Expectations for user experience in haptic communication with mobile devices , 2009, Mobile HCI.

[52]  Galen C. Hunt,et al.  User experiences with activity-based navigation on mobile devices , 2010, Mobile HCI.

[53]  Birsen Donmez,et al.  Supporting intelligent and trustworthy maritime path planning decisions , 2010, Int. J. Hum. Comput. Stud..

[54]  Connor Graham,et al.  A Review of Mobile HCI Research Methods , 2003, Mobile HCI.

[55]  Stephen A. Brewster,et al.  Instrumented Usability Analysis for Mobile Devices , 2009, Int. J. Mob. Hum. Comput. Interact..

[56]  Johannes Schöning,et al.  Impact of item density on the utility of visual context in magic lens interactions , 2009, Personal and Ubiquitous Computing.

[57]  Enrico Rukzio,et al.  Evaluation of picture browsing using a projector phone , 2008, Mobile HCI.

[58]  Jaehyun Park,et al.  Touch key design for target selection on a mobile phone , 2008, Mobile HCI.

[59]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[60]  Antti Oulasvirta Designing mobile awareness cues , 2008, Mobile HCI.

[61]  Ray G. Adams,et al.  Applying advanced concepts of cognitive overload and augmentation in practice: the future of overload , 2006 .

[62]  Ben Fehnert,et al.  Measuring user experience: complementing qualitative and quantitative assessment , 2008, Mobile HCI.

[63]  Dongsong Zhang,et al.  Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications , 2005, Int. J. Hum. Comput. Interact..

[64]  Yvonne Rogers,et al.  Enhancing learning: a study of how mobile devices can facilitate sensemaking , 2010, Personal and Ubiquitous Computing.

[65]  Martin C. Maguire,et al.  Context of Use within usability activities , 2001, Int. J. Hum. Comput. Stud..

[66]  Stephen A. Brewster,et al.  T-Bars: towards tactile user interfaces for mobile touchscreens , 2008, Mobile HCI.

[67]  Oliver Bimber,et al.  Adaptive training of video sets for image recognition on mobile phones , 2009, Personal and Ubiquitous Computing.

[68]  Steve Benford,et al.  MobGeoSen: facilitating personal geosensor data collection and visualization using mobile phones , 2007, Personal and Ubiquitous Computing.

[69]  Jacob O. Wobbrock,et al.  Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces , 2008, Mobile HCI.

[70]  Jun Rekimoto,et al.  GraspZoom: zooming and scrolling control model for single-handed mobile interaction , 2009, Mobile HCI.

[71]  Kyu Ho Park,et al.  Cocktail: Exploiting Bartenders' Gestures for Mobile Interaction , 2010, Int. J. Mob. Hum. Comput. Interact..

[72]  Seongil Lee,et al.  Mobile Internet Services from Consumers' Perspectives , 2009, Int. J. Hum. Comput. Interact..

[73]  Patrik Floréen,et al.  Capricorn - an intelligent user interface for mobile widgets , 2008, Mobile HCI.

[74]  Clifton Forlines,et al.  Contextual push-to-talk: shortening voice dialogs to improve driving performance , 2010, Mobile HCI.

[75]  Carol A. Taylor,et al.  Always On: A Framework for Understanding Personal Mobile Web Motivations, Behaviors, and Contexts of Use , 2009, Int. J. Mob. Hum. Comput. Interact..

[76]  Theodoros N. Arvanitis,et al.  Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities , 2009, Personal and Ubiquitous Computing.

[77]  Hans-Werner Gellersen,et al.  Usage of spatial information for selection of co-located devices , 2008, Mobile HCI.

[78]  Shumin Zhai,et al.  Foundations for designing and evaluating user interfaces based on the crossing paradigm , 2010, TCHI.

[79]  James D. Hollan,et al.  Papiercraft: A gesture-based command system for interactive paper , 2008, TCHI.

[80]  Mikko Honkala,et al.  Linked internet UI: a mobile user interface optimized for social networking , 2010, Mobile HCI.

[81]  Pearl Brereton,et al.  Empirical evidence about the UML: a systematic literature review , 2011, Softw. Pract. Exp..

[82]  Michael Rohs,et al.  Semi-automatic zooming for mobile map navigation , 2010, Mobile HCI.

[83]  Albrecht Schmidt,et al.  An evaluation of product review modalities for mobile phones , 2010, Mobile HCI.

[84]  Gerd Kortuem,et al.  Supporting device discovery and spontaneous interaction with spatial references , 2009, Personal and Ubiquitous Computing.

[85]  Markku Turunen,et al.  User expectations and user experience with different modalities in a mobile phone controlled home entertainment system , 2009, Mobile HCI.

[86]  Dario D. Salvucci Predicting the effects of in-car interface use on driver performance: an integrated model approach , 2001, Int. J. Hum. Comput. Stud..

[87]  Antti Oulasvirta,et al.  Embodied interaction with a 3D versus 2D mobile map , 2009, Personal and Ubiquitous Computing.

[88]  Andrew Sears,et al.  Shifting the focus from accuracy to recallability: A study of informal note-taking on mobile information technologies , 2009, TCHI.

[89]  Sabine Schröder,et al.  Making a completely icon-based menu in mobile devices to become true: a user-centered design approach for its development , 2008, Mobile HCI.

[90]  Niels Henze,et al.  Evaluation of an off-screen visualization for magic lens and dynamic peephole interfaces , 2010, Mobile HCI.

[91]  Beom Suk Jin,et al.  Development of the Conceptual Prototype for Haptic Interface on the Telematics System , 2009, Int. J. Hum. Comput. Interact..

[92]  Matt Jones,et al.  ONTRACK: Dynamically adapting music playback to support navigation , 2008, Personal and Ubiquitous Computing.

[93]  Henry Been-Lirn Duh,et al.  Playing different games on different phones: an empirical study on mobile gaming , 2008, Mobile HCI.

[94]  Shuang Xu,et al.  Development of a Dual-Modal Presentation of Texts for Small Screens , 2008, Int. J. Hum. Comput. Interact..

[95]  Michael Rohs,et al.  Bridging the gap between the Kodak and the Flickr generations: A novel interaction technique for collocated photo sharing , 2009, Int. J. Hum. Comput. Stud..

[96]  Nikhil Welankar,et al.  Rangoli: a visual phonebook for low-literate users , 2008, Mobile HCI.

[97]  Denis Gillet,et al.  End-to-end adaptation scheme for ubiquitous remote experimentation , 2009, Personal and Ubiquitous Computing.

[98]  Chris Baber,et al.  Mobile technology for crime scene examination , 2009, Int. J. Hum. Comput. Stud..

[99]  Yolande Berbers,et al.  Mobile phones assisting with health self-care: a diabetes case study , 2008, Mobile HCI.

[100]  Wan Chul Yoon,et al.  Age- and experience-related user behavior differences in the use of complicated electronic devices , 2008, Int. J. Hum. Comput. Stud..

[101]  Nuria Oliver,et al.  Text versus speech: a comparison of tagging input modalities for camera phones , 2009, Mobile HCI.

[102]  DahlYngve,et al.  A comparison of location and token-based interaction techniques for point-of-care access to medical information , 2008 .

[103]  Matthias Jarke,et al.  Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services , 2009, MobileHCI 2009.

[104]  Hans-Peter Hutter,et al.  Augmented mobile tagging , 2008, Mobile HCI.

[105]  Niels Henze,et al.  Visualization of off-screen objects in mobile augmented reality , 2010, Mobile HCI.

[106]  Martina Ziefle,et al.  Ask and You Will Receive: Training Novice Adults to use a PDA in an Active Learning Environment , 2010, Int. J. Mob. Hum. Comput. Interact..

[107]  Jaakko Keränen,et al.  Hands on music: physical approach to interaction with digital music , 2009, Mobile HCI.

[108]  Rod McCall,et al.  TimeWarp: interactive time travel with a mobile mixed reality game , 2008, Mobile HCI.

[109]  Alison Moore Time in motion. , 2009 .

[110]  Stephen Fickas,et al.  Route-following assistance for travelers with cognitive impairments: A comparison of four prompt modes , 2008, Int. J. Hum. Comput. Stud..

[111]  Yvonne de Kort,et al.  Sharing places: testing psychological effects of location cueing frequency and explicit vs. inferred closeness , 2008, Mobile HCI.

[112]  Luca Chittaro,et al.  Supporting blind users in selecting from very long lists of items on mobile phones , 2010, Mobile HCI.

[113]  Luca Chittaro,et al.  Navigation techniques for small-screen devices: An evaluation on maps and web pages , 2008, Int. J. Hum. Comput. Stud..

[114]  Markku Turunen,et al.  Evaluation of predictive text and speech inputs in a multimodal mobile route guidance application , 2008, Mobile HCI.

[115]  Rachel Harrison,et al.  Evaluating Mobile Applications: A Spreadsheet Case Study , 2012, Int. J. Mob. Hum. Comput. Interact..

[116]  Jakob E. Bardram,et al.  Activity-based computing for medical work in hospitals , 2009, TCHI.

[117]  Ian Oakley,et al.  Motion marking menus: An eyes-free approach to motion input for handheld devices , 2009, Int. J. Hum. Comput. Stud..

[118]  Maxine Glancy,et al.  Fixed in time and "time in motion": mobility of vision through a SenseCam lens , 2009, Mobile HCI.

[119]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[120]  Ray Adams,et al.  Decision and stress: cognition and e-accessibility in the information workplace , 2007, Universal Access in the Information Society.

[121]  Parisa Eslambolchilar,et al.  Control centric approach in designing scrolling and zooming user interfaces , 2008, Int. J. Hum. Comput. Stud..

[122]  Luca Chittaro,et al.  Map, diagram, and web page navigation on mobile devices: the effectiveness of zoomable user interfaces with overviews , 2008, Mobile HCI.

[123]  Luís Carriço,et al.  Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services , 2010, MobileHCI 2010.

[124]  Matt Jones,et al.  Exploring casual point-and-tilt interactions for mobile geo-blogging , 2010, Personal and Ubiquitous Computing.

[125]  Stephen A. Brewster,et al.  Pressure-based text entry for mobile devices , 2009, Mobile HCI.

[126]  Ying Liu,et al.  RotaTxt: Chinese pinyin input with a rotator , 2008, Mobile HCI.

[127]  HongJason,et al.  Understanding and capturing people's privacy policies in a mobile social networking application , 2009 .

[128]  Carl Gutwin,et al.  A model of novice and expert navigation performance in constrained-input interfaces , 2010, TCHI.

[129]  Stephan Winter,et al.  Situated local and global orientation in mobile you-are-here maps , 2010, Mobile HCI.

[130]  Ehud Sharlin,et al.  A tangible user interface for assessing cognitive mapping ability , 2009, Int. J. Hum. Comput. Stud..

[131]  Lorrie Faith Cranor,et al.  Understanding and capturing people’s privacy policies in a mobile social networking application , 2009, Personal and Ubiquitous Computing.

[132]  Roger Wattenhofer,et al.  Exploring music collections on mobile devices , 2008, Mobile HCI.

[133]  Minna Kulju,et al.  Developing a questionnaire for measuring mobile business service experience , 2008, Mobile HCI.

[134]  Tiziana Catarci,et al.  Appropriating Heuristic Evaluation for Mobile Computing , 2009, Int. J. Mob. Hum. Comput. Interact..

[135]  Steinar Kristoffersen,et al.  Design ideas for IT in public spaces , 2010, Personal and Ubiquitous Computing.

[136]  Matti Rossi,et al.  An empirical investigation of mobile ticketing service adoption in public transportation , 2006, Personal and Ubiquitous Computing.

[137]  Xia Wang,et al.  Data logging plus e-diary: towards an online evaluation approach of mobile service field trial , 2010, Mobile HCI.

[138]  David W. McDonald,et al.  Proactive displays: Supporting awareness in fluid social environments , 2008, TCHI.

[139]  Martin Hachet,et al.  A study of direct versus planned 3D camera manipulation on touch-based mobile phones , 2009, Mobile HCI.

[140]  Felix B. Tan,et al.  The Relationship Between Mobile Service Quality, Perceived Technology Compatibility, and Users' Perceived Playfulness in the Context of Mobile Information and Entertainment Services , 2008, Int. J. Hum. Comput. Interact..

[141]  Andrés Lucero,et al.  Collaborative use of mobile phones for brainstorming , 2010, Mobile HCI.

[142]  Kun Yu,et al.  Coupa: operation with pen linking on mobile devices , 2009, Mobile HCI.

[143]  Panagiotis Germanakos,et al.  Estimating Cognitive Overload in Mobile Applications for Decision Support within the Medical Domain , 2012, ICEIS.

[144]  Privender Saini,et al.  The relationship between goal difficulty and performance in the context of a physical activity intervention program , 2008, Mobile HCI.

[145]  David M. Nichols,et al.  Contextual queries express mobile information needs , 2010, Mobile HCI.

[146]  Virpi Roto,et al.  User Experience of Mobile Internet: Analysis and Recommendations , 2009, Int. J. Mob. Hum. Comput. Interact..

[147]  Eija Kaasinen,et al.  User Acceptance of Mobile Services , 2009, Int. J. Mob. Hum. Comput. Interact..

[148]  Mervyn A. Jack,et al.  A usability comparison of three alternative message formats for an SMS banking service , 2008, Int. J. Hum. Comput. Stud..

[149]  Barbara Leporini,et al.  Vibrotactile feedback as an orientation aid for blind users of mobile guides , 2008, Mobile HCI.

[150]  Christos Sintoris,et al.  MuseumScrabble: Design of a Mobile Game for Children's Interaction with a Digitally Augmented Cultural Space , 2010, Int. J. Mob. Hum. Comput. Interact..

[151]  Matthias Böhmer,et al.  Exploiting the icon arrangement on mobile devices as information source for context-awareness , 2010, Mobile HCI.

[152]  Stephen A. Brewster,et al.  Pressure-based menu selection for mobile devices , 2010, Mobile HCI.

[153]  Youngwoo Yoon,et al.  Context-aware photo selection for promoting photo consumption on a mobile phone , 2008, Mobile HCI.

[154]  Janne Vainio,et al.  Interacting with the flow , 2010, Mobile HCI.