On the Relationship between Perception of Usability and Subjective Mental Workload of Web Interfaces

Inspection methods and cheap self-reporting procedures have been significantly employed in the field of Human-Computer Interaction for assessing the usability of interfaces, systems and technologies. However, there is a tendency to overlook aspects related to the context and features of the users during the usability assessment process. This research introduces the concept of mental workload as an aid to enhance usability measurement. A user-study has been designed and executed in the context of human-web interaction. The aim was to investigate the relationship between the perception of usability of three popular web-sites, and the mental workload imposed by a set of typical tasks executed over them. Scores obtained with the System Usability Scale were compared to the mental workload scores obtained from the NASA Task Load Index and the Workload Profile assessment procedures. Findings suggest that perception of usability and mental workload are likely to be two non-overlapping constructs, and there is no clear evidence of their interaction. They measure two different aspects of human-system interaction and therefore they could be jointly employed to better describe user experience.

[1]  B. Cain.  A Review of the Mental Workload Literature , 2007 .

[2]  Luca Longo,et al.  A Computational Analysis of Cognitive Effort , 2010, ACIIDS.

[3]  F. Thomas Eggemeier,et al.  Workload assessment methodology. , 1986 .

[4]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[5]  Michael J. Albers.  Tapping as a measure of cognitive load and website usability , 2011, SIGDOC '11.

[6]  Brian Shackel,et al.  Usability - Context, framework, definition, design and evaluation , 1991, Interact. Comput..

[7]  Thomas S. Tullis,et al.  A Comparison of Questionnaires for Assessing Website Usability , 2004 .

[8]  James C. Schueren,et al.  Using the Subjective Workload Dominance (SWORD) Technique for Projective Workload Assessment , 1991 .

[9]  Haibin Zhu,et al.  Restrain mental workload with roles in HCI , 2009, 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH).

[10]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[11]  G. Salvendy,et al.  Review and reappraisal of modelling and predicting mental workload in single- and multi-task environments , 2000 .

[12]  Daniel Gopher,et al.  Workload: An examination of the concept. , 1986 .

[13]  Susana Rubio,et al.  Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA‐TLX, and Workload Profile Methods , 2004 .

[14]  Waldemar Karwowski,et al.  Mental Workload Measurement , 2006 .

[15]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[16]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[17]  Luca Longo,et al.  Information Foraging Theory as a Form of Collective Intelligence for Social Search , 2009, ICCCI.

[18]  Pamela S. Tsang,et al.  Mental Workload and Situation Awareness , 2006 .

[19]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[20]  Gerhard Fischer,et al.  User Modeling in Human–Computer Interaction , 2001, User Modeling and User-Adapted Interaction.

[21]  P. Tsang,et al.  Diagnosticity and multidimensional subjective workload ratings. , 1996, Ergonomics.

[22]  Karel Brookhuis,et al.  Handbook of Human Factors and Ergonomics Methods , 2009 .

[23]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[24]  Thomas E. Nygren,et al.  The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload , 1988 .

[25]  Luca Longo,et al.  Formalising Human Mental Workload as a Defeasible Computational Concept , 2014 .

[26]  Barry H. Kantowitz,et al.  Mental Workload , 2020, Encyclopedia of Behavioral Medicine.

[27]  Miles Macleod,et al.  Usability in Context: Improving Quality of Use , 1994 .

[28]  Kasper Hornbæk,et al.  Current practice in measuring usability: Challenges to usability studies and research , 2006, Int. J. Hum. Comput. Stud..

[29]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[30]  Vicente Moret-Bonillo,et al.  A Context-of-Use Taxonomy for Usability Studies , 2010, Int. J. Hum. Comput. Interact..

[31]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[32]  Luca Longo,et al.  A defeasible reasoning framework for human mental workload representation and assessment , 2015, Behav. Inf. Technol..

[33]  W. Karwowski.  International encyclopedia of ergonomics and human factors , 2001 .

[34]  P. D Harper,et al.  Improving user satisfaction: the questionnaire for user interaction satisfaction version 5 , 1993 .

[35]  Fabio Rusconi,et al.  The Importance of Human Mental Workload in Web Design , 2012, WEBIST.

[36]  Dick de Waard,et al.  The measurement of drivers' mental workload , 1996 .

[37]  Jacek Gwizdka,et al.  Distribution of cognitive load in Web search , 2010, J. Assoc. Inf. Sci. Technol..

[38]  Luca Longo,et al.  A novel methodology for evaluating user interfaces in health care , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[39]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[40]  Jacek Gwizdka,et al.  Assessing Cognitive Load on Web Search Tasks , 2009, ArXiv.

[41]  A. H. Roscoe,et al.  A Subjective Rating Scale for Assessing Pilot Workload in Flight: A decade of Practical Use , 1990 .

[42]  Longo Luca,et al.  Information Foraging Theory as a Form of Collective Intelligence for Social Search , 2009, ICCCI 2009.

[43]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[44]  George E. Cooper,et al.  The use of pilot rating in the evaluation of aircraft handling qualities , 1969 .