Multi-Language Toolkit for the System Usability Scale

ABSTRACT The System Usability Scale (SUS) is a widely used instrument that measures the subjective usability of products and systems. Although past research has demonstrated the psychometric reliability and criterion-related validity of the SUS in specific languages, the approach and methodology of validating the translations has been somewhat inconsistent. This paper addresses this issue by systematically translating and validating the SUS across multiple languages. Native speakers of Arabic, Chinese, French, German, Hindi, and Spanish evaluated five common everyday products using the translated SUS. Evidence of consistent scale reliability and validity was found in the Chinese, French, German and Spanish SUS, with Cronbach’s alpha often greater than .80, as well as large statistically significant correlations between the SUS and a one-item adjective rating self-report of overall usability (r =.54-.74). Validity of the SUS across languages was also demonstrated by finding reliable differences in mean SUS scores between products. Overall, these translated SUS measures can be used with confidence in practice to measure the usability of everyday products and systems.

[1]  Katarzyna Jach,et al.  Pre-testing of Polish Translation of System Usability Scale (SUS) , 2016, ISAT.

[2]  Frederick L. Oswald,et al.  Is It Time to Go Positive? Assessing the Positively Worded System Usability Scale (SUS) , 2020, Hum. Factors.

[3]  Torkil Clemmensen,et al.  Towards a Theory of Cultural Usability: A Comparison of ADA and CM-U Theory , 2009, HCI.

[4]  Philip T. Kortum,et al.  Usability Ratings for Everyday Products Measured With the System Usability Scale , 2013, Int. J. Hum. Comput. Interact..

[5]  James R. Lewis,et al.  A Slovene Translation of the System Usability Scale: The SUS-SI , 2015, Int. J. Hum. Comput. Interact..

[6]  D. A. Kenny,et al.  Effect of the Number of Variables on Measures of Fit in Structural Equation Modeling , 2003 .

[7]  Oskar Jonsson,et al.  Cross-national usability study of a housing accessibility app: findings from the European innovage project , 2016 .

[8]  Morten Hertzum,et al.  Cultural cognition in usability evaluation , 2009, Interact. Comput..

[9]  Zahra Ghanbari,et al.  Psychometric properties of the persian language version of the system usability scale. , 2014, Health promotion perspectives.

[10]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .

[11]  Bareeq A. Al Ghannam,et al.  Towards a Standard Arabic System Usability Scale: Psychometric Evaluation using Communication Disorder App , 2018, Int. J. Hum. Comput. Interact..

[12]  Nelson Pacheco da Rocha,et al.  European Portuguese Validation of the System Usability Scale (SUS) , 2015, DSAI.

[13]  E. Erdfelder,et al.  Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses , 2009, Behavior research methods.

[14]  J. Horn.  A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[15]  C. Lance,et al.  The Sources of Four Commonly Reported Cutoff Criteria , 2006 .

[16]  Timothy D. Wilson,et al.  The halo effect: Evidence for unconscious alteration of judgments. , 1977 .

[17]  Hsiu Ching Laura Hsieh.  Exploring the Impact of Cultures on Web Usability Test , 2011, HCI.

[18]  Antonella De Angeli,et al.  Introducing ATMs in India: a contextual inquiry , 2004, Interact. Comput..

[19]  Huatong Sun,et al.  Cross-Cultural Technology Design: Creating Culture-Sensitive Technology for Local Users (Human-Technology Interaction) , 2012 .

[20]  Alvin Yeo,et al.  Cultural effects in usability assessment , 1998, CHI Conference Summary.

[21]  Vera Lomazzi,et al.  Using cultural and structural indicators to explain measurement noninvariance in gender role attitudes with multilevel structural equation modeling. , 2019, Social science research.

[22]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[23]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[24]  Stefano Federici,et al.  On the dimensionality of the System Usability Scale: a test of alternative measurement models , 2009, Cognitive Processing.

[25]  Torkil Clemmensen,et al.  An Overview of a Decade of Journal Publications about Culture and Human-Computer Interaction (HCI) , 2009, HWID.

[26]  Jeff Sauro,et al.  The Factor Structure of the System Usability Scale , 2009, HCI.

[27]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[28]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[29]  D. Betsy McCoach,et al.  The Performance of RMSEA in Models With Small Degrees of Freedom , 2015 .

[30]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[31]  Rod Walsh,et al.  Cultural differences in smartphone user experience evaluation , 2010, MUM.

[32]  Thomas K. Landauer,et al.  Research Methods in Human-Computer Interaction , 1988 .

[33]  Stefano Federici,et al.  Assessing User Satisfaction in the Era of User Experience: Comparison of the SUS, UMUX, and UMUX-LITE as a Function of Product Experience , 2015, Int. J. Hum. Comput. Interact..