Psychometric Evaluation of the F-SUS: Creation and Validation of the French Version of the System Usability Scale

ABSTRACT While the System Usability Scale (SUS) is probably one of the most widely used questionnaires to measure the perceived ease of use of interactive systems, there is currently no scientific valid translation in French. This article describes the translation and statistical validation of the French version of the SUS, called F-SUS. On the basis of two translations carried out by a committee of bilingual experts, the various psychometric analyses made it possible to select only one translation. Fidelity measurement, factor analysis and sensitivity measurement obtained results very close or similar to the original version of the SUS. Thus, the F-SUS can be used with confidence by French-speaking usability researchers and practitioners.

[1]  James R. Lewis,et al.  UMUX-LITE: when there's no time for the SUS , 2013, CHI.

[2]  Zahra Ghanbari,et al.  Psychometric properties of the persian language version of the system usability scale. , 2014, Health promotion perspectives.

[3]  James R. Lewis,et al.  A Slovene Translation of the System Usability Scale: The SUS-SI , 2015, Int. J. Hum. Comput. Interact..

[4]  Tao Yang,et al.  DEEP: Design-Oriented Evaluation of Perceived Usability , 2012, Int. J. Hum. Comput. Interact..

[5]  Arzu Çöltekin,et al.  Evaluating the Effectiveness of Interactive Map Interface Designs: A Case Study Integrating Usability Metrics with Eye-Movement Analysis , 2009 .

[6]  John Quarles,et al.  Augmented Reality for Children in a Confirmation Task: Time, Fatigue, and Usability , 2019, VRST.

[7]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[8]  Frederick L. Oswald,et al.  Is It Time to Go Positive? Assessing the Positively Worded System Usability Scale (SUS) , 2020, Hum. Factors.

[9]  Vassilis Koutkias,et al.  Development and usability of a personalized sensor-based system for pervasive healthcare , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[10]  Keith Nesbitt,et al.  Comparing usability and engagement between a serious game and a traditional online program , 2016, ACSW.

[11]  Azham Hussain,et al.  USABILITY EVALUATION OF A WEB-BASED HEALTH AWARENESS PORTAL ON SMARTPHONE DEVICES USING ISO 9241- 11 MODEL , 2015 .

[12]  Annie W. Y. Ng,et al.  Measuring the Usability of Safety Signs : A Use of System Usability Scale ( SUS ) , 2022 .

[13]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[14]  Michalis Nik Xenos,et al.  Perceived Usability Evaluation of Learning Management Systems: A First Step towards Standardization of the System Usability Scale in Greek , 2012, 2012 16th Panhellenic Conference on Informatics.

[15]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[16]  SampaioAlberto Quantifying the user experience , 2013 .

[17]  Chih-Hung Wu,et al.  The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability , 2014, Comput. Hum. Behav..

[18]  Antonio Peñalver Benavent,et al.  How efficient can be a user with a tablet versus a smartphone? , 2014, Interacción '14.

[19]  Christof Lutteroth,et al.  Evaluation of a web-based telehealth system: a preliminary investigation with seniors in New Zealand , 2012, CHINZ '12.

[20]  Jum C. Nunnally,et al.  An Overview of Psychological Measurement , 1978 .

[21]  Aldo von Wangenheim,et al.  Does the use of structured reporting improve usability? A comparative evaluation of the usability of two approaches for findings reporting in a large-scale telecardiology context , 2014, J. Biomed. Informatics.

[22]  Martin Schrepp,et al.  Construction and Evaluation of a User Experience Questionnaire , 2008, USAB.

[23]  van Epwm Erik Veenendaal,et al.  Questionnaire based usability testing , 1999 .

[24]  Katarzyna Jach,et al.  Pre-testing of Polish Translation of System Usability Scale (SUS) , 2016, ISAT.

[25]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[26]  Franca Garzotto,et al.  Quality and Potential for Adoption of Usability Evaluation Methods , 2008, J. Web Eng..

[27]  G. Gronier,et al.  Création et validation d’une version française du questionnaire AttrakDiff pour l’évaluation de l’expérience utilisateur des systèmes interactifs , 2015 .

[28]  Sjaak Brinkkemper,et al.  Computerized Decision Support Improves Medication Review Effectiveness: An Experiment Evaluating the STRIP Assistant’s Usability , 2015, Drugs & Aging.

[29]  Nikolaos K. Tselios,et al.  Does size matter?: investigating the impact of mobile phone screen size on users' perceived usability, effectiveness and efficiency. , 2013, MobileHCI '13.

[30]  Rebecca Ford,et al.  The Usability Perception Scale (UPscale): A Measure for Evaluating Feedback Displays , 2013, HCI.

[31]  Philip T. Kortum,et al.  Usability Ratings for Everyday Products Measured With the System Usability Scale , 2013, Int. J. Hum. Comput. Interact..

[32]  Angel Chi-Poot,et al.  Usability evaluation of an augmented reality system for teaching Euclidean vectors , 2016 .

[33]  Jeff Sauro,et al.  When designing usability questionnaires, does it hurt to be positive? , 2011, CHI.

[34]  Christos Katsanos,et al.  Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale , 2015 .

[35]  K. Baumlin,et al.  A Pilot Study on Usability Analysis of Emergency Department Information System by Nurses , 2012, Applied Clinical Informatics.

[36]  Ellen A. Drost,et al.  Validity and Reliability in Social Science Research. , 2011 .

[37]  Francisco J. García-Peñalvo,et al.  Usability Test of WYRED Platform , 2018, HCI.

[38]  Amalia Suzianti,et al.  Redesign of user interface for e-government application using usability testing method , 2018, ICCIP '18.

[39]  Harry Budi Santoso,et al.  An Indonesian adaptation of the System Usability Scale (SUS) , 2016, 2016 International Conference on Advanced Computer Science and Information Systems (ICACSIS).

[40]  Martin C. Maguire,et al.  Methods to support human-centred design , 2001, Int. J. Hum. Comput. Stud..

[41]  Dag Svanæs,et al.  Assessing seniors' user experience (UX) of exergames for balance training , 2014, NordiCHI.

[42]  Philip T. Kortum,et al.  Measuring the Usability of Mobile Applications for Phones and Tablets , 2015, Int. J. Hum. Comput. Interact..

[43]  Robertas Damaševičius,et al.  Gamification of a Project Management System , 2014, ACHI 2014.

[44]  Devender Kumar,et al.  Usability of a mood assessment smartphone prototype based on humor appreciation , 2018, MobileHCI Adjunct.

[45]  Aaron Marcus,et al.  Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience , 2014, Lecture Notes in Computer Science.

[46]  R. Cattell The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[47]  Buzhou Tang,et al.  Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation , 2018, JMIR mHealth and uHealth.

[48]  Jeff Sauro,et al.  The Factor Structure of the System Usability Scale , 2009, HCI.

[49]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[50]  Larwan Berke,et al.  Design and Psychometric Evaluation of American Sign Language Translations of Usability Questionnaires , 2019, ACM Trans. Access. Comput..

[51]  Jürgen Baumgartner,et al.  Pictorial System Usability Scale (P-SUS): Developing an Instrument for Measuring Perceived Usability , 2019, CHI.

[52]  Shirley Beul-Leusmann,et al.  Usability Evaluation of Mobile Passenger Information Systems , 2014, HCI.

[53]  Daqing Zhang,et al.  NationTelescope: Monitoring and visualizing large-scale collective behavior in LBSNs , 2015, J. Netw. Comput. Appl..

[54]  Rana Tassabehji,et al.  Evaluating biometrics for online banking: The case for usability , 2012, Int. J. Inf. Manag..

[55]  O Mashta Time to go. , 1998, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[56]  Nelson Pacheco da Rocha,et al.  European Portuguese Validation of the System Usability Scale (SUS) , 2015, DSAI.

[57]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[58]  Francesco Ricci,et al.  Usability Assessment of a Context-Aware and Personality-Based Mobile Recommender System , 2014, EC-Web.

[59]  Ibrahim M. Baggili,et al.  Diabetic e-Management System (DEMS) , 2006, Third International Conference on Information Technology: New Generations (ITNG'06).

[60]  Bareeq A. Al Ghannam,et al.  Towards a Standard Arabic System Usability Scale: Psychometric Evaluation using Communication Disorder App , 2018, Int. J. Hum. Comput. Interact..

[61]  Philip T. Kortum,et al.  The Impact of Personality on the Subjective Assessment of Usability , 2018, Int. J. Hum. Comput. Interact..

[62]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .

[63]  M. Thüring,et al.  Developing and Validating an English Version of the meCUE Questionnaire for Measuring User Experience , 2016 .

[64]  Thomas K. Landauer,et al.  Behavioral Research Methods in Human-Computer Interaction , 1997 .

[65]  K. Leung,et al.  Personality in cultural context: methodological issues. , 2001, Journal of personality.

[66]  Noureddine Kerzazi,et al.  Inquiry on usability of two software process modeling systems using ISO/IEC 9241 , 2011, 2011 24th Canadian Conference on Electrical and Computer Engineering(CCECE).

[67]  Nancy Staggers,et al.  Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics , 2015, J. Am. Medical Informatics Assoc..

[68]  Gary Perlman Practical usability evaluation , 1995, CHI 95 Conference Companion.

[69]  Víctor M. González,et al.  UX-AdChat: Semi-automatic, predictable usability evaluator of written dialogue systems , 2015, CLIHC.

[70]  Kasper Hornbæk,et al.  Current practice in measuring usability: Challenges to usability studies and research , 2006, Int. J. Hum. Comput. Stud..

[71]  Jurek Kirakowski,et al.  Measuring the Usability of Web Sites , 1998 .

[72]  Neil Charness,et al.  Remote Health Monitoring for Older Adults and Those with Heart Failure: Adherence and System Usability. , 2016, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[73]  Daniel Zappala,et al.  Why Johnny Still, Still Can't Encrypt: Evaluating the Usability of a Modern PGP Client , 2015, ArXiv.

[74]  Nan Jiang,et al.  Digitising a medical clerking system with multimodal interaction support , 2017, ICMI.

[75]  Peter H. Veltink,et al.  Usability Evaluation of a VibroTactile Feedback System in Stroke Subjects , 2017, Front. Bioeng. Biotechnol..

[76]  Kraig Finstad,et al.  The Usability Metric for User Experience , 2010, Interact. Comput..

[77]  Brian M. Slator,et al.  Using the System Usability Scale in a Classification Learning Environment , 2016 .

[78]  E. Emmanuel,et al.  Anthropometric studies on some primary school children of Ajaokuta local Government area of Kogi State, Nigeria , 2017 .

[79]  Vanessa Larue Apport du system usability scale à l'activité d'ergonomie d'évaluation , 2009, IHM '09.

[80]  Deiana Chiara,et al.  Laser micro-polished stainless steel surfaces with improved bacteria removal capability , 2016 .

[81]  Aaron Marcus Design, User Experience, and Usability. Design Philosophy, Methods, and Tools , 2013, Lecture Notes in Computer Science.

[82]  Jeff Sauro,et al.  SUPR-Q: a comprehensive measure of the quality of the website user experience , 2015 .

[83]  Dilek Karahoca,et al.  Re-assessing the usability metric for user experience (UMUX) scale , 2016 .

[84]  Chih-Kai Chang,et al.  Assessing the effectiveness of learning solid geometry by using an augmented reality-assisted learning system , 2015, Interact. Learn. Environ..

[85]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[86]  Nikolaos K. Tselios,et al.  Extension of technology acceptance model by using system usability scale to assess behavioral intention to use e-learning , 2017, Education and Information Technologies.

[87]  Shengdong Zhao,et al.  Assessing the Utility of the System Usability Scale for Evaluating Voice-based User Interfaces , 2018, CCHI.

[88]  Hirokazu Kato,et al.  Toward Standard Usability Questionnaires for Handheld Augmented Reality , 2015, IEEE Computer Graphics and Applications.

[89]  Min-Chai Hsieh,et al.  Establishment and Usability Evaluation of an Interactive AR Learning System on Conservation of Fish , 2011 .

[90]  James R. Lewis,et al.  The System Usability Scale: Past, Present, and Future , 2018, Int. J. Hum. Comput. Interact..

[91]  Clayton Shepard,et al.  An empirical analysis of smartphone personalisation: measurement and user variability , 2012, Behav. Inf. Technol..

[92]  Gavriel Salvendy,et al.  A proposed index of usability: A method for comparing the relative usability of different software systems , 1997, Behav. Inf. Technol..

[93]  Michael Burmester,et al.  AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität , 2003, MuC.

[94]  Abdelkamel Tari,et al.  Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis , 2016, Comput. Hum. Behav..

[95]  Stefano Federici,et al.  On the dimensionality of the System Usability Scale: a test of alternative measurement models , 2009, Cognitive Processing.

[96]  Weixun Cao,et al.  Learning and Collaboration Technologies. Design, Development and Technological Innovation , 2018, Lecture Notes in Computer Science.

[97]  Martin Schrepp,et al.  Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios , 2014, HCI.

[98]  Alexander Blecken,et al.  Usability Evaluation of a Learning Management System , 2010, 2010 43rd Hawaii International Conference on System Sciences.

[99]  Dara Meldrum,et al.  Virtual reality rehabilitation of balance: assessment of the usability of the Nintendo Wii® Fit Plus , 2012, Disability and rehabilitation. Assistive technology.

[100]  Reetta Heinonen,et al.  Usability and feasibility of mobile phone diaries in an experimental physical exercise study. , 2012, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[101]  Jonathan Earthy,et al.  The Benefits of Using ISO 13407: Human Centred Design Process for Interactive Systems , 2001, INTERACT.

[102]  Grassyara Pinho Tolentino,et al.  Usability of Serious Games for Health , 2011, 2011 Third International Conference on Games and Virtual Worlds for Serious Applications.

[103]  R. J. Vallerand,et al.  Vers une méthodologie de validation trans-culturelle de questionnaires psychologiques: Implications pour la recherche en langue française. , 1989 .

[104]  R. Peterson A Meta-analysis of Cronbach's Coefficient Alpha , 1994 .

[105]  Fleur Fritz,et al.  Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency , 2012, Int. J. Medical Informatics.