Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale

Perceived usability affects greatly student’s learning effectiveness and overall learning experience, and thus is an important requirement of educational software. The System Usability Scale (SUS) is a well-researched and widely used questionnaire for perceived usability evaluation. However, surprisingly few studies have used SUS to evaluate the perceived usability of learning management systems (LMSs). This paper presents an empirical evaluation of the SUS questionnaire in the context of LMSs’ perceived usability evaluation. Eleven studies involving 769 students were conducted, in which participants evaluated the usability of two LMSs (eClass and Moodle) used within courses of their curriculum. It was found that the perceived usability of the evaluated LMSs is at a satisfactory level (mean SUS score 76.27). Analysis of the results also demonstrated the validity and reliability of SUS for LMSs’ evaluation, and that it remains robust even for small sample sizes. Moreover, the following SUS attributes were investigated in the context of LMSs evaluation: gender, age, prior experience with the LMS, Internet self-efficacy, attitude towards the Internet and usage frequency of the LMS.

[1]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[2]  Thomas S. Tullis,et al.  A Comparison of Questionnaires for Assessing Website Usability , 2004 .

[3]  Lisa Gjedde,et al.  Current developments in technology-assisted education , 2006 .

[4]  Andrina Granic,et al.  Usability Testing and Expert Inspections Complemented by Educational Evaluation: A Case Study of an e-Learning Platform , 2011, J. Educ. Technol. Soc..

[5]  I. Pahl.  Design and evaluation of Web-based Learning Environments using Information Foraging models , 2007 .

[6]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[7]  Thomas P. Van Dyke,et al.  Development and validation of an Internet self-efficacy scale , 2001, Behav. Inf. Technol..

[8]  Lei Shi,et al.  Evaluating System Functionality in Social Personalized Adaptive E-Learning Systems , 2013, EC-TEL.

[9]  Eric Zhi-Feng Liu,et al.  Design and Implementation of a Simulation-Based Learning System for International Trade. , 2014 .

[10]  Tobias Ley,et al.  Scaling up Learning for Sustained Impact , 2013, Lecture Notes in Computer Science.

[11]  N. Emelyanova,et al.  Introducing a Learning Management System at a Russian University: Students' and Teachers' Perceptions. , 2014 .

[12]  Stelios Daskalakis,et al.  Assessing the Acceptance of a Blended Learning University Course , 2011, J. Educ. Technol. Soc..

[13]  Philip T. Kortum,et al.  Usability Ratings for Everyday Products Measured With the System Usability Scale , 2013, Int. J. Hum. Comput. Interact..

[14]  Nik Bessis,et al.  User-centred evaluation of an e-learning repository , 2006, NordiCHI '06.

[15]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[16]  Andy P. Field,et al.  Discovering Statistics Using SPSS , 2000 .

[17]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[18]  A. Ituma.  An evaluation of students’ perceptions and engagement with e-learning components in a campus based university , 2011 .

[19]  Nikolaos M. Avouris,et al.  Evaluating website navigability: validation of a tool-based approach through two eye-tracking user studies , 2010, New Rev. Hypermedia Multim..

[20]  Marina Papastergiou,et al.  Enhancing Physical Education and Sport Science students' self-efficacy and attitudes regarding Information and Communication Technologies through a computer literacy course , 2010, Comput. Educ..

[21]  Michalis Nik Xenos,et al.  Perceived Usability Evaluation of Learning Management Systems: A First Step towards Standardization of the System Usability Scale in Greek , 2012, 2012 16th Panhellenic Conference on Informatics.

[22]  Aliana Pereira Simões,et al.  The ergonomic evaluation of a virtual learning environment usability. , 2012, Work.

[23]  Kraig Finstad.  The system usability scale and non-native English speakers , 2006 .

[24]  James R. Lewis,et al.  Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ , 1991, SGCH.

[25]  José A. Gallud,et al.  Collaborative e-Learning through Drag & Share in Synchronous Shared Workspaces , 2013, J. Univers. Comput. Sci..

[26]  James R. Lewis.  Psychometric Evaluation of the Post-Study System Usability Questionnaire: The PSSUQ , 1992 .

[27]  Christophe Batier,et al.  Improving web site usability for a better e-learning experience , 2006 .

[28]  Nikolaos Avouris,et al.  Design and Evaluation of Web-based Learning Environments using Information Foraging Models , 2008 .