Evaluation and measurement of Learning Management System based on user experience

Learning Management System (LMS) becomes more accessible to provide a virtual learning environment. The growth of pervasive and ubiquitous computing promotes development and improvement LMS rapidly. Shifting paradigm in designing LMS from technology-centered to user-centered design drives the developer to consider user experience as the primary concern to improve LMS. Therefore, evaluation and measurement of user experience for LMS become crucial. There were some studies have been conducted to evaluate and measure an LMS product in to gain feedback from the user. However, the studies implement a partial view of LMS product. They have conducted researches that limited to comparing LMS and getting perception rather than observing the quality of the LMS. This paper proposed an evaluation and measurement of user experience (UX) for LMS by implementing user experience questionnaire tools. The study aims to get feedback, benchmark and identify UX while interacting with an LMS. The data are taken from 38 students who have experience using two LMS in a different semester. The first LMS based on Moodle was implemented in the first semester (LMS 1), on the other hand, the LMS using Schoology was applied in the second semester (LMS 2). As the results, LMS 1 and LMS 2 got the relative different impression in user perception. Similarly, they also achieved a different level of relative quality. Generally, evaluation and measurement LMS using user experience questionnaire not only provide comprehensive user perception but also provide relative quality level about LMS. The results give the feedback and support when implementing the LMS.

[1]  M. Machado,et al.  Blackboard vs. moodle: Comparing user experience of learning management systems , 2007, 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports.

[2]  Martin Schrepp,et al.  Construction and Evaluation of a User Experience Questionnaire , 2008, USAB.

[3]  Harry Budi Santoso,et al.  Measuring User Experience of the Student-Centered e-Learning Environment , 2016 .

[4]  Manuel Pérez Cota,et al.  Efficient Measurement of the User Experience. A Portuguese Version , 2013, DSAI.

[5]  Kamlesh Dutta,et al.  A comparative study of moodle with other e-learning systems , 2011, 2011 3rd International Conference on Electronics Computer Technology.

[6]  Krešimir Fertalj,et al.  Comparison of E-Learning Management Systems , 2006 .

[7]  Liang Jingjing,et al.  Design of model for activity-centered web learning and user experience , 2010, 2010 International Conference on Artificial Intelligence and Education (ICAIE).

[8]  M. Kljun,et al.  Evaluating Comparisons and Evaluations of Learning Management Systems , 2007, 2007 29th International Conference on Information Technology Interfaces.

[9]  Vincent P. Wade,et al.  A Web-Based Framework for User-Centred Evaluation of End-User Experience in Adaptive and Personalized E-Learning Systems , 2011, 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[10]  Jason Cole,et al.  Using moodle, 2nd edition , 2007 .

[11]  T. Robal,et al.  Learning from users for a better and personalized web experience , 2012, 2012 Proceedings of PICMET '12: Technology Management for Emerging Technologies.

[12]  Zafer Unal,et al.  Evaluating and Comparing the Usability of Web-Based Course Management Systems , 2011 .

[13]  Chang-ai Sun,et al.  A comparative study of open-source learning management systems , 2011, 2011 IEEE International Workshop on Open-source Software for Scientific Computation.