Experience with usability evaluation of e-learning systems

Progress in the field of e-learning has been slow, with related problems mainly associated with the poor design of e-learning systems. Moreover, because of a depreciated importance of usability, usability studies are not very frequent. This paper reports the experience with the usability assessment of intelligent learning and teaching systems which are based on TEx-Sys model and are intended to enhance the process of knowledge acquisition in daily classroom settings. The applied scenario-based usability evaluation, as a combination of behaviour and opinion based measurements, enabled to quantify usability in terms of users’ (teachers’ and students’) performance and satisfaction. According to the achieved results, the main directions for interface redesign are offered. The acquired experience indicates that useful usability assessments with a significant identification of interface limitations can be performed quite easily and quickly. On the other hand, it raised a series of questions which, in order to be clarified, require further comprehensive research, the more so if the employment of universal design within e-learning context is considered.

[1]  Angeliki Poulymenakou,et al.  Designing On-Line Learning Courses : Implications for Usability , 2002 .

[2]  Peter C. Wright,et al.  A Cost-Effective Evaluation Method for Use by Designers , 1991, Int. J. Man Mach. Stud..

[3]  ZahariasPanagiotis.  Usability and e-learning , 2004 .

[4]  Panagiotis Zaharias.  E-Learning Design Quality , 2005 .

[5]  Ann Quigley.  Usability-tested e-learning? not until the market requires it , 2002, ELERN.

[6]  P. D Harper,et al.  Improving user satisfaction: the questionnaire for user interaction satisfaction version 5 , 1993 .

[7]  Martha C. Polson,et al.  Foundations of intelligent tutoring systems , 1988 .

[8]  Mary Beth Rosson,et al.  Usability Specifications as Tool in Iterative Development. , 1984 .

[9]  Elliot Soloway,et al.  Learner-centered design: addressing, finally, the unique needs of learners , 1998, CHI Conference Summary.

[10]  H. Chertkow,et al.  Semantic memory , 2002, Current neurology and neuroscience reports.

[11]  Andreas Holzinger,et al.  Rapid prototyping for a virtual medical campus interface , 2004, IEEE Software.

[12]  Michael Feldstein.  What is usable e-learning? , 2002, ELERN.

[13]  Tom Murray,et al.  Authoring Intelligent Tutoring Systems: An analysis of the state of the art , 1999 .

[14]  Jaime R. Carbonell,et al.  AI in CAI : An artificial intelligence approach to computer-assisted instruction , 1970 .

[15]  David Sloan,et al.  Contextual web accessibility - maximizing the benefit of accessibility guidelines , 2006, W4A '06.

[16]  Brian Shackel,et al.  Usability - Context, framework, definition, design and evaluation , 1991, Interact. Comput..

[17]  G. Salvendy,et al.  Toward an Information Society for All: An International R&D Agenda , 1998 .

[18]  Teresa Roselli,et al.  An approach to usability evaluation of e-learning applications , 2006, Universal Access in the Information Society.

[19]  Andreas Holzinger,et al.  Usability engineering methods for software developers , 2005, CACM.

[20]  Jeff Rickel,et al.  Intelligent computer-aided instruction: a survey organized around system components , 1989, IEEE Trans. Syst. Man Cybern..

[21]  Brian Shackel,et al.  Human factors for informatics usability , 1991 .

[22]  Ashok Patel,et al.  Intelligent and Adaptive Systems , 2002 .

[23]  Ben Shneiderman,et al.  Designing The User Interface , 2013 .

[24]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[25]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[26]  Ashok Patel.  KNOWLEDGE CHARACTERISTICS: RECONSIDERING THE DESIGN OF INTELLIGENT TUTORING SYSTEMS , 1996 .

[27]  Robert D. Sherwood.  Models of Computer Use in School Settings. Technical Report Series, Report No. 84.2.2. , 1984 .

[28]  Panagiotis Zaharias.  Usability and e-learning: the road towards integration , 2004, ELERN.

[29]  Miles MacLeod,et al.  Usability measurement in context , 1994, Behav. Inf. Technol..

[30]  L. Faulkner.  Beyond the five-user assumption: Benefits of increased sample sizes in usability testing , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[31]  Stuart C. Shapiro,et al.  Encyclopedia of artificial intelligence, vols. 1 and 2 (2nd ed.) , 1992 .

[32]  Jenny Preece,et al.  Distance learning: a CHI 97 special interest group , 1997, SGCH.

[33]  Marvin Minsky,et al.  Semantic Information Processing , 1968 .

[34]  Mark Notess,et al.  Usability, user experience, and learner experience , 2001, ELERN.

[35]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[36]  Sebastiano Bagnara,et al.  Multimedia systems in distance education: effects of usability on learning , 1999, Interact. Comput..

[37]  Jennifer Preece,et al.  Usability and learning: Evaluating the potential of educational software , 1996, Comput. Educ..

[38]  Miles MacLeod,et al.  The MUSiC performance measurement method , 1997, Behav. Inf. Technol..

[39]  Frederick G. Conrad,et al.  Usability testing of world wide web sites , 1997, CHI Extended Abstracts.

[40]  Tiziana Catarci,et al.  A proposal toward the development of accessible e-learning content by human involvement , 2006, Universal Access in the Information Society.

[41]  W. Buxton.  Human-Computer Interaction , 1988, Springer Berlin Heidelberg.

[42]  Heimo H. Adelsberger,et al.  Handbook on information technologies for education and training: with 34 tables , 2008, International handbooks on information systems.

[43]  Peter Szolovits,et al.  What Is a Knowledge Representation? , 1993, AI Mag..

[44]  John D. Bransford,et al.  Computer Strategies for Education: Foundations and Content-Area Applications , 1986 .

[45]  Harold F. O'Neil,et al.  Technology Applications in Education: A Learning View , 2002 .

[46]  Hugh L. Burns,et al.  Foundations of intelligent tutoring systems : an introduction , 1988 .

[47]  Laurie P. Dringus.  An Iterative Usability Evaluation Procedure for Interactive Online Courses , 1995 .

[48]  Notes from e-learning Special Interest Group (SIG) discussion at CHI 2001: Seattle, Washington, April 3, 2001 , 2001, ELERN.

[49]  Allan Collins,et al.  The Role of Different Media in Designing Learning Environments , 2000 .

[50]  Teresa Roselli,et al.  On the Usability Evaluation of E-Learning Applications , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[51]  Arthur B. Markman,et al.  Knowledge Representation , 1998 .

[52]  Gavriel Salvendy,et al.  Toward an Information Society for All: An International Research and Development Agenda , 1998, Int. J. Hum. Comput. Interact..

[53]  Jennifer Preece,et al.  Predicting quality in educational software: Evaluating For Learning, Usability and the Synergy between Them , 1999, Interact. Comput..

[54]  Nigel Bevan.  Usability Issues in Web Site Design , 1997, HCI.

[55]  Marshall G. Jones,et al.  User Interface Design Guidelines for Computer-Based Learning Environments , 1999 .