The effective combination of hybrid usability methods in evaluating educational applications of ICT: Issues and challenges

This paper focuses on usability evaluation of information and communication technologies applications in education (ICTE applications). Various classes of teaching and learning systems are discussed in terms of technologies used and pedagogical approaches. Their usability is analyzed according to various dimensions and the impact of system usability on the learning effectiveness is studied. We argue that various classes of ICTE applications such as multimedia/hypermedia applications, open educational environments and CSCL environments, based on different theoretical perspectives, require fundamentally different approaches in evaluating their usability. The paper is structured as follows: an overview of different usability evaluation approaches is presented first, followed by a discussion on applicability of these techniques in various categories of teaching and learning computer systems. Typical case studies that engage both usability experts and users themselves (students and teachers) are also discussed. The objective is to describe both the methods, and the way to apply them effectively in order to certify the usability of an ICTE application with respect to its teaching and learning objectives.

[1]  Jesse M. Heines.  Evaluating the effect of a course Web site on student performance , 2000, J. Comput. High. Educ..

[2]  T. Landauer,et al.  Handbook of Human-Computer Interaction , 1997 .

[3]  Tonya Barrier,et al.  Human Computer Interaction Development & Management , 2002 .

[4]  Nikolaos Avouris,et al.  Expérimentations sur l'utilisation d'un logiciel de modélisation par petits groupes d'élèves , 2001 .

[5]  P. Dillenbourg,et al.  The evolution of research on collaborative learning , 1996 .

[6]  Mark Guzdial,et al.  Learner-centered design: the challenge for HCI in the 21st century , 1994, INTR.

[7]  Kori Inkpen,et al.  Three Important Research Agendas for Educational Multimedia: Learning, Children, and Gender , 1998 .

[8]  Etienne Wenger,et al.  Communities of Practice: Learning, Meaning, and Identity , 1998 .

[9]  E. Hollnagel.  Handbook of Cognitive Task Design , 2009 .

[10]  Jennifer Preece,et al.  The Effects of Using a Triangulation Approach of Evaluation Methodologies to Examine the Usability of a University Website , 2002 .

[11]  Kamran Sedig,et al.  Role of interface manipulation style and scaffolding on cognition and concept learning in learnware , 2001, TCHI.

[12]  Hans Spada,et al.  Learning in Humans and Machines: Towards an Interdisciplinary Learning Science , 1995 .

[13]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[14]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[15]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[16]  Bernard Blandin.  Usability Evaluation of Online Learning Programs: A Sociological Standpoint , 2003 .

[17]  D. Jonassen.  Thinking Technology: Toward a Constructivist Design Model , 1994 .

[18]  Vassilis Komis,et al.  Design principles for the support of modelling and collaboration in a technology-based learning environment , 2005 .

[19]  Maria Kordaki,et al.  Student Task Modeling in Design and Evaluation of Open Problem-Solving Environments , 2004, Education and Information Technologies.

[20]  E. Wenger.  Communities of Practice: Learning, Meaning, and Identity , 1998 .

[21]  Steven K. Feiner,et al.  Computer graphics: principles and practice (2nd ed.) , 1990 .

[22]  Bruce Gurney,et al.  Constructivism and Professional Development: A Stereoscopic View. , 1989 .

[23]  Clare-Marie Karat,et al.  A comparison of user interface evaluation methods , 1994 .

[24]  John Karat,et al.  Software Evaluation Methodologies , 1988 .

[25]  Nikolaos M. Avouris,et al.  On analysis of collaborative problem solving: an object-oriented approach , 2003, Comput. Hum. Behav..

[26]  Jennifer Preece,et al.  Predicting quality in educational software: Evaluating For Learning, Usability and the Synergy between Them , 1999, Interact. Comput..

[27]  Cathleen Wharton,et al.  Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces , 1990, CHI '90.

[28]  Dan Diaper,et al.  Task Analysis for Human-Computer Interaction , 1990 .

[29]  B. Rogoff.  Developing understanding of the idea of communities of learners , 2009 .

[30]  Claude Ghaoui,et al.  Usability Evaluation of Online Learning Programs , 2003 .

[31]  Chris J. H. Fowler,et al.  Learning technology and usability: a framework for understanding courseware , 1999, Interact. Comput..

[32]  Carl Gutwin,et al.  The mechanics of collaboration: developing low cost usability evaluation methods for shared workspaces , 2000, Proceedings IEEE 9th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE 2000).

[33]  Nikolaos M. Avouris,et al.  Development and evaluation of a computer‐based laboratory teaching tool , 2001, Comput. Appl. Eng. Educ..

[34]  Nikolaos Avouris,et al.  Evaluation of Distance-Learning Environments: Impact of Usability on Student Performance , 2001 .