Evaluating e-Learning Initiatives: A Literature Review on Methods and Research Frameworks

Evaluation aspects, in relation to e-learning initiatives, are gaining substantial attention. As technology continuously influences learning, technical as well as organizational requirements need to be thoroughly investigated across a variety of stakeholders. In this paper, an outline of those aspects is presented, which occurred from a literature review on methods and research frameworks utilized toward the evaluation of e-learning initiatives. The review identified a series of studies that take advantage of well-established theories in the area of users' acceptance of technology combined with additional, e-learning context-specific factors. Results of the review are presented, according to the adopted research model, to ease the process of locating and retrieving e-learning evaluation paradigms per theoretical model. In addition, research findings are discussed and future implications for e-learning evaluation initiatives as well as potential stakeholders are highlighted.

[1]  Stelios Daskalakis,et al.  Assessing the Acceptance of a Blended Learning University Course , 2011, J. Educ. Technol. Soc..

[2]  Matthew K. O. Lee,et al.  Acceptance of Internet-based learning medium: the role of extrinsic and intrinsic motivation , 2005, Inf. Manag..

[3]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .

[4]  Sevgi Özkan,et al.  Multi-dimensional students' evaluation of e-learning systems in the higher education context: An empirical investigation , 2009, Comput. Educ..

[5]  Katelyn Y. A. McKenna,et al.  Oxford Handbook of Internet Psychology , 2007 .

[6]  Yesha Y. Sivan,et al.  Enterprise E-Learning Success Factors: An Analysis of Practitioners' Perspective (with a Downturn Addendum) , 2009 .

[7]  Milena M. Head,et al.  Who is Responsible for E-Learning Success in Higher Education? A Stakeholders' Analysis , 2008, J. Educ. Technol. Soc..

[8]  Raafat George Saadé,et al.  Viability of the "Technology Acceptance Model" in Multimedia Learning Environments: A Comparative Study , 2007 .

[9]  Clyde W. Holsapple,et al.  Defining, Assessing, and Promoting E‐Learning Success: An Information Systems Perspective* , 2006 .

[10]  Kamran Sedig,et al.  Role of interface manipulation style and scaffolding on cognition and concept learning in learnware , 2001, TCHI.

[11]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[12]  Nikolaos Avouris,et al.  Design and Evaluation of Web-based Learning Environments using Information Foraging Models , 2008 .

[13]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[14]  Ellen B. Mandinach,et al.  The Development of Effective Evaluation Methods for E-Learning: A Concept Paper and Action Plan , 2005, Teachers College Record: The Voice of Scholarship in Education.

[15]  Yi-Shun Wang,et al.  Measuring e-learning systems success in an organizational context: Scale development and validation , 2007, Comput. Hum. Behav..

[16]  Su-Chao Chang,et al.  An empirical investigation of students' behavioural intentions to use the online learning course websites , 2007, Br. J. Educ. Technol..

[17]  Deborah Compeau,et al.  Computer Self-Efficacy: Development of a Measure and Initial Test , 1995, MIS Q..

[18]  Nazife Baykal,et al.  Evaluating learning management systems , 2009 .

[19]  Chorng-Shyong Ong,et al.  Gender differences in perceptions and relationships among dominants of e-learning acceptance , 2006, Comput. Hum. Behav..

[20]  Erik M. van Raaij,et al.  The acceptance and use of a virtual learning environment in China , 2008, Comput. Educ..

[21]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[22]  Nitza Geri,et al.  Beyond Adoption: Barriers to an Online Assignment Submission System Continued Use , 2008 .

[23]  Mark Guzdial,et al.  Learner-centered design: the challenge for HCI in the 21st century , 1994, INTR.

[24]  Barbara Means,et al.  Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies , 2009 .

[25]  Keenan A. Pituch,et al.  The influence of system characteristics on e-learning use , 2006, Comput. Educ..

[26]  Marylène Gagné,et al.  Understanding e-learning continuance intention in the workplace: A self-determination theory perspective , 2008, Comput. Hum. Behav..

[27]  Guoqing Chen,et al.  Extended Information Technology Initial Acceptance Model and Its Empirical Test , 2007 .

[28]  Susan Wiedenbeck,et al.  Measuring Online Trust of Websites: Credibility, Perceived Ease of Use, and Risk , 2005, AMCIS.

[29]  Gwo-Dong Chen,et al.  Surveying instructor and learner attitudes toward e-learning , 2007, Comput. Educ..

[30]  Hassan M. Selim,et al.  An empirical investigation of student acceptance of course websites , 2003, Comput. Educ..

[31]  Su-Chao Chang,et al.  A new hybrid model for exploring the adoption of online nursing courses. , 2008, Nurse education today.

[32]  Dowming Yeh,et al.  What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction , 2008, Comput. Educ..

[33]  D'Arcy A. Becker,et al.  Using hypermedia to provide learner control , 1994 .

[34]  Nikolaos M. Avouris,et al.  The effective combination of hybrid usability methods in evaluating educational applications of ICT: Issues and challenges , 2008, Education and Information Technologies.

[35]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[36]  Sung Youl Park,et al.  An Analysis of the Technology Acceptance Model in Understanding University Students' Behavioral Intention to Use e-Learning , 2009, J. Educ. Technol. Soc..

[37]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[38]  N. Geri,et al.  Beyond Adoption: Barriers to an Online Assignment Submission System Continued Use , 2008 .

[39]  Jörg M. Haake,et al.  Hypermedia and cognition: designing for comprehension , 1995, CACM.

[40]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[41]  Charith Nanayakkara,et al.  A Model of User Acceptance of Learning Management Systems , 2006 .

[42]  Eric T. G. Wang,et al.  Understanding Web-based learning continuance intention: The role of subjective task value , 2008, Inf. Manag..

[43]  Chorng-Shyong Ong,et al.  Factors affecting engineers' acceptance of asynchronous e-learning systems in high-tech companies , 2004, Inf. Manag..

[44]  Hans van der Heijden,et al.  User Acceptance of Hedonic Information Systems , 2004, MIS Q..

[45]  Nikolaos Avouris,et al.  Evaluation of Distance-Learning Environments: Impact of Usability on Student Performance , 2001 .

[46]  Xin He,et al.  Exploring dimensions to online learning , 2007, Comput. Hum. Behav..

[47]  T. Teo.  Development and validation of the E-learning Acceptance Measure (ElAM) , 2010, Internet High. Educ..

[48]  Euripidis Loukis,et al.  A Value Flow Model for the Evaluation of an E-Learning Service , 2007, ECIS.

[49]  Eric W. T. Ngai,et al.  Empirical examination of the adoption of WebCT using TAM , 2007, Comput. Educ..

[50]  Christina Keller,et al.  Virtual learning environments: three implementation perspectives , 2005 .

[51]  Shu-Sheng Liaw,et al.  Investigating students' perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system , 2008, Comput. Educ..

[52]  Nikolaos Avouris,et al.  Cognitive Task modelling for system design and evaluation in non-routine task domains , 2002 .

[53]  Hsiu-Fen Lin,et al.  Measuring Online Learning Systems Success: Applying the Updated DeLone and McLean Model , 2007, Cyberpsychology Behav. Soc. Netw..

[54]  Hassan M. Selim,et al.  Critical success factors for e-learning acceptance: Confirmatory factor models , 2007, Comput. Educ..

[55]  Steven Hornik,et al.  An empirical examination of factors contributing to the creation of successful e-learning environments , 2008, Int. J. Hum. Comput. Stud..

[56]  Yan Xu,et al.  An Enhanced Technology Acceptance Model for Web-Based Learning , 2004, J. Inf. Syst. Educ..

[57]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[58]  Raafat George Saadé,et al.  The impact of cognitive absorption on perceived usefulness and perceived ease of use in on-line learning: an extension of the technology acceptance model , 2005, Inf. Manag..