A Value Flow Model for the Evaluation of an E-Learning Service

Information Systems (IS) evaluation has been one of the most important and widely researched topics for more than two decades. Τhe evaluation of e-Learning is of critical importance in order to maximize the benefits from the huge investments made in this area. In this paper is presented a method for e-learning multi-layer evaluation and value flow model estimation. It includes evaluation of e-learning using three layers of value measures (efficiency measures, effectiveness measures, and intended future behaviour measures), and also the estimation of the relations between them. In this way a model of value generation and flow is developed. This method generates rich and useful information concerning the different types of value that e-learning creates, the mechanisms of their creation and also possible interventions for increasing them. A first application of this method is presented for the evaluation of an e-learning service in the area of the European cultural heritage that has been developed in the project e-RMIONE of the eTEN Programme of the European Union. This first application gives interesting and useful conclusions concerning the various levels of value created by the above service, as well as the generation and flow of value between these levels, and also it enables the prioritization of the resources and capabilities of this service that should be improved. In this sense this application provides positive and encouraging evidence concerning the validity and the usefulness of the method.

[1]  B. Bloom,et al.  Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , 1966 .

[2]  Kevin F. Collis,et al.  Evaluating the Quality of Learning: The SOLO Taxonomy , 1977 .

[3]  William E. Cashin,et al.  IDEA Technical Report No. 1. Development of the IDEA System. , 1977 .

[4]  Herbert W. Marsh,et al.  SEEQ: A Reliable, Valid, and Useful Instrument for Collecting Students' Evaluations of University Teaching. , 1982 .

[5]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[6]  Walter Van de Velde,et al.  Quality of Learning , 1988, ECAI.

[7]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[8]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[9]  William E. Cashin,et al.  Using global student rating items for summative evaluation. , 1992 .

[10]  Adamantios Diamantopoulos,et al.  Modelling with LISREL: A guide for the uninitiated , 1994 .

[11]  David Targett,et al.  A taxonomy of information systems applications: the benefits' evaluation ladder , 1995 .

[12]  Christina Soh,et al.  How IT Creates Business Value: A Process Theory Synthesis , 1995, ICIS.

[13]  Peter B. Seddon A Respecification and Extension of the DeLone and McLean Model of IS Success , 1997, Inf. Syst. Res..

[14]  Barry Jackson Evaluation of Learning Technology Implementation Purposes and Scope of Evaluation , 1998 .

[15]  William J. Doll,et al.  Developing a multidimensional measure of system-use in an organizational context , 1998, Inf. Manag..

[16]  R. Hirschheim,et al.  Analysing information systems evaluation: another look at an old problem , 1998 .

[17]  Gc Conole,et al.  A Practical Approach to the Evaluation of Learning Technology: Tools for Practitioners , 1998 .

[18]  David Targett,et al.  IS evaluation: a process for bringing together benefits, costs and risks , 1999 .

[19]  Detmar W. Straub,et al.  Structural Equation Modeling and Regression: Guidelines for Research Practice , 2000, Commun. Assoc. Inf. Syst..

[20]  Thierry Volery,et al.  Critical success factors in online education , 2000 .

[21]  Motoei Azuma SquaRE The next generation of the ISO/IEC 9126 and 14598 international standards series on software product quality , 2001 .

[22]  Hock Chuan Chan,et al.  Critical success factors for on-line course resources , 2001, Comput. Educ..

[23]  Zahir Irani,et al.  Information systems evaluation: navigating through the problem domain , 2002, Inf. Manag..

[24]  Hassan M. Selim,et al.  An empirical investigation of student acceptance of course websites , 2003, Comput. Educ..

[25]  Yi-Shun Wang,et al.  Assessment of learner satisfaction with asynchronous electronic learning systems , 2003, Inf. Manag..

[26]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[27]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[28]  Zahir Irani,et al.  An exploratory study of information technology evaluation and benefits management practices of SMEs in the construction industry , 2004, Inf. Manag..

[29]  Raafat George Saadé,et al.  The impact of cognitive absorption on perceived usefulness and perceived ease of use in on-line learning: an extension of the technology acceptance model , 2005, Inf. Manag..

[30]  Angappa Gunasekaran,et al.  Information technology and systems justification: A review for research and applications , 2006, Eur. J. Oper. Res..

[31]  William K. Horton Evaluating e-learning , 2006 .

[32]  Hassan M. Selim,et al.  Critical success factors for e-learning acceptance: Confirmatory factor models , 2007, Comput. Educ..

[33]  Eric W. T. Ngai,et al.  Empirical examination of the adoption of WebCT using TAM , 2007, Comput. Educ..

[34]  U. Ehlers Quality in e-Learning from a Learner’s Perspective , 2004, Distances et médiations des savoirs.