A model to measure e-learning systems success

E-learning involves adopting and exploiting the potential of new, advanced Information Technology in development and delivery of education. In spite of a rapid growth in the e-learning field there still exists a range of issues facing the stakeholders of e-learning systems. One of the key issues is how to measure e-learning system success. Although considerable attention has been paid to the Information Systems success issue, there remain arguments about the factors which are most effective for measuring Information System success. The issue of measuring Information System success has an impact on evaluating e-learning systems success. This chapter aims to fill this void by proposing an evaluation methodology model to assess e-learning systems success. The contribution of this study is the proposed model to evaluate the success of e-learning systems. The model is based on a thorough review of the e-learning success literature and existing Information Systems success models.

[1]  A. P. Rovai,et al.  Why some distance education programs fail while others succeed in a global environment , 2010, Internet High. Educ..

[2]  Luís Kalb Roses,et al.  Management of perceptions of information technology service quality , 2009 .

[3]  C. McMahon,et al.  Understanding information systems infrastructure in engineering SMEs: A case study , 2010 .

[4]  Emma J. Stodel,et al.  The demand-driven learning model: A framework for Web-based learning , 2001, Internet High. Educ..

[5]  Kjetil L. Nielsen,et al.  Experiences Obtained with Integration of Student Response Systems for iPod Touch and iPhone into e-Learning Environments , 2010 .

[6]  D. Harrison McKnight,et al.  Perceived Information Quality in Data Exchanges: Effects on Risk, Trust, and Intention to Use , 2006, Inf. Syst. Res..

[7]  James J. Jiang,et al.  IT Investment Strategy And IT Infrastructure Services , 2011, BIS 2011.

[8]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[9]  Victor R. Prybutok,et al.  The moderating effect of occupation on the perception of information services quality and success , 2010, Comput. Ind. Eng..

[10]  Stephen M. Mutula,et al.  A proposed model for evaluating the success of WebCT course content management system , 2010, Comput. Hum. Behav..

[11]  Kim S. Cameron,et al.  Domains of Organizational Effectiveness in Colleges and Universities , 1981 .

[12]  D. Brigham Converting Student Support Services to Online Delivery , 2001 .

[13]  Kenneth L. Kraemer,et al.  Executives’ Perceptions of the Business Value of Information Technology: A Process-Oriented Approach , 2000, J. Manag. Inf. Syst..

[14]  Peter Weill,et al.  What IT Infrastructure Capabilities are Needed to Implement E-Business Models? , 2002, MIS Q. Executive.

[15]  William R. King,et al.  The development of global IT infrastructure , 2008 .

[16]  Young A. Park,et al.  Success Factors for Destination Marketing Web Sites: A Qualitative Meta-Analysis , 2007 .

[17]  José Bidarra,et al.  Current Developments and Best Practice in Open and Distance Learning , 2000 .

[18]  Varun Grover,et al.  Building and leveraging information in dynamic environments: The role of IT infrastructure flexibility as enabler of organizational responsiveness and competitive advantage , 2010, Inf. Manag..

[19]  Peter Mayall,et al.  Educational and institutional flexibility of Australian educational software , 2008 .

[20]  Mo Adam Mahmood,et al.  System Development Methods - A Comparative Investigation , 1987, MIS Q..

[21]  Fatemeh Zahedi,et al.  The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach , 2002, Inf. Syst. Res..

[22]  Richard T. Watson,et al.  Service Quality: A Measure of Information System Effectiveness , 1995, MIS Q..

[23]  Gary Klein,et al.  Relation of CIO background, IT infrastructure, and economic performance , 2009, Inf. Manag..

[24]  Ali F. Farhoomand,et al.  A structural model of end user computing satisfaction and user performance , 1996, Inf. Manag..

[25]  Darshana Sedera,et al.  Re-conceptualizing Information System Success: The IS-Impact Measurement Model , 2008, J. Assoc. Inf. Syst..

[26]  S. Pather E-commerce information systems (ECIS) success: A South African study , 2006 .

[27]  P. Weill,et al.  IT Infrastructure for Strategic Agility , 2002 .

[28]  J. Miller,et al.  Measuring the Effectiveness of Computer-Based Information Systems in the Financial Services Sector , 1987, MIS Q..

[29]  P. D. D. Dominic,et al.  Towards Digital Economy: The Development of ICT and E-Commerce in Malaysia , 2011 .

[30]  Yi-Shun Wang,et al.  Multi-criteria evaluation of the web-based e-learning system: A methodology based on learner satisfaction and its applications , 2008, Comput. Educ..

[31]  Iris Vessey,et al.  The Effect of User Involvement on System Success: A Contingency Approach , 1988, MIS Q..

[32]  Chorng-Shyong Ong,et al.  Measuring user satisfaction with knowledge management systems: scale development, purification, and initial test , 2007, Comput. Hum. Behav..

[33]  Chang Liu,et al.  Exploring the factors associated with Web site success in the context of electronic commerce , 2000, Inf. Manag..

[34]  In Lee,et al.  Learners' acceptance of e-learning in South Korea: Theories and results , 2009, Comput. Educ..

[35]  Sherry D. Ryan,et al.  Information-Technology Investment Decisions: When Do Costs and Benefits in the Social Subsystem Matter? , 2002, J. Manag. Inf. Syst..

[36]  Eric W. T. Ngai,et al.  Empirical examination of the adoption of WebCT using TAM , 2007, Comput. Educ..

[37]  Aileen Cater-Steel,et al.  An evaluation of software development practice and assessment-based process improvement in small software development firms , 2004 .

[38]  B. Dale,et al.  Service quality and e‐commerce: an exploratory analysis , 2001 .

[39]  Viswanath Venkatesh,et al.  Technology Acceptance Model 3 and a Research Agenda on Interventions , 2008, Decis. Sci..

[40]  Linda C. Smith,et al.  A framework for information quality assessment , 2007 .

[41]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[42]  Chun-Chieh Wang,et al.  An empirical study of instructor adoption of web-based learning systems , 2009, Comput. Educ..

[43]  Dowming Yeh,et al.  What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction , 2008, Comput. Educ..

[44]  Arun Rai,et al.  Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis , 2002, Inf. Syst. Res..

[45]  Narasimhaiah Gorla,et al.  Determinants of software quality: A survey of information systems project managers , 2010, Inf. Softw. Technol..

[46]  R. Amit,et al.  Value creation in E‐business , 2001 .

[47]  Jia Frydenberg,et al.  Quality Standards in eLearning: A Matrix of Analysis. , 2002 .

[48]  A. Lederer,et al.  An Instrument for Assessing the Organizational Benefits of IS Projects , 1998 .

[49]  Kim S. Cameron,et al.  A Study of Organizational Effectiveness and Its Predictors. ASHE 1983 Annual Meeting Paper. , 1983 .

[50]  Andrew R. Brown,et al.  Elements of Effective e-Learning Design , 2005 .

[51]  A. Parasuraman,et al.  Listening to the Customer -The Concept of a Service-Quality Information System , 1997 .

[52]  Hans H. Bauer,et al.  eTransQual: A Transaction Process-Based Approach for Capturing Service Quality in Online Shopping , 2006 .

[53]  Graeme G. Shanks,et al.  Developing a Measurement Instrument for Subjective Aspects of Information Quality , 2008, Commun. Assoc. Inf. Syst..

[54]  Colla J. MacDonald,et al.  Structure, Content, Delivery, Service, and Outcomes: Quality e-Learning in higher education , 2005 .

[55]  Milena M. Head,et al.  Who is Responsible for E-Learning Success in Higher Education? A Stakeholders' Analysis , 2008, J. Educ. Technol. Soc..

[56]  Diane M. Strong,et al.  Information quality benchmarks: product and service performance , 2002, CACM.

[57]  Narasimhaiah Gorla,et al.  Organizational impact of system quality, information quality, and service quality , 2010, J. Strateg. Inf. Syst..

[58]  Dan Jong Kim,et al.  Customer self-service systems: The effects of perceived Web quality with service contents on enjoyment, anxiety, and e-trust , 2007, Decis. Support Syst..

[59]  Zheng Zhou,et al.  Development and validation of an instrument to measure user perceived service quality of information presenting Web portals , 2005, Inf. Manag..

[60]  Diane M. Strong,et al.  AIMQ: a methodology for information quality assessment , 2002, Inf. Manag..

[61]  Ron Oliver,et al.  Quality assurance and e-learning: blue skies and pragmatism , 2005 .

[62]  J. Strother,et al.  An Assessment of the Effectiveness of e-learning in Corporate Training Programs , 2002 .

[63]  Mark A. Harris,et al.  A New Look at the Relationship between User Involvement in Systems Development and System Success , 2009, Communications of the Association for Information Systems.

[64]  Jung-Yu Lai,et al.  Assessment of employees' perceptions of service quality and satisfaction with e-business , 2006, SIGMIS CPR '06.

[65]  Lior Fink,et al.  Exploring the perceived business value of the flexibility enabled by information technology infrastructure , 2009, Inf. Manag..

[66]  Younghwa Lee,et al.  Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach , 2006, Decis. Support Syst..

[67]  W. Smart Information system success: evaluation of a carbon accounting and sequestration system , 2009 .

[68]  Daniel Berg,et al.  Service quality from the other side: Information systems management at Duquesne Light , 2005, Int. J. Inf. Manag..

[69]  Szu-Yuan Sun,et al.  Usability, quality, value and e-learning continuance decisions , 2005, Comput. Educ..

[70]  J. Arbaugh How Classroom Environment and Student Engagement Affect Learning in Internet-based MBA Courses , 2000 .

[71]  I. Ajzen,et al.  The Prediction of Behavioral Intentions in a Choice Situation. , 1969 .

[72]  Olivia R. Liu Sheng,et al.  e-SELFQUAL: A scale for measuring online self-service quality , 2011 .

[73]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[74]  William J. Kettinger,et al.  Zones of Tolerance: Alternative Scales for Measuring Information Systems Service Quality , 2005, MIS Q..

[75]  Shu-Sheng Liaw,et al.  Investigating students' perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system , 2008, Comput. Educ..

[76]  Carla L. Wilkin,et al.  Towards an integrated view of IT value measurement , 2010, Int. J. Account. Inf. Syst..

[77]  N. Pliskin,et al.  Student LMS use and satisfaction in academic institutions: The organizational perspective , 2010, Internet High. Educ..

[78]  Mie Nakatani,et al.  Information filtering for emergency management , 2000, Proceedings 9th IEEE International Workshop on Robot and Human Interactive Communication. IEEE RO-MAN 2000 (Cat. No.00TH8499).

[79]  Vincent Cho,et al.  Comparing e-Learning tools' success: The case of instructor-student interactive vs. self-paced tools , 2011, Comput. Educ..

[80]  Fred D. Davis A technology acceptance model for empirically testing new end-user information systems : theory and results , 1985 .

[81]  Gwo-Dong Chen,et al.  Surveying instructor and learner attitudes toward e-learning , 2007, Comput. Educ..

[82]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[83]  Thurasamy Ramayah,et al.  The role of quality factors in intention to continue using an e-learning system in Malaysia , 2010 .

[84]  Clyde W. Holsapple,et al.  Defining, Assessing, and Promoting E‐Learning Success: An Information Systems Perspective* , 2006 .

[85]  Linda C. Smith,et al.  Information quality work organization in wikipedia , 2008 .

[86]  Antonio Mauro Saraiva,et al.  An infrastructure for the development of distributed service-oriented information systems for precision agriculture , 2007 .

[87]  Blake Ives,et al.  An empirical study of the impact of user involvement on system usage and information satisfaction , 1986, CACM.

[88]  Yi-Shun Wang Assessing e‐commerce systems success: a respecification and validation of the DeLone and McLean model of IS success , 2008, Inf. Syst. J..

[89]  Daniel J. Power,et al.  AN EMPIRICAL ASSESSMENT OF COMPUTER‐ASSISTED DECISION ANALYSIS* , 1986 .

[90]  Yi-Shun Wang,et al.  The conceptualization and measurement of m-commerce user satisfaction , 2004, Comput. Hum. Behav..

[91]  Ian C. Reid Quality Assurance, Open and Distance Learning, and Australian Universities , 2005 .

[92]  Sevgi Özkan,et al.  Multi-dimensional students' evaluation of e-learning systems in the higher education context: An empirical investigation , 2009, Comput. Educ..

[93]  Juhani Iivari,et al.  An empirical test of the DeLone-McLean model of information system success , 2005, DATB.

[94]  David Georgina,et al.  Integration of technology in higher education: A review of faculty self-perceptions , 2008, Internet High. Educ..

[95]  Joon Koh,et al.  Knowledge sharing in virtual communities: an e-business perspective , 2004, Expert Syst. Appl..

[96]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[97]  Nigel Melville,et al.  Information technology infrastructure, organizational process redesign, and business value: An empirical analysis , 2010, Decis. Support Syst..

[98]  Diane M. Strong,et al.  Beyond Accuracy: What Data Quality Means to Data Consumers , 1996, J. Manag. Inf. Syst..

[99]  Richard T. Watson,et al.  WebQual: An Instrument for Consumer Evaluation of Web Sites , 2007, Int. J. Electron. Commer..

[100]  Lorin M. Hitt,et al.  Productivity, Business Profitability, and Consumer Surplus: Three Different Measures of Information Technology Value , 1996, MIS Q..

[101]  Maarten Gelderman,et al.  The relation between user satisfaction, usage of information systems and performance , 1998, Inf. Manag..

[102]  Victor Bekkers,et al.  Flexible information infrastructures in Dutch E-Government collaboration arrangements: Experiences and policy implications , 2009, Gov. Inf. Q..

[103]  V. Zeithaml Service excellence in electronic channels , 2002 .

[104]  Victor R. Prybutok,et al.  A service quality and success model for the information service industry , 2004, Eur. J. Oper. Res..

[105]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[106]  Peter B. Seddon A Respecification and Extension of the DeLone and McLean Model of IS Success , 1997, Inf. Syst. Res..

[107]  Teresa Roselli,et al.  An approach to usability evaluation of e-learning applications , 2006, Universal Access in the Information Society.

[108]  Ian Richardson,et al.  Diagnosing information systems success: importance-performance maps in the health club industry , 2001, Inf. Manag..

[109]  Glenn Hardaker,et al.  E‐learning communities, virtual markets and knowledge creation , 2002 .

[110]  Hsiu-Fen Lin,et al.  Measuring Online Learning Systems Success: Applying the Updated DeLone and McLean Model , 2007, Cyberpsychology Behav. Soc. Netw..

[111]  Subhasish Dasgupta,et al.  MEASUREMENT OF USER SATISFACTION WITH WEB-BASED INFORMATION SYSTEMS: AN EMPIRICAL STUDY , 2002 .

[112]  Chao-Min Chiu,et al.  Understanding e-learning continuance intention: An extension of the Technology Acceptance Model , 2006, Int. J. Hum. Comput. Stud..

[113]  Hassan M. Selim,et al.  Critical success factors for e-learning acceptance: Confirmatory factor models , 2007, Comput. Educ..

[114]  James T. C. Teng,et al.  Extended conceptualisation of perceived usefulness: empirical test in the context of information system use continuance , 2012, Behav. Inf. Technol..

[115]  Eric Tsui,et al.  Knowledge management perspective on e-learning effectiveness , 2009, Knowl. Based Syst..

[116]  Meir M. Lehman,et al.  Laws of Software Evolution Revisited , 1996, EWSPT.

[117]  Anita Lee-Post e-Learning Success Model: An Information Systems Perspective. , 2009 .

[118]  Małgorzata Nycz,et al.  Learning Objects and E-Learning: an Informing Science Perspective , 2006 .

[119]  Barbara H Wixom,et al.  A Theoretical Integration of User Satisfaction and Technology Acceptance , 2005, Inf. Syst. Res..

[120]  Susan Y McGorry,et al.  Measuring quality in online programs , 2003, Internet High. Educ..

[121]  Fan-Yun Pai,et al.  Evaluating the business value of RFID: Evidence from five case studies , 2008 .

[122]  Wei-Jaw Deng,et al.  The relationships among service quality, perceived value, customer satisfaction, and post-purchase intention in mobile value-added services , 2009, Comput. Hum. Behav..

[123]  M. Fassnacht,et al.  Quality of Electronic Services , 2006 .

[124]  G. W. Smith CIM overview reveals little: Computer integrated manufacturing by Paul G Ranky. Published by Prentice-Hall. 513pp. £36.95 , 1988 .

[125]  Charles Chowa,et al.  Information System Success: Individual and Organizational Determinants , 2006, Manag. Sci..

[126]  Donna Rogers,et al.  A paradigm shift: Technology integration for higher education in the new millennium , 2000 .

[127]  M. Gilly,et al.  eTailQ: dimensionalizing, measuring and predicting etail quality , 2003 .

[128]  N. Melone A theoretical assessment of the user-satisfaction construct in information systems research , 1990 .

[129]  Younghwa Lee,et al.  The Technology Acceptance Model: Past, Present, and Future , 2003, Commun. Assoc. Inf. Syst..

[130]  Izak Benbasat,et al.  An Investigation of the Effectiveness of Color and Graphical Information Presentation Under Varying Time Constraints , 1986, MIS Q..

[131]  Victor R. Prybutok,et al.  A comparison of Magal's service quality instrument with SERVPERF , 2007, Inf. Manag..

[132]  Tzyh-Lih Hsia,et al.  A study of student satisfaction in a blended e-learning system environment , 2010, Comput. Educ..

[133]  Wanda J. Orlikowski,et al.  Studying Information Technology in Organizations: Research Approaches and Assumptions , 1991, Inf. Syst. Res..

[134]  Hsin Hsin Chang,et al.  The impact of e-service quality, customer satisfaction and loyalty on e-marketing: Moderating effect of perceived value , 2009 .

[135]  Jane E. Klobas,et al.  A task-technology fit view of learning management system impact , 2009, Comput. Educ..

[136]  Papassapa Rauyruen,et al.  Relationship quality as a predictor of B2B customer loyalty , 2007 .

[137]  William R. King,et al.  Determinants of the effectiveness of personal decision support systems , 1986, Inf. Manag..

[138]  Yi-Shun Wang,et al.  Measuring e-learning systems success in an organizational context: Scale development and validation , 2007, Comput. Hum. Behav..

[139]  Mie Nakatani,et al.  An Information Filtering Method for Emergency Management , 2002 .

[140]  Robert W. Zmud,et al.  The Effectiveness of External Information Channels in Facilitating Innovation Within Software Development Groups , 1983, MIS Q..

[141]  Leon A. Kappelman,et al.  Measuring Information Systems Service Quality: Concerns on the Use of the SERVQUAL Questionnaire , 1997, MIS Q..

[142]  Kallol Kumar Bagchi,et al.  An assessment of customers' e-service quality perception, satisfaction and intention , 2010, Int. J. Inf. Manag..

[143]  Shun Cai,et al.  Effects of outcome, process and shopping enjoyment on online consumer behaviour , 2006, Electron. Commer. Res. Appl..

[144]  Keenan A. Pituch,et al.  The influence of system characteristics on e-learning use , 2006, Comput. Educ..

[145]  P. Weill,et al.  Building IT Infrastructure for Strategic Agility , 2002 .

[146]  Shin-Yuan Hung Expert versus novice use of the executive support systems: an empirical study , 2003, Inf. Manag..

[147]  P. Weill,et al.  Management by Maxim: How Business and IT Managers Can Create IT Infrastructures , 1997 .

[148]  Richard T. Watson,et al.  Measuring Information Systems Service Quality: Lessons From Two Longitudinal Case Studies , 1998, MIS Q..

[149]  Ian Roffe,et al.  E‐learning: engagement, enhancement and execution , 2002 .

[150]  Gu Lin Estimation method for manufacturing information quality based on material constraint relationship , 2009 .

[151]  Tor J. Larsen,et al.  The role of task-technology fit as users' motivation to continue information system use , 2009, Comput. Hum. Behav..

[152]  Woong-Kyu Lee,et al.  The relationship of e-Learner's self-regulatory efficacy and perception of e-Learning environmental quality , 2008, Comput. Hum. Behav..

[153]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[154]  Zhilin Yang,et al.  Online service quality dimensions and their relationships with satisfaction: A content analysis of customer reviews of securities brokerage services , 2004 .

[155]  Tugrul U. Daim,et al.  Information technology diffusion in higher education , 2007 .

[156]  Peter B. Seddon,et al.  Assessing and managing the benefits of enterprise systems: the business manager's perspective , 2002, Inf. Syst. J..

[157]  Carla Wilkin,et al.  Development of an instrument to evaluate the quality of delivered systems , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[158]  Jurek Kirakowski,et al.  Measuring user-satisfaction with electronic consumer products: The Consumer Products Questionnaire , 2011, Int. J. Hum. Comput. Stud..

[159]  Lior Fink,et al.  Gaining Agility through IT Personnel Capabilities: The Mediating Role of IT Infrastructure Capabilities , 2007, J. Assoc. Inf. Syst..

[160]  Evelyn H. Thrasher,et al.  A process-oriented perspective of IS success: Examining the impact of IS on operational cost , 2006 .

[161]  C. Flavián,et al.  Perceived e‐service quality (PeSQ) , 2007 .

[162]  Kyu Yon Lim,et al.  Online university students' satisfaction and persistence: Examining perceived level of presence, usefulness and ease of use as predictors in a structural model , 2011, Comput. Educ..

[163]  Felix B. Tan,et al.  User Satisfaction: An Overarching Measure of Enterprise System Success , 2005, PACIS.

[164]  Carmel de Nahlik,et al.  Looking under the bonnet: factors affecting student adoption of e-learning systems in Jordan , 2009 .

[165]  Sergio L. Toral Marín,et al.  A technological acceptance of e-learning tools used in practical and laboratory teaching, according to the European higher education area 1 , 2008, Behav. Inf. Technol..

[166]  V. Zeithaml,et al.  E-S-QUAL A Multiple-Item Scale for Assessing Electronic Service Quality , 2004 .

[167]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[168]  G. Doig A study of the factors that influence the perceptions of strategic information value in major UK based organisations , 2002 .