Measuring e-learning systems success in an organizational context: Scale development and validation

Electronic learning (e-learning) has been widely adopted as a promising solution by many companies to offer learning-on-demand opportunities to individual employees in order to reduce training time and cost. While information systems (IS) success models have received much attention among researchers, little research has been conducted to assess the success and/or effectiveness of e-learning systems in an organizational context. Whether traditional IS success models can be extended to investigate e-learning systems success has been scarcely addressed. Based on previous IS success literature, this study developed and validated a multi-dimensional model for assessing e-learning systems success (ELSS) from the perspective of the employee (e-learner). The procedures used in conceptualizing an ELSS construct, generating items, collecting data, and validating a multiple-item scale for measuring ELSS are described. This paper presents evidence of the scale's factor structure, reliability, content validity, criterion-related validity, convergent validity, and discriminant validity on the basis of analyzing data from a sample of 206 respondents. Theoretical and managerial implications of our results are discussed. This empirically validated instrument will be useful to researchers in developing and testing e-learning systems theories, as well as to organizations in implementing successful e-learning systems.

[1]  Richard T. Vidgen,et al.  WebQual: An Exploration of Web-Site Quality , 2000, ECIS.

[2]  Yi-Shun Wang,et al.  The conceptualization and measurement of m-commerce user satisfaction , 2004, Comput. Hum. Behav..

[3]  Gilbert A. Churchill,et al.  Marketing Research: Methodological Foundations , 1976 .

[4]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[5]  J. Nunnally Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[6]  Chang Liu,et al.  Exploring the factors associated with Web site success in the context of electronic commerce , 2000, Inf. Manag..

[7]  湯宗益 Assessing Customer Perceptions of Web Site Service Quality in Digital Marketing Environments , 2003 .

[8]  Linda Carswell Teaching via the Internet: the impact of the Internet as a communication medium on distance learning introductory computing students , 1997, ITiCSE '97.

[9]  Yi-Shun Wang,et al.  Assessment of learner satisfaction with asynchronous electronic learning systems , 2003, Inf. Manag..

[10]  Jane E. Klobas,et al.  User Developed Applications and Information Systems Success: A Test of DeLone and McLean's Model , 2003, Inf. Resour. Manag. J..

[11]  Ali F. Farhoomand,et al.  A structural model of end user computing satisfaction and user performance , 1996, Inf. Manag..

[12]  A. Adam Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[13]  A. Lederer,et al.  An Instrument for Assessing the Organizational Benefits of IS Projects , 1998 .

[14]  Starr Roxanne Hiltz,et al.  Asynchronous learning networks as a virtual classroom , 1997, CACM.

[15]  Yi-Shun Wang,et al.  An Instrument for Measuring Customer Satisfaction Toward Web Sites That Market Digital Products and Services , 2001, J. Electron. Commer. Res..

[16]  Charles E. Downing,et al.  System usage behavior as a proxy for user satisfaction: an empirical investigation , 1999, Inf. Manag..

[17]  J. Hair Multivariate data analysis , 1972 .

[18]  Albert H. Segars,et al.  Strategic Information Systems Planning Success: An Investigation of the Construct and Its Measurement , 1998, MIS Q..

[19]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[20]  Peter B. Seddon A Respecification and Extension of the DeLone and McLean Model of IS Success , 1997, Inf. Syst. Res..

[21]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[22]  Maarten Gelderman,et al.  The relation between user satisfaction, usage of information systems and performance , 1998, Inf. Manag..

[23]  Leon A. Kappelman,et al.  A Comprehensive Model for Assessing the Quality and Productivity of the Information Systems Function: Toward a Theory for Information Systems Assessment , 1997 .

[24]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[25]  William J. Doll,et al.  Developing a multidimensional measure of system-use in an organizational context , 1998, Inf. Manag..

[26]  Dennis F. Galletta,et al.  Some Cautions on the Measurement of User Information Satisfaction , 1989 .

[27]  Frank F. Land,et al.  Moving IS evaluation forward: learning themes and research issues , 1999, J. Strateg. Inf. Syst..

[28]  James C. Anderson,et al.  STRUCTURAL EQUATION MODELING IN PRACTICE: A REVIEW AND RECOMMENDED TWO-STEP APPROACH , 1988 .

[29]  Tor Guimaraes,et al.  Client/Server System Success: Exploring the Human Side , 1997 .

[30]  Magid Igbaria,et al.  Personal Computing Acceptance Factors in Small Firms: A Structural Equation Model , 1997, MIS Q..

[31]  Ingoo Han,et al.  Performance measure of information systems (IS) in evolving computing environments: an empirical investigation , 2003, Inf. Manag..

[32]  W. Doll,et al.  The development of a tool for measuring the perceived impact of information technology on work , 1999 .

[33]  Mayuram S. Krishnan,et al.  Evaluating the cost of software quality , 1998, CACM.

[34]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[35]  David Heath,et al.  THE CONGRESS Of THE UNITED STATES. , 1906, Science.

[36]  Timo Saarinen,et al.  An expanded instrument for evaluating information system success , 1996, Inf. Manag..

[37]  Prashant C. Palvia,et al.  Developing and validating an instrument for measuring user-perceived web quality , 2002, Inf. Manag..

[38]  Carol Stoak Saunders,et al.  Measuring Performance of the Information Systems Function , 1992, J. Manag. Inf. Syst..

[39]  Fatemeh Zahedi,et al.  The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach , 2002, Inf. Syst. Res..

[40]  R. Sitgreaves Psychometric theory (2nd ed.). , 1979 .

[41]  J. L. Price,et al.  Handbook of Organizational Measurement , 1975 .

[42]  Prashant C. Palvia,et al.  A model and instrument for measuring small business user satisfaction with information technology , 1996, Inf. Manag..

[43]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[44]  Gilbert A. Churchill A Paradigm for Developing Better Measures of Marketing Constructs , 1979 .

[45]  Peter B. Seddon,et al.  A Partial Test and Development of the DeLone and McLean Model of IS Success , 1994, ICIS.

[46]  Margaret Tan,et al.  The consequences of information technology acceptance on subsequent individual performance , 1997, Inf. Manag..

[47]  Yi-Shun Wang,et al.  Assessing Customer Perceptions of Website Service Quality in Digital Marketing Environments , 2003, J. Organ. End User Comput..

[48]  Kristi Yuthas,et al.  Material matters: Assessing the effectiveness of materials management IS , 1998, Inf. Manag..

[49]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[50]  Eldon Y. Li Perceived importance of information system success factors: A meta analysis of group differences , 1997, Inf. Manag..

[51]  Arun Rai,et al.  Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis , 2002, Inf. Syst. Res..

[52]  Jaak Jurison,et al.  The temporal nature of IS benefits: A longitudinal study , 1996, Inf. Manag..

[53]  Erik Brynjolfsson,et al.  The productivity paradox of information technology , 1993, CACM.

[54]  D. Sandy Staples,et al.  Dimensions of Information Systems Success , 1999, Commun. Assoc. Inf. Syst..

[55]  David F. McAllister,et al.  Providing education electronically to nontraditional sites: new delivery to a new audience , 1996, SIGDOC '96.

[56]  John F. Rockart,et al.  The management of end user computing , 1983, CACM.

[57]  Jay F. Nunamaker,et al.  Powering E-Learning In the New Millennium: An Overview of E-Learning and Enabling Technology , 2003, Inf. Syst. Frontiers.

[58]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[59]  Steve Muylle,et al.  The conceptualization and empirical validation of web site user satisfaction , 2004, Inf. Manag..

[60]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[61]  William J. Kettinger,et al.  Perceived Service Quality and User Satisfaction with the Information Services Function , 1994 .

[62]  Lei-da Chen,et al.  Measuring user satisfaction with data warehouses: an exploratory study , 2000, Inf. Manag..

[63]  Brian Cameron,et al.  “But what did we learn…?”: evaluating online learning as process , 1998, SIGDOC '98.

[64]  Darshana Sedera,et al.  Enterprise Systems Success: A Measurement Model , 2003, ICIS.

[65]  Kathryn A. Marold,et al.  Web-based learning: is it working? A comparison of student performance and achievement in web-based courses and their in-classroom counterparts , 2000, IRMA Conference.

[66]  Peter A. Todd,et al.  Understanding Information Technology Usage: A Test of Competing Models , 1995, Inf. Syst. Res..

[67]  Lorin M. Hitt,et al.  Productivity, Business Profitability, and Consumer Surplus: Three Different Measures of Information Technology Value , 1996, MIS Q..