Evaluating Complex Online Technology-enabled Course Delivery: A Contextualized View of a Decomposed IS Success Model

In this paper, we focus on understanding the factors that influence whether online courses that involve complex technologies succeed. In particular, we conducted a study on SAP software as the complex technology that students learned online via several course-management platforms (e.g., Blackboard). We hypothesized the antecedent variables system quality, information quality, and service quality to influence students’ perceived learning outcomes, satisfaction, and intention to continue using online learning. Grounded on the information systems (IS) success model, we decomposed core constructs into contextual factors. We surveyed business students from four mid-sized state universities in the United States that had membership in the SAP university alliances program, and the students had taken at least one online SAP-enabled course. We used structural equation modeling with partial least squares (PLSSEM) to analyze data. The findings indicate that system quality, information quality, and service (instructor) quality were all significant antecedents of student satisfaction, system quality and information quality were significant antecedents of perceived learning outcomes, and only system quality was a significant antecedent of students’ intention to continue using online learning.

[1]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[2]  A. Bandura.  Self-efficacy mechanism in human agency. , 1982 .

[3]  A. Bandura.  The Explanatory and Predictive Scope of Self-Efficacy Theory , 1986 .

[4]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[5]  I. Ajzen.  The theory of planned behavior , 1991 .

[6]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[7]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[8]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[9]  Tor Guimaraes,et al.  Client/Server System Success: Exploring the Human Side , 1997 .

[10]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[11]  Abhijit Gopal,et al.  Understanding GDSS in Symbolic Context: Shifting the Focus from Technology to Interaction , 2000, MIS Q..

[12]  Detmar W. Straub,et al.  Structural Equation Modeling and Regression: Guidelines for Research Practice , 2000, Commun. Assoc. Inf. Syst..

[13]  Wanda J. Orlikowski,et al.  Research Commentary: Desperately Seeking the "IT" in IT Research - A Call to Theorizing the IT Artifact , 2001, Inf. Syst. Res..

[14]  Izak Benbasat,et al.  The Identity Crisis Within the IS Discipline: Defining and Communicating the Discipline's Core Properties , 2003, MIS Q..

[15]  Richard Baskerville,et al.  Generalizing Generalizability in Information Systems Research , 2003, Inf. Syst. Res..

[16]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[17]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[18]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[19]  Izak Benbasat,et al.  Trust In and Adoption of Online Recommendation Agents , 2005, J. Assoc. Inf. Syst..

[20]  Tamara Babaian,et al.  Identifying Usability Issues with an ERP Implementation , 2005, ICEIS.

[21]  Sean B. Eom,et al.  The Determinants of Students' Perceived Learning Outcomes and Satisfaction in University Online Education: An Empirical Investigation* , 2006 .

[22]  Yi-Shun Wang,et al.  Measuring e-learning systems success in an organizational context: Scale development and validation , 2007, Comput. Hum. Behav..

[23]  Hsiu-Fen Lin,et al.  Measuring Online Learning Systems Success: Applying the Updated DeLone and McLean Model , 2007, Cyberpsychology Behav. Soc. Netw..

[24]  Yi-Shun Wang.  Assessing e‐commerce systems success: a respecification and validation of the DeLone and McLean model of IS success , 2008, Inf. Syst. J..

[25]  Viswanath Venkatesh,et al.  Technology Acceptance Model 3 and a Research Agenda on Interventions , 2008, Decis. Sci..

[26]  Ephraim R. McLean,et al.  Measuring information systems success: models, dimensions, measures, and interrelationships , 2008, Eur. J. Inf. Syst..

[27]  Brayden G. King,et al.  The Practice of Theory Borrowing in Organizational Studies: Current Issues and Future Directions , 2009 .

[28]  D. Whetten.  An Examination of the Interface between Context and Theory Applied to the Study of Chinese Organizations , 2009, Management and Organization Review.

[29]  Gaby Odekerken-Schröder,et al.  Using PLS path modeling for assessing hierarchial construct models: guidelines and impirical illustration , 2009 .

[30]  Jung Wan Lee.  Online support service quality, online learning acceptance, and student satisfaction , 2010, Internet High. Educ..

[31]  Khaled A. Alshare,et al.  Predicting Student-Perceived Learning Outcomes and Satisfaction in ERP Courses: An Empirical Investigation , 2011, Commun. Assoc. Inf. Syst..

[32]  Hei-Chia Wang,et al.  Assessing e-learning 2.0 system success , 2011, Comput. Educ..

[33]  Sean B. Eom,et al.  Relationships among e-learning systems and e-learning outcomes: A path analysis model , 2011 .

[34]  Marko Sarstedt,et al.  PLS-SEM: Indeed a Silver Bullet , 2011 .

[35]  Nils Urbach,et al.  The Updated DeLone and McLean Model of Information Systems Success , 2012 .

[36]  Silvana Trimi,et al.  The Impact of CMS Quality on the Outcomes of E‐learning Systems in Higher Education: An Empirical Study , 2012 .

[37]  Zetian Fu,et al.  An empirical study on behavioural intention to reuse e-learning systems in rural China , 2012, Br. J. Educ. Technol..

[38]  Teoh Sian Hoon,et al.  System Quality and its Influence on Students’ Learning Satisfaction in UiTM Shah Alam , 2013 .

[39]  Carol Jordan,et al.  Comparison of International Baccalaureate (IB) chemistry students’ preferred vs actual experience with a constructivist style of learning in a Moodle e‐learning environment , 2013 .

[40]  Victor Chang,et al.  A University of Greenwich Case Study of Cloud Computing: Education as a Service , 2013 .

[41]  Marko Sarstedt,et al.  Partial least squares structural equation modeling (PLS-SEM): An emerging tool in business research , 2014 .

[42]  Kweku-Muata Osei-Bryson,et al.  Examining influence of national culture on individuals' attitude and use of information and communication technology: Assessment of moderating effect of culture through cross countries study , 2013, Int. J. Inf. Manag..

[43]  A. K. M. Najmul Islam,et al.  Investigating e-learning system usage outcomes in the university context , 2013, Comput. Educ..

[44]  Rudolf R. Sinkovics,et al.  A Critical Look at the Use of SEM in International Business Research , 2014 .

[45]  Gurpreet Dhillon,et al.  A Framework and Guidelines for Context-Specific Theorizing in Information Systems Research , 2014, Inf. Syst. Res..

[46]  Saeid Nahavandi,et al.  The impact of self-efficacy and perceived system efficacy on effectiveness of virtual training systems , 2014, Behav. Inf. Technol..

[47]  Hossein Mohammadi,et al.  Investigating users' perspectives on e-learning: An integration of TAM and IS success model , 2015, Comput. Hum. Behav..

[48]  Anthony C. Keys,et al.  How Does ERPsim Influence Students' Perceived Learning Outcomes in an Information Systems Course? An Empirical Study , 2015, J. Inf. Syst. Educ..

[49]  Fatemeh Zahedi,et al.  Fake-Website Detection Tools: Identifying Elements that Promote Individuals' Use and Enhance Their Performance , 2015, J. Assoc. Inf. Syst..

[50]  Victor Chang,et al.  Review and discussion: E-learning for academia and industry , 2016, Int. J. Inf. Manag..

[51]  Edward E. Rigdon,et al.  On Comparing Results from CB-SEM and PLS-SEM: Five Perspectives and Five Recommendations , 2017 .

[52]  Heikki Topi,et al.  A Conceptual Model for User-system Collaboration: Enhancing Usability of Complex Information Systems , 2017, Commun. Assoc. Inf. Syst..

[53]  Changyong Liang,et al.  Influence of mechanism of patient-accessible hospital information system implementation on doctor-patient relationships: A service fairness perspective , 2017, Inf. Manag..

[54]  Rodina Ahmad,et al.  A sustainable quality assessment model for the information delivery in E-learning systems , 2017 .

[55]  Sean B. Eom,et al.  A System's View of E‐Learning Success Model , 2018 .

[56]  M. Mital,et al.  Adoption of Internet of Things in India: A test of competing models using a structured equation modeling approach , 2017, Technological Forecasting and Social Change.

[57]  Dheeraj Sharma,et al.  Online learning: Adoption, continuance, and learning outcome - A review of literature , 2018, Int. J. Inf. Manag..

[58]  Heikki Topi,et al.  Invited Paper: Reflections on the Current State and Future of Information Systems Education , 2019, J. Inf. Syst. Educ..