Is Usage a Missing Link in Explaining the Perceived Learning Outcome of Technology-Mediated Learning?

Information systems (IS) researchers have demonstrated that usage is a key variable in explaining the performance impact of information technology. However, existing technology-mediated learning (TML) studies have not examined the influence of usage on learning outcome and the factors that determine the usage of TML. To address this research gap, our study presents and tests a TML model by drawing insights from two research streams. First, following the IS literature, we incorporate the impact of technology usage on individual performance. Second, building on the social cognitive theory, we study the influences of self-efficacy beliefs (system and subject domain) and affective responses (affect and anxiety) on technology usage. Based on 503 matched responses collected using two-stage questionnaire surveys, our analyses confirm the significance of usage in mediating the effects of system self-efficacy and anxiety on perceived learning outcome, but not in mediating the effects of affect and subject-domain self-efficacy. We find strong support for the influences of self-efficacy beliefs on affective responses. Self-efficacy beliefs of the users are also observed to change over time and perceived learning outcome plays a significant role in explaining this change. Our research enhances the existing TML theory by producing useful insights regarding the influence of social cognitive factors of learners on the usage of TML and how usage mediates the influence of these variables on perceived learning outcome.

[1]  Sirkka L. Jarvenpaa,et al.  The Use of Information Technology to Enhance Management School Education: A Theoretical View , 1995, MIS Q..

[2]  Fadi P. Deek,et al.  Innovation and education in the digital age: reconciling the roles of pedagogy, technology, and the business of learning , 2003, IEEE Trans. Engineering Management.

[3]  Fred D. Davis,et al.  A Model of the Antecedents of Perceived Ease of Use: Development and Test† , 1996 .

[4]  S. Jarvenpaa,et al.  Will the Internet revolutionize business education and research? Sloan Management Review , 1996 .

[5]  Susan A. Brown,et al.  E-Learning and Individual Characteristics: The Role of Computer Anxiety and Communication Apprehension , 2006, J. Comput. Inf. Syst..

[6]  Grandon Gill,et al.  Teaching with the Case Method Online: Pure versus Hybrid Approaches. , 2005 .

[7]  Michael S. Cole,et al.  Student Learning Motivation and Psychological Hardiness: Interactive Effects on Students' Reactions to a Management Class , 2004 .

[8]  Feng-yang Kuo,et al.  The More We Study, the Less We Learn: A Primer on the Analysis of TML Effectiveness , 2006, J. Organ. Comput. Electron. Commer..

[9]  M. Lynne Markus,et al.  If we build it, they will come: Designing information systems that people want to use , 1994 .

[10]  Mark J. Brosnan,et al.  The impact of computer anxiety and self-efficacy upon performance , 1998, J. Comput. Assist. Learn..

[11]  Hans van der Heijden,et al.  User Acceptance of Hedonic Information Systems , 2004, MIS Q..

[12]  William J. Doll,et al.  Developing a multidimensional measure of system-use in an organizational context , 1998, Inf. Manag..

[13]  David F. Larcker,et al.  Structural Equation Models with Unobservable Variables and Measurement Error: Algebra and Statistics: , 1981 .

[14]  Younghwa Lee,et al.  The Technology Acceptance Model: Past, Present, and Future , 2003, Commun. Assoc. Inf. Syst..

[15]  Mun Y. Yi,et al.  Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model , 2003, Int. J. Hum. Comput. Stud..

[16]  Viswanath Venkatesh,et al.  Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model , 2000, Inf. Syst. Res..

[17]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[18]  Martha Reeves,et al.  Evaluation of Training , 1993 .

[19]  Marcia J. Simmering,et al.  Conscientiousness, goal orientation, and motivation to learn during the learning process: A longitudinal study. , 1998 .

[20]  Louis A. Le Blanc,et al.  An Empirical Investigation of the Relationship Between DSS Usage and System Performance: A Case Study of a Navigation Support System , 1990, MIS Q..

[21]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[22]  Robert W. Stone,et al.  The Roles of Computer Self-Efficacy and Outcome Expectancy in Influencing the Computer End-User's Organizational Commitment , 2003, J. Organ. End User Comput..

[23]  J. B. Arbaugh,et al.  An Investigation of Epistemological and Social Dimensions of Teaching in Online Learning Environments , 2006 .

[24]  John T. E. Richardson,et al.  Researching Student Learning: Approaches to Studying in Campus-Based and Distance Education , 2000 .

[25]  Sean B. Eom,et al.  The Determinants of Students' Perceived Learning Outcomes and Satisfaction in University Online Education: An Empirical Investigation* , 2006 .

[26]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[27]  W. Neuman,et al.  Social Research Methods: Qualitative and Quantitative Approaches , 2002 .

[28]  Deborah Compeau,et al.  Computer Self-Efficacy: Development of a Measure and Initial Test , 1995, MIS Q..

[29]  J. Colquitt,et al.  Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research. , 2000, The Journal of applied psychology.

[30]  Peter G. W. Keen,et al.  Decision support systems : an organizational perspective , 1978 .

[31]  A. Bandura.  The self system in reciprocal determinism. , 1978 .

[32]  Charles Chowa,et al.  Information System Success: Individual and Organizational Determinants , 2006, Manag. Sci..

[33]  Joseph J. Martocchio,et al.  EFFECTS OF FEEDBACK AND COGNITIVE PLAYFULNESS ON PERFORMANCE IN MICROCOMPUTER SOFTWARE TRAINING , 2006 .

[34]  Viswanath Venkatesh,et al.  Why Don't Men Ever Stop to Ask for Directions? Gender, Social Influence, and Their Role in Technology Acceptance and Usage Behavior , 2000, MIS Q..

[35]  J. Keller.  Development and use of the ARCS model of instructional design , 1987 .

[36]  A. Isen,et al.  Toward understanding the role of affect in cognition. , 1984 .

[37]  L. Sjodahl.  Evaluation of Training. , 1975 .

[38]  Fred D. Davis,et al.  Developing and Validating an Observational Learning Model of Computer Software Training and Skill Acquisition , 2003, Inf. Syst. Res..

[39]  S. Chaiken,et al.  Attitude structure and function. , 1998 .

[40]  Chelley Vician,et al.  Investigating Computer Anxiety and Communication Apprehension as Performance Antecedents in a Computing-Intensive Learning Environment , 2003, J. Comput. Inf. Syst..

[41]  Gilad Chen,et al.  Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. , 2000, The Journal of applied psychology.

[42]  Susan A. Brown,et al.  Who's Afraid of the Virtual World? Anxiety and Computer-Mediated Communication , 2004, J. Assoc. Inf. Syst..

[43]  J. Mathieu,et al.  Meeting trainees' expectations: The influence of training fulfillment on the development of commitment, self-efficacy, and motivation. , 1991 .

[44]  Rajiv Kohli,et al.  Contribution of institutional DSS to organizational performance: evidence from a longitudinal study , 2004, Decis. Support Syst..

[45]  Rajiv Kohli,et al.  Performance Impacts of Information Technology: Is Actual Usage the Missing Link? , 2003, Manag. Sci..

[46]  S. Sugahara,et al.  The Effectiveness of PowerPoint presentations in the Accounting Classroom , 2006 .

[47]  Lorne Olfman,et al.  The Importance of Learning Style in End-User Training , 1990, MIS Q..

[48]  J. Webster,et al.  Teaching Effectiveness in Technology-Mediated Distance Learning , 1997 .

[49]  Jack Arthur Gowan,et al.  Video conferencing human-machine interface: A field study , 1994, Inf. Manag..

[50]  A Bandura,et al.  Cognitive processes mediating behavioral change. , 1977, Journal of personality and social psychology.

[51]  Wynne W. Chin.  The partial least squares approach for structural equation modeling. , 1998 .

[52]  Dorothy E. Leidner,et al.  Research Commentary: Technology-Mediated Learning - A Call for Greater Depth and Breadth of Research , 2001, Inf. Syst. Res..

[53]  Wanda J. Orlikowski,et al.  Technological frames: making sense of information technology in organizations , 1994, TOIS.

[54]  A. Romiszowski,et al.  Computer-mediated communication , 2011 .

[55]  Detmar W. Straub,et al.  Measuring System Usage: Implications for IS Theory Testing , 1995 .

[56]  James W. Westerman,et al.  A Cross-Cultural Empirical Analysis of Person-Organization Fit Measures as Predictors of Student Performance in Business Education: Comparing Students in the United States and India , 2005 .

[57]  Yan Li,et al.  Senior Citizens' Acceptance of Information Systems: A Study in the Context of e-Government Services , 2006, IEEE Transactions on Engineering Management.

[58]  Mani Subramani,et al.  How Do Suppliers Benefit from Information Technology Use in Supply Chain Relationships? , 2004, MIS Q..

[59]  Juan-Carlos Cano,et al.  Webgene$_{\rm OS}$: A Generative and Web-Based Learning Architecture to Teach Operating Systems in Undergraduate Courses , 2006, IEEE Transactions on Education.

[60]  Vallabh Sambamurthy,et al.  Research Report: The Evolving Relationship Between General and Specific Computer Self-Efficacy - An Empirical Assessment , 2000, Inf. Syst. Res..

[61]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[62]  Sirkka L. Jarvenpaa,et al.  The Information Age Confronts Education: Case Studies on Electronic Classrooms , 1993, Inf. Syst. Res..

[63]  P. Ackerman.  Individual differences in skill learning: An integration of psychometric and information processing perspectives. , 1987 .

[64]  Mohamed Khalifa,et al.  Web-based learning: effects on learning process and outcome , 2002, IEEE Trans. Educ..

[65]  Daniel J. Brass,et al.  Efficacy-Performing Spirals: A Multilevel Perspective , 1995 .

[66]  Nancy E. Betz,et al.  The Relationship of Career-Related Self-Efficacy Expectations to Perceived Career Options in College Women and Men. , 1981 .

[67]  Robert G. Fuller,et al.  The Air Force Academy Instructor Workstation (IWS): II. Effectiveness , 1989 .

[68]  J. Hair.  Multivariate data analysis , 1972 .

[69]  Anol Bhattacherjee,et al.  Understanding Changes in Belief and Attitude Toward Information Technology Usage: A Theoretical Model and Longitudinal Test , 2004, MIS Q..

[70]  Maryam Alavi,et al.  Using Information Technology in Learning: Case Studies in Business and Management Education Programs , 2003 .

[71]  Deborah Compeau,et al.  Social Cognitive Theory and Individual Reactions to Computing Technology: A Longitudinal Study , 1999, MIS Q..

[72]  Magid Igbaria,et al.  Computer anxiety and attitudes towards microcomputer use , 1990 .

[73]  Robert P. Bostrom,et al.  Training End Users: An Experimental Investigation of the Roles of the Computer Interface and Training Methods , 1993, MIS Q..

[74]  Jack McGourty.  Using multisource feedback in the classroom: a computer-based approach , 2000, IEEE Trans. Educ..

[75]  Maryam Alavi,et al.  Computer-Mediated Collaborative Learning: An Empirical Evaluation , 1994, MIS Q..

[76]  D. Vogel,et al.  Using Information Technology to Add Value to Management Education , 1997 .

[77]  Robert L. Craig,et al.  Training and development handbook: A guide to human resource development , 1976 .

[78]  Vallabh Sambamurthy,et al.  Sources of Influence on Beliefs about Information Technolgoy Use: An Empirical Study of Knowledge Workers , 2003, MIS Q..

[79]  Blake Ives,et al.  Web-based Virtual Learning Environments: a Research Framework and a Preliminary Assessment of Effectiveness in Basic It Skills Training Author(s): Piccoli Et Al./web-based Virtual Learning Environments Web-based Virtual Learning Environments: a Research Framework and a Preliminary Assessment of Effe , 2022 .

[80]  Jr. Henry C. Lucas,et al.  Performance and the Use of an Information System , 1975 .

[81]  E. Salas,et al.  Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. , 1993 .

[82]  Daniel J. Brass,et al.  Efficacy-performance spirals: A multilevel perspective. , 1995 .

[83]  David Kember,et al.  Learning approaches, study time and academic performance , 1995 .

[84]  Jay F. Nunamaker,et al.  Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness , 2006, Inf. Manag..