Learning Analytics implementations in universities: Towards a model of success, using multiple case studies

n these pioneering days of Learning Analytics in higher education, universities are pursuing a diverse range of in-house implementation strategies, with varying degrees of success. In this exploratory study we compare and contrast the approaches taken at three demographically different Australian universities. The comparison is made in the context of Delone and McLean’s information system success model (1992). In time, a consensus-driven method for using Learning Analytics to improve student learning outcomes will eventuate, including individualized learning, but we are still some distance from this level of maturity. It seems likely that user-friendly proprietary platforms will prosper in the climate of uncertainty. Participants in the study see potential in Learning Analytics but are not sure about how best to realize that potential as the implementation of Learning Analytics systems at Australian universities are still very much in their infancy. Proprietary approaches offering sophisticated functionality seem likely to emerge and take precedence over the trial and error approach. This study addresses an apparent gap in the research as limited studies exist targeting both learning analytics and information system success. The methodology taken explores the research topic through a qualitative lense utlising thematic analysis. The study concludes that digital interventions such as Learning Analytics has great potential to optimize teaching and learning practices. Information systems success research can provide insights into what works and what does not in terms of Learning Analytics implementations. The discipline needs to be systematized for efficient implementation, and must deliver tangible benefits over time.

[1]  Geoff Walsham,et al.  The Emergence of Interpretivism in IS Research , 1995, Inf. Syst. Res..

[2]  Anthony Herrington Adult educators’ authentic use of smartphones to create digital teaching resources , 2008 .

[3]  Thanh D. Nguyen,et al.  Information Systems Success: A Literature Review , 2015, FDSE.

[4]  Philippa Gerbic,et al.  Collaborative self-study supporting new technology: The Mahara e-portfolio project , 2008 .

[5]  Jason M. Lodge,et al.  Learning analytics and teaching: a conceptual framework for translation and application , 2019 .

[6]  Ojelanki K. Ngwenyama,et al.  Contextual influences on user satisfaction with mobile computing: findings from two healthcare organizations , 2006, Eur. J. Inf. Syst..

[7]  Sakinah S. J. Alhadad,et al.  Visualizing Data to Support Judgement, Inference, and Decision Making in Learning Analytics: Insights from Cognitive Psychology and Visualization Science , 2018, J. Learn. Anal..

[8]  George Siemens,et al.  Penetrating the fog: analytics in learning and education , 2014 .

[9]  A. Leclercq The perceptual evaluation of information systems using the construct of user satisfaction: case study of a large french group , 2007, DATB.

[10]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[11]  John P. Campbell,et al.  Analytics in Higher Education: Establishing a Common Language , 2012 .

[12]  Universities Australia,et al.  A smarter Australia: an agenda for Australian higher education 2013-2016 , 2013 .

[13]  Greg Kearsley,et al.  Explorations in learning and instruction: The theory into practice database , 1994 .

[14]  Erik Brynjolfsson,et al.  Big data: the management revolution. , 2012, Harvard business review.

[15]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[16]  Andy Koronios,et al.  Big Data Project Success - a Meta Analysis , 2014, PACIS.

[17]  D MyersMichael,et al.  A set of principles for conducting and evaluating interpretive field studies in information systems , 1999 .

[18]  Kenneth David Strang,et al.  Do the Critical Success Factors From Learning Analytics Predict Student Outcomes? , 2016 .

[19]  Meg O'Reilly,et al.  Sharing quality resources for teaching and learning: A peer review model for the ALTC Exchange in Australia , 2009 .

[20]  Angelika Kokkinaki,et al.  Business Information Systems , 2014, Lecture Notes in Business Information Processing.

[21]  Cathy Gunn,et al.  A design-based research approach for eportfolio initiatives , 2008 .

[22]  S. Antwi,et al.  Qualitative and Quantitative Research Paradigms in Business Research: A Philosophical Reflection , 2015 .

[23]  Yair Levy,et al.  A Systems Approach to Conduct an Effective Literature Review in Support of Information Systems Research , 2006, Informing Sci. Int. J. an Emerg. Transdiscipl..

[24]  J. Rowley Using case studies in research , 2002 .

[25]  Hamish Coates,et al.  A model of online and general campus‐based student engagement , 2007 .

[26]  Ephraim R. McLean,et al.  Measuring information systems success: models, dimensions, measures, and interrelationships , 2008, Eur. J. Inf. Syst..

[27]  Neil F. Doherty,et al.  The Importance of User Ownership and Positive User Attitudes in the Successful Adoption of Community Information Systems , 2001, J. Organ. End User Comput..

[28]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[29]  Olaf Zawacki-Richter,et al.  Mobile Learning: Transforming the Delivery of Education and Training , 2009 .

[30]  Austin Gibbons,et al.  Learning Analytics , 2014, Encyclopedia of Social Network Analysis and Mining.

[31]  Danny Toohey,et al.  Learning Analytics: Assisting Universities with Student Retention , 2015 .

[32]  Robert K. Yin,et al.  Qualitative Research from Start to Finish , 2010 .

[33]  Linda Corrin,et al.  Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement , 2015 .

[34]  Philipa Levy "Living" Theory: A Pedagogical Framework for Process Support in Networked Learning , 2006 .