Implementing learning analytics for learning impact: Taking tools to task

Abstract Learning analytics has the potential to impact student learning, at scale. Embedded in that claim are a set of assumptions and tensions around the nature of scale, impact on student learning, and the scope of infrastructure encompassed by ‘learning analytics’ as a socio-technical field. Drawing on our design experience of developing learning analytics and inducting others into its use, we present a model that we have used to address five key challenges we have encountered. In developing this model, we recommend: A focus on impact on learning through augmentation of existing practice; the centrality of tasks in implementing learning analytics for impact on learning; the commensurate centrality of learning in evaluating learning analytics; inclusion of co-design approaches in implementing learning analytics across sites; and an attention to both social and technical infrastructure.

[1]  Simon Knight,et al.  Analytics-enabled teaching as design: reconceptualisation and call for research , 2018, LAK.

[2]  Simon Buckingham Shum,et al.  Reflective writing analytics for actionable feedback , 2017, LAK.

[3]  Hendrik Drachsler,et al.  Cooking with learning analytics recipes , 2017, LAK.

[4]  Kate Thompson,et al.  Understanding the mediating role of teacher inquiry when connecting learning analytics with design for learning , 2017, IxD&A.

[5]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..

[6]  Maren Scheffel,et al.  Quality Indicators for Learning Analytics , 2014, J. Educ. Technol. Soc..

[7]  George Siemens,et al.  Current state and future trends: a citation network analysis of the learning analytics field , 2014, LAK.

[8]  Jon Mason,et al.  Data Sharing for Learning Analytics - Questioning the Risks and Benefits , 2015, ICCE 2015.

[9]  Linda Corrin,et al.  A conceptual framework linking learning design with learning analytics , 2016, LAK.

[10]  Erik Duval,et al.  Technology Enhanced Learning , 2017 .

[11]  Laura K. Allen,et al.  Writing analytics literacy: bridging from research to practice , 2017, LAK.

[12]  Carlos Delgado Kloos,et al.  Stepping out of the box: towards analytics outside the learning management system , 2011, LAK.

[13]  Peter Goodyear,et al.  Pedagogical frameworks and action research in open and distance learning , 1999 .

[14]  Alyssa Friend Wise,et al.  Designing pedagogical interventions to support student use of learning analytics , 2014, LAK.

[15]  James E. Willis,et al.  An Overview of Learning Analytics , 2017 .

[16]  Shane Dawson,et al.  Informing Pedagogical Action , 2013 .

[17]  Xiaolong Wang,et al.  Designing Academic Writing Analytics for Civil Law Student Self-Assessment , 2018, International Journal of Artificial Intelligence in Education.

[18]  Aleksandra Pawlik,et al.  Data Carpentry: Workshops to Increase Data Literacy for Researchers , 2015 .

[19]  Rebecca Ferguson,et al.  Where is the evidence?: a call to action for learning analytics , 2017, LAK.

[20]  Guillaume Durand,et al.  A Critical Perspective on Learning Analytics and Educational Data Mining , 2017 .

[21]  Simon Knight Augmenting Assessment with Learning Analytics , 2020 .

[22]  Stefaan Ternier,et al.  The Proof of the Pudding: Examining Validity and Reliability of the Evaluation Framework for Learning Analytics , 2017, EC-TEL.

[23]  Simon Buckingham Shum,et al.  Theory and Learning Analytics , 2017 .

[24]  Mandy Lupton,et al.  Learning analytics beyond the LMS: the connected learning analytics toolkit , 2015, LAK.

[25]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[26]  Bart Rienties,et al.  Implementing a Learning Analytics Intervention and Evaluation Framework: what works? , 2017 .

[27]  Simon Buckingham Shum,et al.  AcaWriter: A Learning Analytics Tool for Formative Feedback on Academic Writing , 2020, Journal of Writing Research.

[28]  Xavier Ochoa,et al.  When Are Learning Analytics Ready and What Are They Ready For , 2018, J. Learn. Anal..

[29]  Peter Goodyear,et al.  Using pattern languages to mediate theory-praxis conversations in design for networked learning , 2006 .

[30]  Stephen G. Sheldon,et al.  Conditions for Classroom Technology Innovations , 2002, Teachers College Record: The Voice of Scholarship in Education.

[31]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[32]  Olga Viberg,et al.  The current landscape of learning analytics in higher education , 2018, Comput. Hum. Behav..

[33]  Charles Lang,et al.  The pragmatic maxim as learning analytics research method , 2018, LAK.

[34]  Simon Buckingham Shum,et al.  Contextualizable Learning Analytics Design: A Generic Model and Writing Analytics Evaluations , 2019, LAK.

[35]  William A. Sandoval,et al.  Conjecture Mapping: An Approach to Systematic Educational Design Research , 2014 .

[36]  Zaidatun Tasir,et al.  A Systematic Review of Learning Analytics Intervention Contributing to Student Success in Online Learning , 2017, 2017 International Conference on Learning and Teaching in Computing and Engineering (LaTICE).

[37]  Ryan Shaun Joazeiro de Baker,et al.  Stupid Tutoring Systems, Intelligent Humans , 2016, International Journal of Artificial Intelligence in Education.

[38]  D. Boud Sustainable Assessment: Rethinking assessment for the learning society , 2000 .

[39]  Allan Collins,et al.  Design Research: Theoretical and Methodological Issues , 2004 .

[40]  W. Penuel,et al.  Organizing Research and Development at the Intersection of Learning, Implementation, and Design , 2011 .

[41]  Ran Liu,et al.  Closing the loop: Automated data-driven cognitive model discoveries lead to improved instruction and learning , 2017, EDM.

[42]  Linda Corrin,et al.  Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement , 2015 .

[43]  Rebecca Ferguson,et al.  Setting learning analytics in context: overcoming the barriers to large-scale adoption , 2014, LAK '14.

[44]  Simon Buckingham Shum,et al.  Embracing imperfection in learning analytics , 2018, LAK.

[45]  A. Wise,et al.  Why Theory Matters More than Ever in the Age of Big Data , 2015, J. Learn. Anal..

[46]  Michail N. Giannakos,et al.  Learning Analytics for Learning Design: A Systematic Literature Review of Analytics-Driven Design to Enhance Learning , 2019, IEEE Transactions on Learning Technologies.

[47]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[48]  H. Rangwala,et al.  Learning Analytics in Higher Education , 2021, Advances in Electronic Government, Digital Divide, and Regional Development.