Empowering online teachers through predictive learning analytics

This study presents an advanced predictive learning analytics system, OU Analyse (OUA), and evidence from its evaluation with online teachers at a distance learning university. OUA is a predictive system that uses machine learning methods for the early identification of students at risk of not submitting (or failing) their next assignment. Teachers have access, via interactive dashboards, to weekly predictions of risk of failing for each of their students. In this study, we examined how the degree of OUA usage by 559 teachers, of which 189 were given access to OUA, related to student learning outcomes of more than 14 000 students in 15 undergraduate courses. Teachers who made “average” use of OUA, that is accessed OUA throughout the life cycle of a course presentation, and in particular between 10% and 40% of the weeks a course was running, and intervened with students flagged as at risk were found to benefit their students the most; after controlling for differences in academic performance, these students were found to have significantly better performance than their peers in the previous year's course presentation during which the same teachers made no use of predictive learning analytics. Predictive learning analytics is an innovative student's support approach in online pedagogy that, as shown in this study, can empower online teachers in effectively monitoring and intervening with their students, over and above other approaches, and result in improved learning outcomes.

[1]  Marcelo Worsley,et al.  Leveraging multimodal learning analytics to differentiate student learning strategies , 2015, LAK.

[2]  M. J. Edwards,et al.  The making of an exemplary online educator , 2011 .

[3]  A. I. Starčič,et al.  Teachers' perception of data-driven school ecosystem and data analytics , 2019, IC4E '19.

[4]  Olga Viberg,et al.  The current landscape of learning analytics in higher education , 2018, Comput. Hum. Behav..

[5]  Yishay Mor,et al.  Supporting teachers in data-informed educational design , 2015, Br. J. Educ. Technol..

[6]  Anouschka van Leeuwen,et al.  Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL , 2014, Comput. Educ..

[7]  Maren Scheffel,et al.  Widget, Widget on the Wall, Am I Performing Well at All? , 2017, IEEE Transactions on Learning Technologies.

[8]  Marek Hatala,et al.  Penetrating the black box of time-on-task estimation , 2015, LAK.

[9]  Zdenek Zdráhal,et al.  Ouroboros: early identification of at-risk students without models based on legacy data , 2017, LAK.

[10]  Tina Papathoma.  MOOC Educators: Who They Are And How They Learn , 2019 .

[11]  Z. Zdráhal,et al.  A large-scale implementation of predictive learning analytics in higher education: the teachers’ role and perspective , 2019, Educational Technology Research and Development.

[12]  Steve Walker,et al.  How do tutors use data to support their students? , 2019, Open Learning: The Journal of Open, Distance and e-Learning.

[13]  M. G. Sanmamed,et al.  Pedagogical Roles and Competencies of University Teachers Practicing in the E-Learning Environment , 2013 .

[14]  Bart Rienties,et al.  Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making , 2016, Br. J. Educ. Technol..

[15]  Brian Dorn,et al.  Examining Instructor Use of Learning Analytics , 2017, CHI Extended Abstracts.

[16]  Rianne Conijn,et al.  Predicting Student Performance from LMS Data: A Comparison of 17 Blended Courses Using Moodle LMS , 2017, IEEE Transactions on Learning Technologies.

[17]  Zdenek Zdráhal,et al.  Implementing predictive learning analytics on a large scale: the teacher's perspective , 2017, LAK.

[18]  Christothea Herodotou,et al.  Predictive Learning Analytics "At Scale": Towards Guidelines to Successful Implementation in Higher Education Based on the Case of the Open University UK. , 2019 .

[19]  Rebecca Ferguson,et al.  Research Evidence on the Use of Learning Analytics: Implications for Education Policy , 2016 .

[20]  Petrea Redmond,et al.  From face-to-face teaching to online teaching: Pedagogical transitions , 2011 .

[21]  Bruno D. Zumbo,et al.  A New Nonparametric Levene Test for Equal Variances , 2010 .

[22]  Tai-Hung Lee,et al.  Emotion recognition and communication for reducing second‐language speaking anxiety in a web‐based one‐to‐one synchronous learning environment , 2011 .

[23]  C. McLoughlin,et al.  Threshold concepts about online pedagogy for novice online teachers in higher education , 2019 .

[24]  Bart Rienties,et al.  Making Sense of Learning Analytics Dashboards: A Technology Acceptance Perspective of 95 Teachers , 2018 .

[25]  Rebecca Ferguson,et al.  Where is the evidence?: a call to action for learning analytics , 2017, LAK.

[26]  W. Thurston.  The evidence , 1985 .

[27]  Rebecca Ferguson,et al.  Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates , 2017, Comput. Hum. Behav..

[28]  Jason Dockter,et al.  The Problem of Teaching Presence in Transactional Theories of Distance Education , 2016 .

[29]  William Sugar,et al.  Researcher and Researched: The Phenomenology of Change from Face-to-Face to Online Instruction. , 2009 .

[30]  Anouschka van Leeuwen,et al.  Teacher regulation of cognitive activities during student collaboration: Effects of learning analytics , 2015, Comput. Educ..

[31]  Abelardo Pardo,et al.  From Study Tactics to Learning Strategies: An Analytical Method for Extracting Interpretable Representations , 2019, IEEE Transactions on Learning Technologies.

[32]  Heinz Ulrich Hoppe,et al.  Analysing content and patterns of interaction for improving the learning design of networked learning environments , 2015, Br. J. Educ. Technol..

[33]  Janet Mei-Chuen Lin,et al.  Pedagogy * technology: A two-dimensional model for teachers' ICT integration , 2012, Br. J. Educ. Technol..

[34]  Mar Pérez-Sanagustín,et al.  Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses , 2017, Comput. Educ..